Assessment of Students ’ Higher-level Text Comprehension Skills in Basic School

Text comprehension is a sophisticated process that is influenced by the reader’s cognitive skills, prior knowledge and the type of texts. The aim of the present longitudinal study was to assess the students’ ability to analyse and evaluate the narrative text in Grades 4 and 5. A total of 831 Estonian students were tested in two consecutive years. The results indicated that fewer than half of the students succeeded in analysing and evaluating the contents of the text, while the students’ text comprehension skills in Grade 5 were significantly better than in Grade 4. Students had more difficulties answering the questions that examined their skill of evaluating the text, compared to their ability to analyse the text. This led to the conclusion that teachers should pay more attention to the students’ higher-level cognitive processes and support their text comprehension skills.


Introduction
Text comprehension can be defined as a reader's ability to find the main idea of the text and to learn from the context (Snow, 2002).To this end the readers have to construct a whole by making several inferences and integrating different phrases, sentences and arguments appearing in the text (Eason, Goldberg, Young, Geist, & Cutting, 2012;Kaplan, 2013).In addition to this, the text must be integrated to readers' prior knowledge and experiences to guarantee the successful understanding (Schoenbach, Greenleaf & Cziko, 2012).
Text comprehension has been studied for decades in the areas of psychology and education.Attention has been focused mainly on the development of students' lower-level cognitive processes and text comprehension skills, for example, word decoding speed and accuracy, the extent of vocabulary, etc. (Geva, & Farnia, 2011;Seigneuric, & Ehrlich, 2005).Students' higher-level text comprehension skills have been studied to significantly lesser degree (Basabara, Yovanoff, Alonzo, & Tindal, 2013).These skills require the students to possess the ability to make inferences and relate different types of sentences as well as make conclusions and evaluate the text.
In common with other countries, Estonian students' text comprehension skills have been assessed in international comparative studies (for example, Programme for International Student Assessment PISA) as well as in the national standardised tests.Estonian students have been found to have very good basic skills in reading but higher-level text comprehension, for example, analysing texts, finding the main idea of a text and reading tables, is difficult for many Estonian students (Pandis, 2008;Tire et al., 2013).Although, Estonian students' results in text comprehension according to the national standardized test have recently shown some improvement (Hennoste, 2013).Therefore, the present study focused on basic school students' ability to analyse and evaluate the narrative text.The purpose of the study was to examine the students' text comprehension skills and how they change from Grade 4 to Grade 5.

Levels of Text Comprehension
Readers can understand the texts on different levels (Basabara et al., 2013).Firstly, at the literal comprehension level, the readers understand the explicit information conveyed by the text (Kibui, 2012), without interpreting or associating it with readers' prior knowledge (Butcher & Kintsch, 2012).Secondly, at the inferential comprehension level readers have to analyze the text and make assumptions about the information which is not explicitly stated in the text (Basabara et al., 2013).At that level, readers connect different events, actions and arguments that occur from the text.Also, the new information is related to reader's prior knowledge at this level (Butcher & Kintsch, 2012;Eason et al., 2012).
The readers' ability to evaluate texts is at the top of the comprehension skills.For that reason, the readers discuss their ideas and feelings that texts evoke in them (Kibui, 2012).In case of evaluative comprehension readers adopt a broader based information platform, going beyond the text (Basabara et al., 2013).Readers who have more accomplished evaluative skills know better how to evaluate the content of the text, how to interpret it, make conclusions and have an opinion about the context (Kaplan, 2013;Kibui, 2012).This level is based on combining literal and inferential comprehension levels.In order to evaluate any text, readers have to understand the meaning of the text and be able to make inferences based on it (Basabara et al., 2013).Therefore, it is important for teachers, to teach first the basic skills of the text comprehension and then step by step move onto the upper levels of the text comprehension.

Changes in text comprehension skills at the basic school level
The changes in text comprehension skills can be assessed according to the extent students are able to comprehend texts.During the first school year (at the age of 7-8) texts are read mainly at the literal level (Seigneuric & Ehrlich, 2005).The older the children the wider is their knowledge, their memory capacity and the ability to use different reading strategies with different types of texts (Geva & Farnia, 2011;Kaplan, 2013;Lynch et al., 2008).It means that older students are generally better prepared to understand the text at a higher-level.
At every text comprehension level, different text comprehension components are important.For beginners, who are reading texts at the level of literal comprehension, word decoding ability and speed are the most important factors in understanding the text (Geva & Farnia, 2011).Later, when the cognitive processes (e.g.working memory, verbal skills) are more evolved and texts are being understood at a higher-level, the reader's prior knowledge, experiences, and ability to infer and interpret the information are becoming more important (Kibui, 2012;Seigneuric & Ehrlich, 2005).
Comparing the students' ability to answer the questions at different comprehension level has revealed that the most difficult task is to answer to the questions that need skills of the evaluative comprehension level (Basabara et al., 2013;Veeravagu, Muthusamy, Marimuthu & Michael, 2010).However, it has been also found that students, who understood text better at the literal comprehension level, were more likely to comprehend the texts at the higher, inferential and evaluative level (Uibu & Mannamaa, 2014;Wang, 2009).
The studies on the Estonian students' text comprehension skills have shown that very few students at the age of 15 can understand texts at the highest level (Tire et al., 2013).The basic skills of text comprehension have been mastered by almost all students.There is no problem for students from the age of 9 to answer the factual questions.Problems occur when texts have to be analyzed, evaluated and integrated with student's prior knowledge (Pandis, 2008;Sinka, 2008).By the age of 15 students should have the cognitive skills to analyse, evaluate and integrate texts (Van den Broek, Lunch, Naslund, Levers-Landis & Verduin, 2003).That is why it is important to study the students' text comprehension at the age of 10-12.Although, the highest text comprehension level does not have to be accomplished by that age, it is important to consistently support and develop skills required for being able to comprehend texts at the evaluative level.

Aims and Hypotheses
Some studies have shown that students have better skills in literal comprehension, compared to their analysing and evaluative skills (Basabara et al., 2013;Veeravagu et al., 2010).The aim of the present study was to examine Estonian students' higher-level text comprehension skills: their ability to analyse and evaluate the content of the text and the changes in comprehension skill in Grades 4 and 5.In order to achieve this aim the following objectives and hypotheses were addressed.
1. To find out the differences between the students' ability to analyse and evaluate the texts.Previous studies have shown that a students' ability to analyse the content of texts is more evolved than their evaluative skills (Basabara et al., 2013;Veeravagu et al., 2010).Therefore, we expected that in Grade 4 as well Grade 5 students have better analysing than evaluating text comprehension skills.2. To investigate changes in students' analysing and evaluating skills during two consecutive years.The development of students' text comprehension skills may improve by years (Lynch et al., 2008;Seigneuric & Ehrlich, 2005).Accordingly, we assumed that students, who have better text analysing and evaluating skills in Grade 4 have also better skills in Grade 5. 3. To examine the correlations between students' analysing and evaluating skills.Previous studies have shown that there is a significant correlation between a students' ability to analyse and evaluate texts (Wang, 2009).Therefore, we assumed that students, who are better at analysing the texts, also evaluate them better.

Sample
A total of 831 students (48.1% of boys and 51.9% of girls) from 29 Estonian schools participated in the longitudinal study.The students' average age in Grade 4 was 10.0 years, SD = .36,and in Grade 5 it was 10.98 years, SD = .37.The schools and classes were chosen to be representative of Estonia, taking into account the location and types of school, the number of classes in the school and class size.The same students were tested at two time points: in the autumn of Grades 4 and 5.All data was analysed anonymously.

Measurement and procedure
Students' language tests were used to identify the students' ability to analyse and evaluate the content of texts.Both tests for Grades 4 and 5 were compiled on the basis of levels of revised Bloom's hierarchical taxonomy (Krathwhol, 2002), curriculum requirements and national standardised tests (see Uibu & Mannamaa, 2014;Uibu & Timm, 2014).In order to examine students' analysing and evaluating skills two at different reading level tasks were compiled.First, students were asked to read out the narrative text, consisting of 161 words in Grade 4 and 185 words in Grade 5. Next, the students' ability to analyse the text was examined by asking students why the main character of the text acted the way he did.Then, students' skills to evaluate the content of the text were tested.Therefore, students had to write about the educative aspects they had found in the story.
Students' answers were coded dichotomously: the right answer (1) and wrong (0).To improve the reliability of the coded answers, the authors of the article coded together 10% of answers.In case of differences the authors justified their decisions.After that all students' answers were coded separately and the differences were confabulated until consensus was achieved.
The tests were carried out by class teachers in Estonian language lessons.It took approximately one lesson (45 minutes) to implement the test.Students were tested twice: in the beginning of Grades 4 and 5. Completing the test was voluntary and subject to parents' approval.When tests were completed, teachers returned them to the researchers.

Data Analyzes
The descriptive analyzes were carried out with SPSS Statistics, version 20.0.For investigating differences between students' abilities to analyze and evaluate texts, non-parametric Wilcoxon's tests were used.To examine the relations between these tasks, Spearmen's correlation coefficient was found out, using Cohen's proposed guidelines for explaining correlations: r > .65 strong correlation; .35< r < .65 moderate correlation; r < .35slight correlation (Cohen, Manion, & Morrison, 2007).

Results
In order to find out the students' ability to analyze and evaluate the texts and the changes in it in two consecutive years, the descriptive analyses were carried out.The findings indicated that fewer than half of the examined students succeeded in analysing and evaluating the text, whereby the students' higher-level text comprehension skills in Grade 5 were better than in Grade 4. As expected, the students' ability to analyse text was better than evaluating it in both grades.However, in Grade 4 only 28.9 percent of the students performed analysing correctly and 21.2 percent of them passed the evaluation task.In Grade 5 the respective numbers were 41.6 percent and 29.1 percent.The table below presents the descriptive statistics related to students' results in analysing and evaluating the text (see Table 1).Then, the differences between students' ability to analyze and evaluate the content of the text were analyzed by using the Wilcoxon's test.The results showed that in both grades students did the analyzing tasks significantly better than the evaluating tasks, respectively Z = -3.90,p < .001 in Grade 4, and Z = -6.13,p < .001, in Grade 5.
Secondly, the relations between students' ability to analyze and evaluate the content of the text were assessed.The Spearmen's correlation coefficient indicated a statistically significant, but slight correlation between students' ability to analyse the content of the texts in Grades 4 and 5, ρ = .28,p < .001.In addition, we found significant correlation between students' ability to evaluate the content of the texts in Grades 4 and 5, ρ = .16,p < .001.As expected, those students who analysed and evaluated text better in Grade 4, did it also significantly better in Grade 5.
Thirdly, the Spearmen's correlation coefficient was used to find out the relations between students' ability to analyse and evaluate the text.We found significant, but slight correlation between students' ability to analyse and evaluate text in both Grades, ρ = .14,p < .001 in Grade 4, and ρ = .27,p < .001 in Grade 5. Thus, the students who had generally better analysing skills had also better evaluating skills.

Discussion
We examined students' higher-level text comprehension in Grades 4 and 5. Similarly to earlier studies, we found that our sample included less than 50% students with higher-level text comprehension skills (cf.Tire et al., 2013;Veeravagu et al., 2010).Furthermore, in accordance with earlier studies, we indicated the increase in students' ability to comprehend texts during the two years (Lynch et al., 2008).In addition, the relations between students' higher-level text comprehension skills, such as analyzing and evaluating skills of texts were detected.
First, we compared the students' ability to analyse the text with their ability to evaluate it.As expected, the students were significantly better in analyzing the text than in evaluating it.In revised Bloom's hierarchical taxonomy, the analyzing skill is in lower cognitive level than the evaluating skill (see Krathwhol, 2002).Also, at the levels of text comprehension, the analysing skill is needed at the inferential level, and evaluating skill together with the analysing skill is needed at the highest, evaluative level (Basabara et al., 2013).To analyse any text it is important to know how to integrate different parts of the text to a whole, to find the main idea of the text and to use the background knowledge (Butcher, & Kintsch, 2012;Eason et al. 2012;Kibui, 2012).During the evaluation of a text a reader has to use the highest cognitive processes: the text has to be critically evaluated, the author's thoughts or messages must be detected and associated with the reader's opinion.In addition, the reader has to analyse the text before evaluating it (Kaplan, 2013).Thus, our results are in accordance with earlier studies which have shown that students at this age are better in tasks which require much lower-level cognitive skills (Kaplan, 2013;Uibu, & Mannamaa, 2014;Van den Broek et al., 2003;Veeravagu et al., 2010).
Next, we assumed that students' higher-level comprehension skills are correlated in two consecutive years.We found that students who were better in analysing texts in Grade 4 were also better in Grade 5.The analogous correlations were found with text evaluating skills.Text comprehension is an ongoing and continuous process (Lynch et al., 2008;Seigneuric & Ehrlich, 2005).While working memory and general knowledge are growing over time, the older students are more capable of making content-rich inferences and they are better at integrating and evaluating texts (Lynch et al., 2008).However, in our study the students had a tendency to belong to the same text comprehension group in different grades.In the case of being a good reader (or on the contrary a poor reader) in Grade 4, there is a high possibility of being the same kind of reader in Grade 5 (cf.Phillips, Norris, Osmond, & Maynard, 2002;Uibu, & Mannamaa, 2014).
Lastly, we studied the relations between the students' ability to analyze and evaluate the text.We discovered that the students who were better in analyzing content of the texts were also more likely to be better in evaluating it.Previous studies have also shown that there is a correlation between the respective skills (Wang, 2009).The reason for this might be that the evaluative comprehension requires from reader the ability to construct a whole from different paragraphs of the text, make inferences, and possess other skills that are necessary for analysing the text (Basabara et al., 2013).In other words, to evaluate a text the analysing skills must be mastered first.
The study had several limitations.Firstly, our longitudinal study was carried out over a one-year period.A longer duration would have given a better general idea of students' text comprehension change.Secondly, students' higher-level text comprehension skills should have been studied together with some other reading components (for example reading speed, verbal ability).Also, the students' text comprehension together with the teachers' choice of instructional methods should have been analysed.
To conclude, to develop students' text comprehension, it is important to concentrate more on their higher-level cognitive skills (for example, analysing and evaluating the content of texts) because these skills are generally less evolved.It is also important to practice first the processes that are needed in inferential comprehension and from there proceed to teaching the processes that are required in evaluative comprehension.