Tuvalu Early Grade Reading Assessment (TuEGRA) RESULTS REPORT 1 Table of Content Table of Content ................................................................................................................. 2 List of Tables ...................................................................................................................... 3 List of Figures ..................................................................................................................... 4 Executive Summary ............................................................................................................ 5 Summary of EGRA results and findings ........................................................................ 6 Factors contributing to greater fluency and comprehension in Tuvalu .......................... 7 Recommendations ........................................................................................................... 7 Chapter 1: Introduction ..................................................................................................... 12 Chapter 2: Instrument Design and Implementation .......................................................... 14 2.1. TuEGRA Survey Implementation..................................................................... 14 2.2. Development of the TuEGRA Instrument ........................................................ 14 2.3. Enumerator training .......................................................................................... 15 2.4. Data collection .................................................................................................. 15 2.5. Reliability of the Instrument ............................................................................. 15 Chapter 3: EGRA Results ................................................................................................. 17 3.1. Sub-test 1 – Letter Name Knowledge ............................................................... 17 3.2. Sub-test 2 – Letter Sound Knowledge .............................................................. 19 3.3. Sub-test 3 – Initial Sound Identification ........................................................... 21 3.4. Sub-test 4 – Familiar Word Reading ................................................................ 22 3.5. Sub-test 5 – Nonword Reading ......................................................................... 24 3.6. Sub-test 6 – Oral Passage Reading ................................................................... 26 3.7. Sub-test 7 – Reading Comprehension ............................................................... 28 3.8. Sub-test 8 – Listening Comprehension ............................................................. 33 3.9. Sub-test 9 – Dictation........................................................................................ 35 3.10. Summary of assessment results ........................................................................ 39 Chapter 4 – Performance in reading fluency and reading comprehension ....................... 42 Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes .......................................................................................................................... 48 5.1. Association of student characteristics to student reading outcomes ................. 48 5.2. Association of teacher characteristics to student reading performance ............ 50 5.3. Association of teacher expectations to student reading performance ............... 52 5.4. Association of teaching training and guides to student reading performance .. 53 5.5. Association of Classroom Environment to Student Performance .................... 54 5.6. Association of Reading Materials to Student Performance .............................. 56 5.7. Association of Teaching and Assessment Methods to Student Performance ... 56 Chapter 6 - Conclusions and Recommendations .............................................................. 61 ANNEX 1 / TABLES ....................................................................................................... 65 ANNEX 2 / INSTRUMENTS .......................................................................................... 73 2 List of Tables Table 1: EGRA Sample by Region, Year and Gender...................................................... 14 Table 2: Reliability of the EGRA Sub-tests ...................................................................... 16 Table 3 Sub-test 1 Letter Name Knowledge: Results by Year and Gender...................... 18 Table 4: Letter Name Knowledge: Results by Region and Year ...................................... 18 Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender ................. 20 Table 6: Letter Sound Identification: Results by Region and Year .................................. 20 Table 7: Sub-test 3 Initial Letter Sound Identification: Results by Year and Gender ...... 21 Table 8: Initial Letter Sound Recognition: Results by Region and Year ......................... 22 Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender ...................... 23 Table 10: Familiar Word Reading: Results by Region and Year ..................................... 23 Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender............................. 25 Table 12: Nonword Reading: Results by Region and Year .............................................. 26 Table 13: Sub-test 6 Oral Passage Reading: Results by Year and Gender ....................... 27 Table 14: Oral Passage Reading: Results by Region and Year ........................................ 27 Table 15 Sub-test 6b Reading Comprehension: Results By Year and Gender ................. 29 Table 16: Distribution of Number of Correct Answers .................................................... 32 Table 17: Reading Comprehension: Results by Region and Year .................................... 32 Table 18: Sub-test 7: Listening Comprehension: Results by Year and Gender ............... 33 Table 19: Listening Comprehension Percentage of Correct Answers by Question.......... 34 Table 20: Listening Comprehension: Results by Region and Year .................................. 35 Table 21: Sub-test 9 Dictation: Results by Year and Gender ........................................... 36 Table 22: Dictation: Results by Region and Year ............................................................ 38 Table 23: Distribution of percent correct scores for reading comprehension by sub-test per year .................................................................................................................. 42 Table 24: Percent of students reading with 80% or more comprehension by Year .......... 43 Table 25: ORF score by reading comprehension score .................................................... 44 Table 26: Distribution of ORD Scores for students meeting and not meeting 80% benchmark ....................................................................................................... 46 Table 27: Student background characteristics .................................................................. 48 Table 28: Association of student characteristics to Oral Reading Fluency (ORF) scores 49 Table 29: Profile of teachers in TuEGRA......................................................................... 50 Table 30: Association of teachers' characteristics to Oral Reading Fluency Score (ORF) ......................................................................................................................... 52 Table 31: Association of teacher expectations to oral reading fluency (ORF) scores ...... 53 Table 32: Teacher training and guides .............................................................................. 54 Table 33: Association of training and guides to student oral reading fluency (ORF) scores ......................................................................................................................... 54 Table 34: Average Number of Classroom Displays and Materials Observed .................. 54 Table 35: Frequency and Type of Classroom Displays/Resources Available .................. 55 Table 36: Association of Classroom Environment to Student ORF Scores ..................... 55 Table 37:Average Number of Reading Materials and Books in Classroom ..................... 56 Table 38: Association of Reading Instructional Resources to Student ORF Scores ........ 56 Table 39: Frequency of methods used during reading instruction .................................... 58 3 Table 40: Association of teacher instructional and assessment methods to student performance .................................................................................................... 58 List of Figures Figure 1: Letter Name Knowledge: Results by Region and Year .................................... 19 Figure 2: Letter Sound Identification: Results by Region and Year ................................. 21 Figure 3: Initial Letter Sound Recognition: Results by Region and Year ........................ 22 Figure 4: Familiar Word Reading: Results by Region and Year ...................................... 24 Figure 5: Nonword Reading: Results by Region and Year............................................... 26 Figure 6: Oral Passage Reading: Results by Region and Year ......................................... 28 Figure 7: Reading Comprehension: Distribution of percentage of correct answers by year ................................................................................................................................... 30 Figure 8: Reading Comprehension: Distribution of percentage of correct answers by gender ................................................................................................................................... 31 Figure 9: Reading Comprehension: Results by Region and Year .................................... 33 Figure 10: Listening Comprehension: Results by Region and Year................................. 35 Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year ............ 37 Figure 12: Dictation Results: Percentage of Correct Answers by Item and Gender ........ 37 Figure 13: Dictation: Results by Region and Year ........................................................... 38 Figure 14: Summary Results: Number correct answers for timed sub-tests by year ........ 39 Figure 15: Summary Results: Percentage of correct answers for untimed sub-tests by Year ................................................................................................................................... 40 Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year .............. 40 Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender .......... 41 Figure 18: ORF distribution by reading comprehension level.......................................... 44 Figure 19: Distribution of ORF scores for students reading with at least 80% comprehension .......................................................................................................... 45 Figure 20: Distribution of ORF scores for students meeting and not meeting 80% benchmark ................................................................................................................. 46 4 TUVALU EARLY GRADE READING ASSESSMENT Results Report Executive Summary This report summarizes the results of an Early Grade Reading Assessment (EGRA) conducted in Tuvalu from September 21 and October 27, 2016. With funding from the Global Partnership for Education, the World Bank and Education Technology for Development (Et4d) carried out the assessment in collaboration with the Tuvaluan Ministry of Education, Youth and Sports (MEYS). The overall purpose of the EGRA was to inform education policymakers of students’ basic reading skills in years 1-3 of primary school and to identify factors that contribute to language development in Tuvalu. The findings are expected to assist policymakers with designing effective early grade reading interventions to improve school performance and literacy outcomes in Tuvalu. This activity is part of the Pacific Early Age and Readiness Program (PEARL), which was established to improve the school readiness and literacy outcomes of children throughout the Pacific region. The Tuvalu EGRA (TuEGRA) was administered as a census survey to all 10 primary schools in all regions of Tuvalu. A total of 593 of 709 students enrolled participated in the assessment. Due to a high number of student absences and the proximity of a national holiday, it was not possible to survey all students. The EGRA tool consisted of six reading skills tests and two reading-related tests (listening comprehension and dictation). Unlike most EGRAs, which primarily test reading and listening skills, the TuEGRA included a short dictation exercise to assess early writing skills. In addition, a teacher and student survey that collected information on characteristics associated with reading outcomes was administered to identify factors contributing to reading fluency. The assessors also carried out a classroom observation in each school visited to assess the classroom environment and teaching resources available. The analysis of TuEGRA data included descriptive statistics (means and standard deviations) to measure average levels in basic reading skills; an analysis of variance to determine statistical significance of gender and regional differences; and regression analysis to estimate the association of a given teacher or student characteristic and reading fluency outcomes. 5 The key findings, factors associated with reading performance, and recommendations are presented below. Summary of EGRA results and findings • Students expressed good emergent literacy skills. Students have good pre- reading skills, initial phonics and early writing skills demonstrated by high scores on listening comprehension, dictation and letter naming sub-tests. Students scored over 50 percent on listening comprehension and dictation sub-tests compared to 16 percent on reading comprehension. Listening comprehension and early writing skills are precursors to reading development. • Students are not yet fluent readers: In order to understand a simple passage, students must read fast enough to retain the words in short-term memory. Research1 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. Overall, reading fluency scores averaged 20 correct words per minute on the oral reading fluency sub-test. Year 3 students read an average of 38 correct words per minute, which is well below the minimum standard. However, oral reading fluency scores were relatively higher than other timed fluency sub-tests, which suggests that students were better able to read words in context rather than in isolation. • Students’ struggle with decoding. To read text fluently, students must be able to decode unfamiliar words by sounding out individual letters and syllables. Students scored low across two sub-tests that measure these skills: initial sounds and nonwords. Overall, students identified an average of 5 initial sounds and 8 nonwords. • Reading comprehension levels are low across all years. The majority of students in Years 1 and 2 and 40 percent in Year 3 have zero reading comprehension. In Year 3, comprehension improves where about one-third of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the recommended 80 percent comprehension benchmark. Only 9 percent of all students met the benchmark and about 20 percent in Year 3. The breakthrough point for reading fluency and comprehension is likely much later than Year 3 for the majority of students. An analysis of fluency scores falling in the 80 percent comprehension range revealed that students must be able to read between 50-79 words per minute with accuracy. • Girls have higher reading fluency and comprehension . Girls tended to read better than boys across all sub-tests and the difference was statistically significant. • Regional differences were not statistically significant. Students in the northern and southern regions performed better than other regions. Yet, the differences were insignificant. 1 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative 6 Factors contributing to greater fluency and comprehension in Tuvalu The following factors2 were positively associated with better oral reading fluency: Student characteristics: • Pre-school attendance (+10 correct words read per minute (cwpm)) • Students reading to herself/himself at home (+10 cwpm) • If students like to read (+10 cwpm) • Having reading materials in English (+7 cwpm) • Having a literate father /receiving help with homework from mother (+6 cwpm) Teacher expectations: • When teachers expect students to reach the competency later than the expected grade level, students tend to have higher scores (+3-13 cwpm) Classroom environment: • Maintaining a folder with student work and student info/student profile (+10 cwpm) • Having a reading corner (+5 cwpm) • Having sufficient classroom space for organized group activities (+5 cwpm) Teaching pedagogy and assessment: • Teaching new vocabulary words 1-2 days a week (+14 cwpm) • Reading daily aloud or silently (+6-10 cwpm) • Teaching unfamiliar words (+5-7 cwpm) • Teaching reading comprehension (+1-3 cwpm) Recommendations Based on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Tuvaluan schools: 1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that 2 Note that the average effects of the factors are not cumulative. In other words, students who possess more than one characteristic do not have double the effect by adding the values of both factors. 7 are required for schools to succeed in achieving them. As a first step, stakeholders in Tuvalu should decide on the level of comprehension required to understand grade level text (e.g., 60%, 80%) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders in Tuvalu agree with the 80 percent benchmark, then looking at the TuEGRA data, an acceptable fluency range may be 70-75 cwpm. Given that the current mean ORF score is 21 cwpm, that range may seem too high for students to achieve. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 57 cwpm, although this decision will carry important educational implications that ought to be discussed in country. Once the benchmark is decided, the next step is to consider the targets. Currently, 9 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Tuvaluan MOE should then conduct a national reading campaign to sensitize all stakeholders of the new benchmarks, inform educators and schools of the support mechanisms available to achieve them, and then monitor and report progress towards achieving the targets at all levels (national/regional MOE, school and communities). 2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Teacher training and assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that so few teachers were trained (15%) so there was lack of support to master the new methodologies or that training was not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress. 3. Strengthen teacher training for reading instruction with a focus on vocabulary, phonics and reading comprehension. Students whose teachers spent more time on teaching vocabulary, letter sounds, decoding skills and reading comprehension had higher ORF scores. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Tuvalu, the large shares of zero scores observed in various reading subtasks included in TuEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Teacher training should incorporate the 8 curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results. In addition, classrooms should be equipped with reading corners to provide students with sufficient resources and opportunities to read. 4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Tuvalu, 80 percent of classrooms have reading corners with an average of four books, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies3. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50-75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Tuvaluan educators are already producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. Additionally, in low- resource countries where publishing may be expensive and there are high teacher- pupil ratios, a low-cost option is to provide e-readers. E-readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides additional support for emergent readers45. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners6. 5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluency skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. If more titles are produced to assist children in their path to literacy, teachers and parents should ensure adequate time is given to read aloud activities at school and at home even among students considered to be reading fluently and with comprehension. 3 Neuman, S. (undated). The importance of the classroom library. Scholastic. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf 4 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of- children/ 5 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf 6 Neuman, S. (undated). The importance of the classroom library. 9 6. Address the issue of non-readers in Years 1 and 2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approachâ€? is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students. 7. Teach students how to develop reading comprehension skills in the early grades. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide potential solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 17. 8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, 7 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3 rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third- Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading- Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx) 10 science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government)8. 8 Sample guide for improving boys’ literacy: Ontario Ministry of Education. (2004). Me Read? No Way! A practical guide to improving boys’ literacy skills. Available at: http://www.edu.gov.on.ca/eng/document/brochure/meread/meread.pdf 11 Chapter 1: Introduction Tuvalu is a country consisting of nine coral atolls in the South Pacific Ocean located about halfway between Hawaii and Australia. It is one of the smallest and most remote countries in the world with a population of only 10,959 (July 2016 est.) 9. Education is free and compulsory for the first eight years of schooling. Primary education consists of Years 1 to 6, and forms 1 and 2.10 There are 10 primary schools (nine public and one private), 18 early childhood care and education centers, and two secondary schools. Among the total official school age population (6-13 years old), 82 percent were enrolled in primary school in 2015. Including those over and under-age, the gross enrollment rate was 93.3 percent11. Despite the government’s commitment to education and high enrollment rates, literacy rates are alarmingly low in the country. According to the 2012 Pacific Islands Literacy and Numeracy Assessment (PILNA), 19 percent of Year 4 students in Tuvalu are performing at the expected level for literacy and 25 percent of students are meeting literacy benchmarks in Year 6. The PILNA acknowledges a lack of evidence to explain such low performance and calls on Pacific Island countries to investigate the cause and determine the most appropriate and relevant interventions. In response, the Pacific Early Age and Readiness Program (PEARL) was established to improve the school readiness and literacy outcomes of young children throughout the region. The World Bank, in collaboration with the Tuvaluan government and local education stakeholders, are collaborating in this joint effort to undertake an Early Grade Reading Assessment and to design effective interventions to improve school readiness and reading outcomes in Tuvalu. The Early Grade Reading Assessment (EGRA) measures basic reading skills in the first three grades of primary school. The EGRA tool has been administered in over 100 languages in more than 60 countries around the world, including the Pacific islands of Tonga, Vanuatu, Papua New Guinea and Timor Leste. The EGRA tool was adapted to the Tuvaluan context and administered nationally from September 21 to October 27, 2016. The Tuvaluan EGRA (TuEGRA) aimed to achieve three objectives: 1. To develop survey of basic reading skills and temporary reference standards to monitor reading performance in schools and system wide; 2. To build local capacity to replicate early grade reading assessments in the future; and 3. To work with local education stakeholders to interpret EGRA findings and analyze their policy and sector investment implications. 9 CIA World Factbook. (2016). Tuvaluan population. Available at https://www.cia.gov/library/publications/the-world-factbook/geos/tv.html 10 In the Tuvalu education system, grade levels are called years. Year 1 is a synonym for Grade 1. 11 Tuvalu Education Department Ministry of Education, Youth and Sports (2016). 2015 Annual and Statistical Report. 12 The overall purpose of the TuEGRA is to inform education stakeholders of reading skills students are struggling with and to identify factors that appear to contribute to reading development in Tuvalu. The findings will help policymakers to design responsive strategies to improve early grade reading instruction and assessment. In particular, the EGRA results may assist Tuvaluan policymakers with building the capacity of teachers, allocating budgetary and human resources, and with developing reading indicators and benchmarks. The ultimate goal of the TuEGRA is to ensure that all children develop the skills needed to become fluent readers. Towards this aim, the assessment was designed to respond to the following questions: 1. What are the basic reading skills of students in Years 1, 2, and 3 in Tuvalu? 2. What is the year in which students in Tuvalu “break intoâ€? reading fluently with comprehension? 3. How does student performance in Years 1, 2, and 3 compare to curriculum expectations in Tuvalu language for these grades? 4. What are the main skills students struggle with? What are the main skills students have strengthened? 5. What are the differences in performance between boys and girls? 6. What are the student and teacher factors associated with strong and weak early grade reading outcomes? a. What are teacher expectations around when students should develop key basic reading skills? How different are these from curricular expectations in the early grades? b. What are the resources available in classrooms to support the reading development of children? c. What are the pedagogical practices supporting reading that are prevalent in the early grades in Tuvaluan classrooms? The report is divided into six chapters beginning with this introductory section in Chapter 1. Chapter 2 will discuss the TuEGRA survey design and implementation. Chapter 3 presents overall results of the reading assessment per sub-test, gender and year as well as a summary of the overall strengths and weaknesses responding to questions 1, 4 and 5. The correlation between oral reading fluency and comprehension (question #2) is discussed in Chapter 4. As there are no established standards for reading fluency, the report used 80 percent reading comprehension benchmark as the basis for determining fluency scores. Chapter 5 provides an analysis of student and teacher factors associated with improved reading outcomes, which corresponds with question #6 (6a, 6b, and 6c). Finally, the conclusions and policy recommendations to improve literacy outcomes are presented in Chapter 6. 13 Chapter 2: Instrument Design and Implementation 2.1. TuEGRA Survey Implementation In order to build local capacity to replicate early grade reading assessments, Tuvaluan Ministry of Education staff were directly involved in the EGRA design and implementation. From August 16 to October 27, 2016, the World Bank provided in-country support to selected Ministry staff to undertake all aspects of the assessment including, sampling, adapting the TuEGRA instruments into the Tuvaluan language, facilitating the training of enumerators and supervisors, coordinating survey logistics during the pilot and fieldwork, and administering the EGRA using Tangerine software. Given the small primary school population, rather than selecting a representative sample, the Ministry of Education, Youth and Sport (MEYS) of Tuvalu decided to conduct a census of early grade reading performance for all 709 students enrolled in the 10 primary schools from years 1 to 3. Therefore, no sampling design was necessary. While this could have posed a challenge during the pre-test and piloting of the instrument, the evaluation team took precautions to minimize risk. For instance, Year 4 and Year 1 students were selected for practice during the enumerator training since Year 4 students were outside of the target population for the assessment, and Year 1 students would likely not comprehend enough of the text to be able to recall and disseminate knowledge. The final assessment student population was 593 students. Because of high absenteeism, partially due to the Tuvaluan independence day, only 593 students were assessed out of 709 students enrolled during the data collection, among which 311 were boys and 282 girls (See Table 1). Table 1: EGRA Sample by Region, Year and Gender Region Year 1 Year 2 Year 3 Total Girl Boy Total Girl Boy Total Girl Boy Total Central 14 23 37 17 23 40 13 18 31 108 Funafuti 52 57 109 56 60 116 55 48 103 328 North 18 12 30 15 12 27 15 21 36 93 South 2 7 9 5 5 10 4 7 11 30 Nui 9 7 16 6 10 16 1 1 2 34 Total 95 106 201 99 110 209 88 95 183 593 2.2. Development of the TuEGRA Instrument Due to differences in language, culture, and expectations about learning outcomes, the EGRA tool was adapted to the Tuvaluan context and piloted. From August 16 to 24, 2016, a team of language experts, curriculum officers and teachers attended an adaptation workshop focused on learning the research foundations and structure of the EGRA tool in order to develop the TuEGRA instrument in the Tuvaluan language. Before the workshop, a language analysis was carried out to gain a good understanding of what orthography and language issues needed to be taken into account when developing the EGRA tools. Also a letter frequency list, word frequency list and possible non-word list were developed. The 14 letter names and letter sound grid had been drafted prior to the workshop. During the workshop, participants worked in small groups to develop a number of potential listening and reading texts with comprehension questions. Participants also translated the instructions into Tuvalu. Two versions of the instruments were pre-tested on August 22, 2016. The results of the pre-test were shared and feedback from workshop participants and ministry staff were incorporated into the instrument. The revised instrument was piloted on August 24, 2016, and a few minor changes were applied to some of the comprehension questions and word items. 2.3. Enumerator training The Enumerator Training Workshop for TuEGRA was held in Funafuti on September 14 to 20, 2016. Pre-selected enumerators, a head teacher and the local coordinator attended the training workshop to learn about the scope and purpose of the survey and the manipulation of tablets for data collection. Assessors were trained three days in a classroom environment, and they had the chance to administer a practice EGRA in a nearby primary school, where the instruments had been previously piloted. An Assessor Accuracy Measurement12 (AAM) was conducted three times in order to familiarize the enumerators with the process and examine their accuracy level. Results were used to select 7 enumerators. All candidates' results were above 95 percent overall. 2.4. Data collection Data collection took place in Tuvalu between September 21 and October 27, 2016, by a team of seven enumerators supervised by a local coordinator and a data quality assurance specialist. Fieldwork began in the two schools of Funafuti. Then the team proceeded with a trip to the Central Islands followed by the Northern Islands. Upon their return, the team returned to Nauti primary school in Funafuti, attempting to assess some of the students who were absent during the first round of data collection due to Independence Day celebrations. In late October, three enumerators and their supervisor visited schools in the Southern Islands. 2.5. Reliability of the Instrument Cronbach’s alpha was computed for every sub-test of the EGRA instrument to estimate the level of reliability of each sub-test. Results showed strong internal consistency for almost every sub-test except Reading Comprehension and Listening Comprehension. As a rule of thumb, an alpha coefficient of 0.80 is considered good while a 0.70 is the minimum acceptable result. The lower values of alpha coefficients for Reading and Listening 12 The AAM evaluates the degree of agreement among enumerators administering the same test at the same time to the same student. This type of measurement determines the trainees’ abili ty to accurately administer the EGRA. A script prepared beforehand, complete with deliberate errors, becomes the gold standard. After a practice exercise, the enumerators’ responses are compared against the gold standard. The benchmark is set at 90% agreement between the gold standard and the enumerator’s responses. Therefore, if the enumerators’ responses are in agreement with at least 90% of the responses on the gold standard, the assessor is considered qualified to administer the EGRA. The results are used to address any weaknesses identified during the enumerator training. 15 comprehension could be explained by a smaller number of items (five) in those sub-tests since the number of items in a sub-test influences the value of the Cronbach’s alpha. Table 2: Reliability of the EGRA Sub-tests Sub-tests Number of items Number of Alpha respondent Letter Name Knowledge 100 593 0.976 Initial Sound Identification 10 593 0.930 Letter Sound Knowledge 100 593 0.976 Familiar Word Reading 50 593 0.985 Non Word reading 50 593 0.976 Oral reading 54 593 0.990 Reading Comprehension 5 593 0.853 Listening Comprehension 5 593 0.592 Dictation 22 404 0.943 Correlations between tasks are presented in Annex 1.B. 16 Chapter 3: EGRA Results This section reports the results of the EGRA per sub-test. There were eight sub-tests comprised of the following: 1. Letter Name Knowledge 2. Letter Sound Knowledge 3. Initial Sound Identification 4. Familiar Word Reading 5. Nonword Reading 6. Oral Passage Reading 7. Reading Comprehension 8. Listening Comprehension 9. Dictation The average scores of the above sub-tests are provided and disaggregated by year and gender. Mean scores are presented for the entire sample as well as for the sample that was able to read at least one item (minus zero scores). The evaluators agreed that it was important to present results with and without zero scores. The overall results show the actual reading performance of the entire population, including those who can and cannot read. The results minus zero scores show the average score of the population who can read at least one item, which addresses research concerns that the inclusion of a large percentage of zero-scores could underestimate the result of the population who can read. Results disaggregated by region are also provided at the end of each section to highlight any regional differences in average reading performance. 3.1. Sub-test 1 – Letter Name Knowledge13 The test of letter name knowledge is the most basic assessment of reading skills. It measures students’ ability to read the names of the letters of the alphabet with accuracy. Automaticity and fluency of letter name knowledge is a predictive skill for later reading success. During the EGRA, students were given a page of 100 randomly distributed upper- and lowercase letters and asked to say the names of as many letters as possible within one minute. The test included the full Tuvaluan alphabet and randomization was used to prevent students from reciting a memorized alphabet. The test is scored by the number of letters that students correctly name in one minute (correct letters per minute—clpm). Table 3 shows overall scores for sub-test 1 by year, gender and minus zero scores. The results were quite low for all years. Overall, students were able to identify 21 letters in one minute. Excluding the zero scores, the results increase to 25 correct letters per minute. As students progress from Years 1-3, they are able to identify an additional six letters in Year 2 and an additional five letters in Year 3. Overall, girls have higher knowledge of letter 13 The sub-test descriptions are based on the Early Grade Reading Assessment Toolkit (RTI, 2009). 17 names than boys, identifying 23 letters compared to 18.5 for boys. Some boys in Years 2 and 3 were able to identify 100 letter names while the maximum score for girls was 88. Table 3 Sub-test 1 Letter Name Knowledge: Results by Year and Gender Sub-test 1 – Number of correct letter N Mean SD Min Max names identified per minute (CLPM) CLPM – overall 593 21.0 18.4 0 100 CLPM – minus zero score students 496 25.1 17.3 1 100 Year CLPM – overall 201 15.5 14.9 0 69 Year 1 CLPM – minus zero score students 152 20.5 13.9 1 69 CLPM – overall 209 21.3 19.5 0 100 Year 2 CLPM – minus zero score students 176 25.3 18.7 1 100 CLPM – overall 183 26.8 18.7 0 100 Year 3 CLPM – minus zero score students 168 29.2 17.6 1 100 Gender CLPM – overall 311 23.4 18.9 0 88 Girls CLPM – minus zero score students 269 27.0 17.7 1 88 CLPM – overall 282 18.5 17.5 0 100 Boys CLPM – minus zero score students 227 22.9 16.6 1 100 According to regional results in Table 4, the Northern region had the highest scores for Years 1-3 while the Southern region had the lowest overall mean score. Table 4: Letter Name Knowledge: Results by Region and Year Central Funafuti North South Year 1 13.1 15.6 19.8 13.4 Year 2 21.0 20.7 27.1 14.6 Year 3 26.2 25.7 30.3 27.9 Overall 19.3 20.6 25.9 19.1 18 Figure 1: Letter Name Knowledge: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 5 10 15 20 25 30 35 Number of correct letters per minute 3.2. Sub-test 2 – Letter Sound Knowledge In order to read fluently, students must understand the alphabetic principle (that words are made up of sounds) and that letters (e.g., graphemes) are symbols that represent sounds. Thus, knowledge of how letters correspond to sounds is a critical skill children must master to become successful readers. The letter sound knowledge test is administered similarly to the letter name knowledge subtest. Students were provided a page of 100 randomly distributed upper- and lowercase letters of the Tuvaluan alphabet and asked to provide the sounds (not the names) of as many letters as they could identify within a one-minute period. The results of this sub-test by year, gender and with and without zero scores are shown in Table 5. The overall results for letter sound identification were similar to letter name knowledge. Students were able to identify 21 letter sounds on average and 25 letter sounds when zero scores are excluded. Year 1 students accurately identified 16 letters. Students in Year 2 could name an additional four letter sounds, for a total mean score of 20. In Year 3, there is a large increase in knowledge as students could name nearly 30 letter sounds – nearly twice as many as Year 1 students. Girls performed significantly higher than boys, reading 8.3 more letter sounds than boys did. 19 Table 5: Sub-test 2 Letter Sound Identification: Results by Year and Gender Sub-test 2 – Number of correct letter sounds N Mean SD Min Max identified per minute (CSPM) CSPM – overall 593 21.6 18.8 0 95 CSPM – minus zero score students 485 26.4 17.4 1 95 Year CSPM – overall 201 16.4 15.9 0 69 Year 1 CSPM – minus zero score students 156 21.1 15.0 1 69 CSPM – overall 209 19.7 17.1 0 80 Year 2 CSPM – minus zero score students 174 23.7 15.9 1 80 CSPM – overall 183 29.4 20.9 0 95 Year 3 CSPM – minus zero score students 155 34.7 18.2 2 95 Gender CSPM – overall 311 25.5 19.0 0 95 Girls CSPM – minus zero score students 273 29.1 17.6 1 95 CSPM – overall 282 17.2 17.4 0 80 Boys CSPM – minus zero score students 212 22.9 16.6 1 80 Students in the Northern and Central regions scored about five points higher than other regions (Table 6). The Southern and Funafuti regions performed the lowest averaging 19- 20 correct letter sounds per minute. Table 6: Letter Sound Identification: Results by Region and Year Central Funafuti North South Year 1 23.7 13.2 15.1 16.7 Year 2 21.6 18.5 22.1 16.7 Year 3 32.9 26.4 35.8 25.7 Overall 25.0 19.2 25.1 20.0 20 Figure 2: Letter Sound Identification: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 10 20 30 40 Number of correct letters per minute 3.3. Sub-test 3 – Initial Sound Identification Initial sound identification assesses phonemic awareness (the ability to hear, identify and manipulate individual sounds –phonemes- in spoken words). In this test, the assessor read aloud a word and then asked the student to identify the first sound, or phoneme, in the word. The test is comprised of 10 words and is administered orally. The assessor read aloud a word and then asked the student to identify the first sound, or phoneme, in the word. The test is untimed and is measured according to the number of items answered correctly. Overall, students could identify 50 percent of the initial sounds attempted (5 out of 10 items). Excluding the zero score students, the average score increases to six. Identifying initial sounds was a particularly difficult task for all students. Students identified an average of four initial sounds in Year 1, with marginal increases to 4.9 in Year 2, and 6.3 in Year 3. Similar to other sub-tests, girls performed better than boys, but by a slight margin - 1.7 more initial sounds for girls than boys (5.8 initial sounds identified by girls compared to 4.1 for boys). Table 7: Sub-test 3 Initial Letter Sound Identification: Results by Year and Gender Sub-test 3 – Number of correct initial sounds N Mean SD Min Max identified Overall 593 5.0 3.8 0 10 Minus zero score students 496 6.0 3.4 1 10 Year Overall 201 4.0 3.7 0 10 Year 1 Minus zero score students 152 5.3 3.4 1 10 Overall 209 4.9 3.8 0 10 Year 2 Minus zero score students 180 5.7 3.4 1 10 Overall 183 6.3 3.7 0 10 Year 3 Minus zero score students 164 7.0 3.1 1 10 21 Gender Overall 311 5.8 3.8 0 10 Girls Minus zero score students 275 6.6 3.3 1 10 Overall 282 4.1 3.7 0 10 Boys Minus zero score students 221 5.3 3.4 1 10 Based on the results by region in Table 8, the Central region demonstrated the highest scores on initial letter sound while the Southern and Funafuti regions scored the lowest. Table 8: Initial Letter Sound Recognition: Results by Region and Year Central Funafuti North South Year 1 5.6 3.3 4.0 3.1 Year 2 5.6 4.5 5.1 4.7 Year 3 7.3 5.9 6.6 5.5 Overall 6.0 4.6 5.3 4.5 Figure 3: Initial Letter Sound Recognition: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 2 4 6 8 Number of correct sounds 3.4. Sub-test 4 – Familiar Word Reading The familiar word sub-test measures students’ ability to read familiar words with fluency and accuracy, both of which are necessary to become fluent readers. For the familiar-word reading test, students were given a list of 50 familiar words (selected from readers available in the classroom) with instructions to read as many as they could in one minute. Familiar- word reading is a timed test scored by the number of correct words read per minute (cfwpm). 22 Overall, students can read about 13 familiar words correctly in one minute (See Table 9). When the zero scores are removed, the mean score jumps to 21 correct words per minute. The difference in means scores between overall and minus zero score students indicates that there are a significant proportion of non-readers in Year 1 and Year 2, and that most students can read at least one word by Year 3. Year 1 readers could read an average of 10 words per minute, but when the non-readers are included in the sample the mean score drops to four words per minute. In Year 2, the mean score was 18 words per minute among readers, and 11 words per minute when non-readers were included. By Year 3, the difference in scores between readers and non-readers shrinks to five words per minute (29 words per minute for readers and 24 words per minute for readers and non-readers). Thus, by Year 3 most students can read at least one familiar word per minute. On average, girls read seven more words than boys (16.2 and 9.1, respectively) and boys had a higher percentage of zero scores. Table 9: Sub-test 4 Familiar Word Reading: Results by Year and Gender Sub-test 4 – Number of correct familiar words N Mean SD Min Max read per minute (CFWPM) CFWPM – overall 593 12.8 16.7 0 73 CFWPM – minus zero score students 361 21.1 16.9 1 73 Year CFWPM – overall 201 4.5 8.4 0 40 Year 1 CFWPM – minus zero score students 85 10.7 10.1 1 40 CFWPM – overall 209 11.0 15.3 0 73 Year 2 CFWPM – minus zero score students 126 18.2 16.0 1 73 CFWPM – overall 183 24.0 18.9 0 71.4 Year 3 CFWPM – minus zero score students 150 29.3 16.7 1 71.4 Gender CFWPM – overall 311 16.2 17.6 0 73 Girls CFWPM – minus zero score students 224 22.5 17.0 1 73 CFWPM – overall 282 9.1 14.8 0 69.8 Boys CFWPM – minus zero score students 137 18.7 16.4 1 69.8 On the familiar word reading sub-test, regional averages were highest in the southern region and lowest in Funafuti (See Table 10). Unlike previous sub-tests in which the southern region had the lowest performance, the southern region scored second highest on this sub-test. Table 10: Familiar Word Reading: Results by Region and Year Central Funafuti North South Year 1 6.0 2.8 6.4 10.0 23 Year 2 9.8 9.8 15.0 21.1 Year 3 26.6 19.9 30.2 35.6 Overall 12.3 10.6 18.2 23.1 Figure 4: Familiar Word Reading: Results by Region and Year 3.5. Central Sub-test 5– Funafuti Nonword Reading Year 1 To be North Year 2 able to Year 3 South 0 10 20 30 40 Number of correct words per minute comprehend unfamiliar text, children must be able to decode new unfamiliar words. Nonword reading measures students’ ability to decode words that follow the linguistic rules of the Tuvaluan language but that do not exist in real life. Unlike familiar word reading in which students can read from memory or sight recognition, on the nonword reading sub- test students must sound out the words; thus, it is a purer measure of decoding ability. Students were provided with a table of 50 made-up words and instructed to read as many as they could within one-minute. The test is timed and is measured by the number of correct nonwords read per minute (cnwpm). Table 11 shows the results of the nonword reading sub-test by year, gender and with and without zero scores. The results indicate that students in all three years are struggling with decoding skills. The overall mean score was 8.1 correct words per minute and 18.8 excluding zero scores. This test was most difficult for Year 1 students who could only read an average of 2.8 nonwords correctly per minute. Performance improves in Year 2, in which students read an average of 6.9 nonwords per minute, and most significantly in Year 3 where the overall mean score was 15.4 words per minute. Similar to familiar word reading, when zero scores are removed the results improve significantly for readers in Years 1 and 2, but there is not much difference in Year 3 since most students can read at least one word correctly by the end of year 3. Girls achieved nearly twice as much as boys scoring 10.4 compared to 5.6 for boys. 24 Table 11: Sub-test 5 Nonword Reading: Results by Year and Gender Sub-test 5 – Number of correct nonwords read N Mean SD Min Max per minute (CNWPM) CNWPM – overall 593 8.1 11.7 0 51.7 CNWPM – minus zero score students 256 18.8 10.8 1 51.7 Year CNWPM – overall 201 2.8 6.6 0 31 Year 1 CNWPM – minus zero score students 45 12.6 8.4 2 31 CNWPM – overall 209 6.9 10.6 0 39 Year 2 CNWPM – minus zero score students 83 17.3 10.1 1 39 CNWPM – overall 183 15.4 13.6 0 51.7 Year 3 CNWPM – minus zero score students 128 22.0 10.9 1 51.7 Gender CNWPM – overall 311 10.4 12.7 0 51.7 Girls CNWPM – minus zero score students 165 19.6 11.1 1 51.7 CNWPM – overall 282 5.6 9.9 0 46.3 Boys CNWPM – minus zero score students 91 17.4 10.0 2 46.3 According to regional results in Table 12, the southern region scored the highest with a mean score of 16.9 correct nonwords per minute followed by the northern region with an average score of 11.6. Funafuti had the lowest scores (6.4 correct nonwords per minute). 25 Table 12: Nonword Reading: Results by Region and Year Central Funafuti North South Year 1 3.3 1.6 4.6 9.1 Year 2 6.1 5.7 10.1 15.5 Year 3 18.0 12.4 18.6 24.7 Overall 7.8 6.4 11.6 16.9 Figure 5: Nonword Reading: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 5 10 15 20 25 30 Number of correct nonwords per minute 3.6. Sub-test 6 – Oral Passage Reading The best single measure of a child’s reading proficiency in the primary grades is oral reading fluency (ORF). Oral reading fluency measures students’ ability to read a short passage with sufficient speed and accuracy. It encompasses all of the previous reading skills plus the skills needed for comprehension -- the ability to translate letters into sounds, unify sounds into words, process connections, relate text to meaning, and make inferences. 14 ORF has been shown to be a powerful predictor of overall reading competence and comprehension15. The oral passage reading sub-test is a timed test. In order for students to understand a simple passage, they must be able to read it fast enough to retain the words in short-term memory. Research 16 suggests a minimum fluency rate of 45-60 words per minute, depending on the complexity of the language. In this sub-test, students were asked to read 14 Hasbrouck, J., & Tindal, G. A. (2006). “Oral reading fluency norms: A valuable assessment tool for reading teachers.â€? The Reading Teacher, 59(7), 636–644. 15 Ibid 16 Abadzi, H. (2011). Reading Fluency Measurements in EFA FTI Countries: Outcomes and Improvement Prospects. Education for all Fast Track Initiative 26 a very short story comprised of about 60 words in one minute. After one minute, the assessor stopped students and recorded the number of words read correctly (cwpm). If the child could not read any words correctly in the first line, the assessor stopped the test early and the child received a score of zero. The overall mean score for this sub-test was 20 correct words per minute, well below the 45-60 cwpm standard. Among those who could read at least one word (excluding zero scores), students read 26.9 words per minute. By the end of Year 1, students read 7.6 words per minute on average (See Table 13). The average score increases to 16.9 by the end of Year 2, and to 38.1 by the end of Year 3. Girls read an average of 25.7 correct words per minute, 11 more correct words per minute than boys who read an average of 14.4 correct words per minute. Oral passage reading results were higher than the familiar word reading sub-test. One reason could be that it was easier for students to read words in the context of a story rather than reading words in isolation. Table 13: Sub-test 6 Oral Passage Reading: Results by Year and Gender Sub-test 6a – Number of correct words read per N Mean SD Min Max minute (CWPM) CWPM – overall 593 20.3 25.9 0 124.6 CWPM – minus zero score students 446 26.9 26.7 1 124.6 Year CWPM – overall 201 7.6 11.1 0 71.4 Year 1 CWPM – minus zero score students 133 11.5 11.9 1 71.4 CWPM – overall 209 16.9 22.3 0 104.5 Year 2 CWPM – minus zero score students 155 22.8 23.1 1 104.5 CWPM – overall 183 38.1 31.1 0 124.6 Year 3 CWPM – minus zero score students 158 44.1 29.2 1 124.6 Gender CWPM – overall 311 25.7 28.2 0 124.6 Girls CWPM – minus zero score students 256 31.2 28.2 1 124.6 CWPM – overall 282 14.4 21.6 0 108 Boys CWPM – minus zero score students 190 21.3 23.4 1 108 Average results by region illustrated in Table 14 indicate that the southern region had the highest fluency rates (35.9 cwpm) while Funafuti had the lowest (17.3 cwpm). Table 14: Oral Passage Reading: Results by Region and Year Central Funafuti North South Year 1 9.2 5.6 8.8 18.7 Year 2 15.7 15.3 22.1 28.5 Year 3 42.7 31.9 46.1 56.6 Overall 19.5 17.3 27.1 35.9 27 Figure 6: Oral Passage Reading: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 10 20 30 40 50 60 Number of correct words per minute 3.7. Sub-test 7 – Reading Comprehension The reading comprehension subtask measures the ability to answer several comprehension questions based on the passage read. There are a total of five questions for this sub-test, consisting of direct, fact-based questions and at least one question requiring inference from the passage read. Students were asked questions only up to the point where they had stopped reading. For instance, if the child read the first sentence (6 words), s/he was asked one question. If s/he read half of the text (27 words), s/he was asked three questions; and, if s/he read all five sentences (54 words), s/he was asked all five comprehension questions. Similarly, if learners did not read any of the text, they were not asked any questions and received a score of zero. This sub-test is scored by the number of questions answered correctly. The scores are also calculated to determine the percentage of questions answered correctly out of a total possible (five). The overall results in Table 15 indicate very low reading comprehension levels across all years. On average, students could not respond to at least one comprehension question based on the passage read. The mean score was 0.8 for all years and 2.7 for readers (minus zero scores). The large disparity between overall mean scores and mean scores minus zero denotes a high proportion of zero scores on this sub-test. The highest percentage of students scoring zero was in Years 1 and 2 (see Figure 7), where the mean scores were 0.1 and 0.6, respectively. These scores are explained by the low percentage of readers on earlier reading sub-tests (familiar reading, nonword reading, and oral reading passage). On average, Year 3 students were able to correctly answer nearly two (36%) of the five questions asked, and three questions (58%) when zero score students were excluded. Therefore, the breakthrough period for reading comprehension appears to begin in Year 3. Only 5 percent of the entire sample read the entire passage and were able to correctly respond to at least 80 percent of the comprehension questions (4 out of 5 questions asked). 28 Table 15 Sub-test 6b Reading Comprehension: Results By Year and Gender Sub-test 6b – Number of questions N Mean SD Min answered correctly Max Number of correct answers – 593 0.8 1.4 0 5 overall Number of correct answer – 177 2.7 1.4 1 5 minus zero score students Number of correct answer – 201 0.1 0.4 0 3 overall Year 1 Number of correct answer – 17 1.4 0.7 1 3 minus zero score students Number of correct answer – 209 0.6 1.3 0 5 overall Year 2 Number of correct answer – 50 2.6 1.5 1 5 minus zero score students Number of correct answers – 183 1.8 1.8 0 5 overall Year 3 Number of correct answers – 110 2.9 1.4 1 5 minus zero score students Number of correct answers – 311 1.0 1.5 0 5 overall Girls Number of correct answer – 120 2.6 1.3 1 5 minus zero score students Number of correct answers – 282 0.6 1.3 0 5 overall Boys Number of correct answers – 57 2.8 1.5 1 5 minus zero score students Figure 7 below shows the percentage of correct answers by Year. About 90 percent of students in Year 1, nearly 80 percent in Year 2 and 40 percent in Year 3 showed zero percent comprehension of the passage read. In Year 1, two percent of students could comprehend 40 percent of the text and only one percent could comprehend 60 percent. The results for Year 2 are slightly better as four percent of students could comprehend 100 percent of the text and five percent could comprehend at least 60 percent. Year 3 students had more distribution across the categories. One-third of Year 3 students could comprehend at least 60 percent of the text and 20 percent could comprehend 80 percent or more. Overall, the majority of students in all three years have reading comprehension levels below 60-80 percent. 29 Figure 7: Reading Comprehension: Distribution of percentage of correct answers by year 100% 90% 80% 70% 60% %students 50% 40% 30% 20% 10% 0% 0% 20% 40% 60% 80% 100% Reading comprehension (% correct responses) Class1 Class2 Class3 Although girls read 11 more words in the passage than boys, the point difference between boys and girls’ reading comprehension scores was trivial. Table 15 shows that girls on average correctly responded to 0.4 more questions than boys (1.0 versus 0.6, respectively). However, the distribution of responses by percentage of correct answers illustrated in 30 Figure 8 shows that girls had a higher percentage of correct answers in all categories and a lower percentage of zero scores compared to boys. Figure 8: Reading Comprehension: Distribution of percentage of correct answers by gender 90% 80% 70% 60% %students 50% 40% 30% 20% 10% 0% 0% 20% 40% 60% 80% 100% Reading Comprehension (% correct responses) Girls Boys The average results by year and gender according to the type of question asked and the order in which it was presented is provided in Table 8. Questions 1, 2, 3, and 5 are direct factual questions. Question 4 was slightly more complicated as it required the student to combine facts from two separate sentences to arrive at the correct response. The easiest question was Question 1, in which nearly one-third of students accurately responded. Questions 2-5 had fairly equal levels of difficulty with correct answers averaging 13 percent. Table 8: Correct Answers by Questions in sub-test 6b Questions Year 1 Year 2 Year 3 Total Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total 1. What does Tui 12% 0% 0% 32% 11% 22% 64% 43% 54% 35% 18% 27% like doing? 2. What is the 3% 0% 0% 15% 8% 11% 29% 20% 25% 15% 9% 12% name of Tui’s friend? 3. When did the 3% 0% 0% 9% 6% 8% 40% 25% 33% 16% 10% 13% two of them go to play soccer? 4. Why did Tui 0% 0% 0% 15% 8% 12% 40% 25% 33% 18% 11% 14% play in/with Seti’s boots? 5. Which team 0% 0% 0% 10% 8% 8% 40% 22% 31% 16% 9% 13% won? 31 The percentage and number of questions students answered correctly based on those attempted are presented in Table 16. The first column shows number and percent correct for the entire sample. Based on these results, the majority of students (69%) were not able to answer one comprehension question correctly. Columns 0-5 show the number of questions that students attempted and scored correctly. Of 387 students who scored zero, 135 were not asked any questions because they had scored zero on the oral reading passage test. Of those who attempted at least two questions, 96 percent scored zero. Nearly half (46%) of students who attempted three questions also scored zero. Therefore, the more text students read, the more they were able to comprehend. Only 5 percent of all students attempted and correctly responded to four questions, and 4 percent attempted and correctly responded to five questions. Table 16: Distribution of Number of Correct Answers Number of Total Number of questions attempted Questions correct Correct n 0 1 2 3 4 5 0 387 135 35 174 40 2 1 69% 100% 97% 96% 46% 6% 1% 1 43 1 7 27 6 2 7% 3% 4% 31% 18% 2% 2 41 1 17 11 12 7% 0.6% 19% 33% 14% 3 36 3 9 23 6% 5% 28% 27% 4 28 5 23 5% 15% 27% 5 24 23 4% 28% Total (N) 559 135 36 182 88 33 85 Scores were similarly low on this sub-test across all regions and classes. The southern region performed slightly better than the rest with a mean comprehension score of 1.5 and the central region had the lowest comprehension score (0.6). Funafuti have also a low comprehension score (0.7). Table 17: Reading Comprehension: Results by Region and Year Central Funafuti North South Year 1 0.1 0.1 0.1 0.3 Year 2 0.5 0.6 1.0 0.7 Year 3 1.8 1.5 2.1 3.2 Overall 0.6 0.7 1.1 1.5 32 Figure 9: Reading Comprehension: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 0.5 1 1.5 2 2.5 3 3.5 Number of questions answered correctly 3.8. Sub-test 8 – Listening Comprehension The purpose of the listening comprehension assessment is to measure whether the student can listen to a short passage being read aloud and then answer several questions correctly with a word or a simple statement. Poor performance on a listening comprehension tool would suggest that children simply do not have the basic vocabulary that the reading materials expect, or that they have difficulty processing what they hear. The assessor read a short fable to students and then asked five comprehension questions. Students had 15 seconds to respond to each question. As this was an untimed test, all students heard the entire story and responded to all five questions. For this reason, scores are based on the percentage of questions answered correctly. Students performed exceptionally well on this sub-test. Students correctly responded to over 60 percent of the questions asked (See Table 18). The results indicate progressive improvements in listening comprehension levels between 8-12 percent each year. By the end of Year 1, students comprehended 54 percent of the passage read aloud. Year 2 students comprehended 62 percent and Year 3 students understood nearly three-quarters of the story. Boys and girls had equal listening comprehension skills (63 percent for girls and 63 percent for boys). Table 18: Sub-test 7: Listening Comprehension: Results by Year and Gender Sub-test 8 – Percentage of overall listening N Mean SD Min Max comprehension CLPM – overall 593 64% 26.2 0 100% CLPM – minus zero score students 562 67% 22.2 20% 100% Year Year 1 CLPM – overall 201 54% 26.8 0 100% 33 CLPM – minus zero score students 183 59% 21.5 20% 100% CLPM – overall 209 62% 26.9 0 100% Year 2 CLPM – minus zero score students 198 66% 23.1 20% 100% CLPM – overall 183 74% 20.6 0 100% Year 3 CLPM – minus zero score students 181 75% 19.2 20% 100% Gender CLPM – overall 311 63% 24.4 0 100% Girls CLPM – minus zero score students 300 65% 21.6 20% 100% CLPM – overall 282 63% 28.2 0 100% Boys CLPM – minus zero score students 262 68% 22.9 20% 100% Table 19 shows average performance per question type. Questions 1, 3 and 4 were fact- based questions (answers can be found within the story). Questions 2 and 5 required use of inferential skills. Students scored well above 60 percent on both factual and inferential questions with the exception of Question 4, which asked students to recall the name of the fish. One possible explanation is that the word is not a familiar term. Table 19: Listening Comprehension Percentage of Correct Answers by Question Questions Year 1 Year 2 Year 3 Total Girl Boy Total Girl Boy Total Girl Boy Total Girl Boy Total 1. Where did Meli 68% 64% 66% 76% 74% 75% 85% 86% 86% 76% 74% 75% and her father go fishing? 2. Why did Meli 49% 49% 49% 56% 65% 60% 79% 84% 81% 61% 66% 63% call out? 3. Who helped 87% 73% 80% 86% 79% 83% 97% 91% 94% 90% 81% 85% Meli? 4. What is the 10% 13% 11% 15% 28% 22% 24% 32% 28% 16% 24% 20% name of the fish that Meli and her father caught? 5. Why was Meli 63% 64% 64% 74% 67% 70% 78% 85% 81% 71% 72% 72% happy? All regions performed equally well on the listening comprehension sub-test (See Table 20). Students across all regions could correctly respond to at least three of the five questions asked. The highest performance was in the southern region where students correctly responded to 3.5 questions and lowest performance was in Funafuti and central region where students accurately responded to 3.1 questions. The northern region did well with 3.4 average scores. 34 Table 20: Listening Comprehension: Results by Region and Year Central Funafuti North South Year 1 2.4 2.8 2.7 3.1 Year 2 3.3 2.9 3.5 3.4 Year 3 3.8 3.6 3.9 3.8 Overall 3.1 3.1 3.4 3.5 Figure 10: Listening Comprehension: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 1 2 3 4 5 Number of questions answered correctly 3.9. Sub-test 9 – Dictation The dictation sub-test measures students’ alphabet knowledge and ability to hear and distinguish individual letter sounds in words and to spell words correctly. For this sub-test, the assessor read aloud a short sentence of 10 words and asked students to write down what they had heard. The assessor read the sentence three times, once before students began writing and twice while they were writing. The test was untimed and scored by the percentage of items written correctly. The dictation sub-test assessed four basic writing skills: spelling, spacing, capitalization and punctuation (use of commas and full stops). The dictation sentence contained 10 words, consisting of short high-frequency words and more complex words. To assess punctuation, it included a comma. These features were found to give a good spread of scores across the years tested. The scores were calculated using weights to arrive at a maximum score of 100 percent. The spelling items received a weight of 60 percent while other components (spacing, capitalization, comma and full stop) received a weight of 40 percent. Given the focus of the EGRA is to assess reading ability, the spelling component received more weight. 35 Similar to Listening Comprehension, students performed exceptionally well on this writing task. The maximum score was 91 percent in Year 1 and 100 percent in Years 2 and 3. The average score for the entire sample was 54 percent correct and 58 percent excluding zero scores. Students’ writing skills improved substantially each year (See Table 21). Scores increased about 20 percent each year -- from 33 percent in Year 1 to 52 percent in Year 2 to an average score of 71 percent in Year 3. Girls scored 10 percent higher than boys (59 percent compared to 49 percent for boys). Table 21: Sub-test 9 Dictation: Results by Year and Gender Task 8 – Percentage of overall early writing N Percentage SD Min Max skills Mean Average percent of correct answer 404 54% 32% 0% 100% – overall Average percent of correct answer 381 58% 30% 2.9% 100% – minus zero score students Year Average percent of correct answer 98 30% 27% 0% 91.4% – overall Year 1 Average percent of correct answer 90 33% 27% 2.9% 91.4% – minus zero score students Average percent of correct answer 145 52% 30% 0% 100% – overall Year 2 Average percent of correct answer 134 57% 27% 2.9% 100% – minus zero score students Average percent of correct answer 161 71% 26% 0% 100% – overall Year 3 Average percent of correct answer 157 73% 23% 5.7% 100% – minus zero score students Gender Average percent of correct answer 237 59% 30% 0% 100% – overall Girls Average percent of correct answer 229 61% 29% 2.9% 100% – minus zero score students Average percent of correct answer 167 49% 33% 0% 100% – overall Boys Average percent of correct answer 152 53% 31% 2.9% 100% – minus zero score students As presented in Figure 11 and mentioned above, the dictation test measured spelling, spacing, comma and full stop. Students demonstrated progressive improvements for each category from Year 1 to Year 3. The greatest improvements were with use of the comma between Years 2 and 3 (increase of 26 percent) and with use of the full stop between Years 1 and 2 (increase of 23 percent) and between Years 2 and 3 (increase of 25 percent). The skill students struggled with most was correctly using a comma in a sentence. Girls scored 36 an average of 6 percent to 8 percent higher than boys across all writing categories (Figure 14). Figure 11: Dictation Results: Percentage of Correct Answers by Item and Year 80% 70% 60% 50% 40% 30% 20% 10% 0% Spelling Spacing Comma Full stop Class1 Class2 Class3 Figure 12: Dictation Results: Percentage of Correct Answers by Item and Gender 60% 50% 40% 30% 20% 10% 0% Spelling Spacing Comma Full stop Girls Boys 37 Regional results for the dictation sub-test are presented in Table 22. Students fared well across all regions with average scores between 51.3 percent and 64.8 percent correct. The northern and southern regions performed slightly better with 62.8 percent and 64.8 percent correct, respectively. The lowest scoring region was Funafuti and Central where students achieved 51.3 percent correct. Table 22: Dictation: Results by Region and Year Central Funafuti North South Year 1 36.2 23.4 40.9 40.8 Year 2 48.5 52.2 54.5 62.6 Year 3 72.0 67.2 76.9 82.1 Overall 52.4 51.3 62.8 64.8 Figure 13: Dictation: Results by Region and Year Central Funafuti Year 1 North Year 2 Year 3 South 0 20 40 60 80 100 Dictation precent correct score 38 3.10. Summary of assessment results The overall EGRA results showed that the majority of Tuvaluan students are not yet fluent readers. Students possess good listening comprehension and basic reading and writing skills, which are precursors to reading. On the timed reading fluency sub-tests, students scored highest on letter name, letter sound and oral reading passage. Students struggled with identifying initial sounds of words and with decoding familiar and unfamiliar words. Students could only identify an average of 5 initial sounds, 12 familiar words, and 8 nonwords. Oral reading fluency scores were much higher averaging 20 correct words per minute (cwpm), which suggests that students were better able to read words in context rather than in isolation. Despite relatively higher scores on the oral reading fluency passage, the majority of students in Years 1-3 were unable to read fast enough to comprehend a simple reading passage. The reading comprehension sub-test had the highest percentage of zero scores across all years as well as the lowest mean scores (See Figures 16 and 17). Over 90 percent of Year 1 students and nearly 80 percent of Year 2 students could not accurately respond to one comprehension question. Students did, however, show improvements as they progressed throughout the levels. By the end of Year 3, students could read an average of 38 correct words per minute and could accurately respond to 35 percent of the comprehension questions asked (See Figures 14 and 15). With regards to gender, there were more female readers than male readers, and girls read better than boys in all reading skills tested. Performance across regions varied. The northern and southern regions tended to perform better than the central and Funafuti regions. However, the rankings for each sub-test were erratic so there was no discernible trend to explain regional differences. Figure 14: Summary Results: Number correct answers for timed sub-tests by year 45 Number of correct answers per minute 40 35 30 25 20 15 Year 1 10 Year 2 5 Year 3 0 Letter Familiar Oral Letter Name Sound Nonword Word Passage Knowledge Identificatio Reading Reading Reading n Year 1 15.5 16.4 4.5 2.8 7.6 Year 2 21.3 19.7 11 6.9 16.9 Year 3 25.3 29.4 24 16.4 38.1 39 Figure 15: Summary Results: Percentage of correct answers for untimed sub-tests by Year 80% Percentage of correct answer 70% 60% 50% 40% 30% Year 1 Year 2 20% Year 3 10% 0% Reading Listening Comprehension Comprehension Year 1 2.4% 54.1% Year 2 12.2% 62.1% Year 3 35.3% 74.1% Figure 16: Summary Results: Percentage of Zero Scores by Sub-test and Year 100% 90% Percent of Zero Scores 80% 70% 60% 50% 40% 30% 20% 10% 0% Oral Letter Initial Letter Familiar Nonwor Reading Listenin Dictatio Passage Name Sound Sounds Word d Comp. g Comp. n Reading Year 1 24% 24% 22% 58% 78% 34% 92% 9% 8% Year 2 16% 14% 17% 40% 60% 26% 76% 5% 8% Year 3 8% 10% 15% 18% 30% 14% 40% 1% 6% 40 Figure 17: Summary Results: Percentage of Zero Scores by Sub-test and Gender Dictation Listening Comp. Reading Comp. Oral Passage Reading Nonword Familiar Word Letter Sounds Initial Sound Letter Name 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Boys Girls 41 Chapter 4 – Performance in reading fluency and reading comprehension According to the second edition of the EGRA toolkit17, there are two steps to identifying a reading fluency benchmark. The first is to identify the level of reading comprehension that is expected for the grade level. In most countries, this level is set at 80 percent or higher (4 correct answer out of 5 questions) for Years 1-3. This threshold will be used for the purposes of this EGRA study. Once the reading comprehension benchmark is set, the second step is to use EGRA data to show the range of ORF scores obtained by students able to achieve the desired level of comprehension. Because students will achieve 80 percent comprehension at different fluency rates, and there could be a wide range of scores, we have added a third step to the analysis. The third step is to identify the range of scores with the highest proportion of students meeting the comprehension benchmark. With this information, then stakeholders may decide on the value within the fluency range that should be put forward as the reading fluency benchmark. Table 23 shows the actual distribution of percent correct scores for reading comprehension. Overall, 5 percent of students scored 80 percent correct and 4 percent achieved 100 percent. Hence, the total percentage of students achieving 80 percent or more reading comprehension is 9 percent. Table 23: Distribution of percent correct scores for reading comprehension by sub-test per year Comp. Score Year 1 Year 2 Year 3 Overall Mean 0% 91.5% 76.1% 39.9% 70% 20% 5.9% 8.6% 8.7% 8% 40% 1.5% 3.8% 16.9% 7% 60% 1% 4.3% 13.7% 6% 80% 0% 3.8% 11.5% 5% 100% 0% 3.4% 9.3% 4% Total 100% 100% 100% 100% As mentioned above, as a first step, we decided to set the level of comprehension at 80 percent as an indicator that students demonstrate full understanding of the text. Table 25 shows the percentage of students by Year and gender that comprehended 80 percent or more of the text read. The distribution of scores by Year is: 0 percent in Year 1, 7 percent in Year 2 and 21 percent for Year 3 students. In terms of gender, 11 percent of female students and 7 percent of male students reached the 80 percent benchmark in reading comprehension. 17 RTI International. (2016). Early Grade Reading Assessment (EGRA) Toolkit, Second Edition. Washington, DC: United States Agency for International Development. 42 Table 24: Percent of students reading with 80% or more comprehension by Year 80% or more reading comprehension Mean (%) SD (%) N Overall 9% 28.6% 593 Year 1 0% - 201 Year 2 7% 25.8% 209 Year 3 21% 40.7% 183 Female 11% 30.8% 311 Male 7% 25.7% 282 For the second step, we need to look at the distribution of fluency scores of students who reached the threshold level in reading comprehension. Figure 18 shows the distribution of ORF scores for each level of reading comprehension. Globally, we could see that an increase in level of reading comprehension is associated with an increase in ORF scores. Table 26 confirms that mean scores in ORF were higher when students had higher levels of reading comprehension. However, we could also see that some students who did not reach the 80 percent reading comprehension benchmark could achieve higher ORF scores than students who did reach the benchmark. For example, students who have 80 percent in reading comprehension could have a fluency score as low as 44 correct words per minute. Meanwhile, some students who were unable to answer a single reading comprehension question correctly could read up to 5518 correct words per minute. 18 Note that ORF scores may exceed the total number of words in a passage since it is a timed test. If students could read the entire passage within the 60 seconds allowed, the fluency score would take into account the time remaining from the 60 seconds. This explains why some students present such high value as 129 correct words per minute although the passage had only 60 words. 43 Figure 18: ORF distribution by reading comprehension level Table 25: ORF score by reading comprehension score Reading Comp. Mean Fluency SD Min Max Sample (n) Score Score 0% 9.7 8.1 1 55 269 20% 31.8 11.8 3 57 46 40% 45.4 13.6 25 91 42 60% 57.4 15.7 21 108 36 80% 68.0 17.8 44 112 29 100% 84.2 21.4 51 125 24 Distribution of ORF scores for students who reach 80 percent in reading comprehension Figure 19 shows the distribution of ORF scores for the 53 students who reached the level of 80 percent correct in reading comprehension. The majority of the students who met the benchmark read between 50 and 79 correct words per minute. However, scores ranged between 40 and 129 words per minute. Scores above 60 indicate that students finished the paragraph in less than one minute while scores below 60 imply that students who did not finish the paragraph were able to infer enough to correctly respond to all questions. 44 Figure 19: Distribution of ORF scores for students reading with at least 80% comprehension The histogram in Figure 20 compares the distribution of ORF scores for students who did and did not meet the reading comprehension benchmark. As illustrated, the number who reached the benchmark is quite small compared to those who did not and there is overlap between the distributions of the two groups. The benchmark set for reading fluency should ensure that a maximum number of students who did reach the 80 percent level in reading comprehension would be identified as fluent readers but it should also ensure not to include too many students who did not reach 80 percent in reading comprehension. This means that setting the benchmark value for fluency at 45 correct words per minute could include too many students who didn’t reach the 80 percent reading comprehension level while setting it at 75 correct words per minute would exclude too many students who reached the benchmark. 45 Figure 20: Distribution of ORF scores for students meeting and not meeting 80% benchmark Table 26 shows that the fluency rate of 70 cwpm has a greater proportion of students meeting rather than not meeting the benchmark. Policymakers may decide on an acceptable range (e.g., 70-75 cwpm) that can be considered proficient or acceptable. If the benchmark for reading comprehension were lowered to 60 percent, then the fluency scores would also be lower (closer to 50 cwpm). Table 26: Distribution of ORD Scores for students meeting and not meeting 80% benchmark Correct number of % meeting 80% % not meeting 80% words per minute (ORF) RC benchmark RC benchmark 45-49 5.9% 94.1% 50-54 33.3% 66.7% 55-59 54.6% 45.5% 60-64 50% 50% 65-69 50% 50% 70-74 55.6% 44.4% 75 and more 79.3% 20.7% In summary, greater oral reading fluency is associated with higher levels of reading comprehension with only 9 percent of students who are identified as being able to comprehend 80 percent of what they read. More females than males reached that level of 46 comprehension. ORF scores for students achieving the benchmark shows great variability. Stakeholders should discuss the number of correct words per minute that could qualify a student as a fluent reader. The decision should be based on the distribution of scores for students who reached the reading comprehension benchmark. 47 Chapter 5- Analysis of Student and Teacher Factors Associated with Better Reading Outcomes This section reports the results 19 of regression analyses conducted to explore the association of student and teacher factors (independent variables) on oral reading fluency scores20 (dependent variable). The factors are organized into five categories: 1) student demographics; 2) teacher characteristics; 3) teacher expectations; 4) classroom environment; 5) teaching training and guides; 6) reading materials; and, 7) teacher instructional and assessment methods. The results of the analyses directly respond to research question #7. 5.1. Association of student characteristics to student reading outcomes General background characteristics and reading activities were collected in the student questionnaire. The factors as shown in Table 27 include whether the student attended preschool, ate breakfast before arriving to school, language spoken at home, parents’ literacy, whether students receive help with homework, availability of reading materials and whether students read or are read to at home. Over 90 percent of students surveyed attended pre-school, speak Tuvaluan at home, and state that they like to read. The majority of students (89%) eat breakfast before arriving to school. Eighty percent of students read at home. Over half of students have reading materials in Tuvaluan or English. Table 27: Student background characteristics Student Characteristics % of cases N Student attends preschool before Year 1 97% 578 Student speaks Tuvaluan at home 91% 593 Student eats before arriving to school 89% 580 Student has a literate mother 59% 593 Student has a literate father 48% 593 Student has both parents literate 41% 593 Student has literate sibling 30% 593 Student receives help with homework from the mother 40% 593 Student receives help with homework from the father 20% 593 Student receives help with homework from the sibling 9% 593 Student receives help with homework from any other 9% 593 person Someone asks student about what he/she did in school 81% 572 Student tells someone at home when he/she gets good 81% 554 marks 19 As the results reflect a population not a sample, there is no need to identify where results are statistically significant. Any difference observed are real differences in this population. 20 ORF scores were selected as the outcome measure given the low comprehension scores. Results of reading comprehension analysis were similar (See Annex D: Tuvalu – Regression Analysis with Reading Comprehension as Outcome) 48 Student has books, newspapers or other things to read at 63% 583 home Student has books, newspapers or other things to read at 47% 593 home in Tuvaluan Student has books, newspapers or other things to read at 27% 593 home in English Someone reads to student at home 70% 593 Student reads aloud to someone at home 64% 593 Student reads to himself/herself at home 80% 593 Student reads on a computer or mobile device at home 42% 593 Student likes to read 92% 583 The three factors that had the greatest positive impact on ORF scores (with an increase of 10 correct word per minute) were: whether students attended pre-school, whether the student reads to himself/herself at home, and whether the student likes to read. In terms of availability of reading materials, interestingly, having reading materials in English had a higher positive impact on reading performance (increase of 6.98 words per minute) compared to having reading materials in Tuvaluan (increase of 1.32). This seems counterintuitive thus further research is necessary to understand the characteristics of families that have reading materials in English at home to explain this association. Parents’ literacy presented a positive association with ORF scores. When both parents are literate, students read 5.65 more words. Students who had a literate father read twice as many words as students with a literate mother (6.24 more words per minute compared to 3.54 more words). However, this difference was negligible when mothers helped students with their homework (6.14 more words compared to 3.18 for fathers). Students who received help from siblings read five fewer words per minute. Table 28: Association of student characteristics to Oral Reading Fluency (ORF) scores Student Characteristics Change in Rsqr ORF Score (+/-) Student attends preschool before Year 1 10.85 0.29 Student eats breakfast before arriving to school 0.22 0.27 Student speaks Tuvaluan at home 3.91 0.27 Student has a literate mother 3.54 0.27 Student has a literate father 6.24 0.29 Student has both parents literate 5.65 0.29 Student has a literate sibling 1.81 0.27 Student receives help with homework from 6.14 0.28 the mother Student receives help with homework from 3.18 0.27 father Student receives help with homework from -5.09 0.27 sibling Someone asks student about what he/she did 2.61 0.27 in school 49 Student tells someone at home when he/she 4.97 0.27 gets good marks Student has books, newspapers or other things 1.24 0.27 to read at home Student has books, newspapers or other things 1.32 0.27 to read at home in Tuvaluan Student has books, newspapers or other 6.98 0.29 things to read at home in English Student reads aloud to someone at home 3.16 0.27 Student reads to himself/herself at home 10.39 0.30 Someone reads to student at home -2.23 0.27 Student reads on a computer or mobile device -3.79 0.24 at home Student likes to read 10.06 0.28 5.2. Association of teacher characteristics to student reading performance The association between teachers’ characteristics and student reading performance on the ORF sub-test was explored using data collected in the teacher interview questionnaire. Teacher profiles are presented in Table 25. Teachers in Tuvalu have an average of 16 years of experience. Most of their experience seems to be on the island they are currently teaching. Nearly all primary school teachers (94%) are female and the majority holds a primary school teaching certificate. Eighty-five percent of teachers have a reading corner in the classroom and have met with their students’ parents. Finally, half of teachers have not been absent from school in the last term. Table 29: Profile of teachers in TuEGRA Teacher demographics Mean (sd) N Age of the teacher 43.1 (12.1) 34 Number of years of teaching experience 16.3 (10.8) 34 Number of years spent teaching on the island 11.8 (11.3) 34 Number of minutes from home to school 8.3 (5.1) 34 Teacher characteristics % of cases N Female 94% 34 Has a primary teaching certificate 85% 34 Has not been absent from school in the last term 47% 34 Has met with parents of his/her students 85% 34 In order to identify the teacher characteristics associated with better student reading outcomes, separate regression analysis was conducted for each teacher characteristic (see table 30). The dependent variable was the average number of word read per minute in the oral reading passage (scores from sub-test 6a) and the independent variables were the teacher characteristics described above (Table 29). Gender and year level were considered in the regression analysis to control for these two variables. 50 The results in Table 30 indicate that most factors did not have a statistically significant association with ORF scores. Whether teachers held a primary school teaching certificate showed the greatest effect. Students with primary certified teachers read four fewer words per minute than students whose teachers did not have a primary teaching certificate. 51 Table 30: Association of teachers' characteristics to Oral Reading Fluency Score (ORF) Teacher Characteristics Change in Rsqr ORF Score (+/-) Has a primary teaching certificate -4.44 0.73 Has not been absent from school in the last 0.51 0.72 term Has met with the parents of his/her students -1.38 0.73 Age of the teacher -0.24 0.75 Number of year of experience in teaching -0.16 0.73 Number of year of experience in teaching on -0.20 0.74 this island Number of minute from home to school 0.12 0.72 5.3. Association of teacher expectations to student reading performance Teachers were asked to report what grades they expected students to meet certain reading competencies (e.g., recognize letters, understand stories read aloud). Table 31 provides results of regression analyses that show the relationship of five indicators of teacher expectations on student oral reading fluency scores (number of words read correctly per minute). Each regression analysis included gender and year as control variables. The reference level for each regression is the median expectation of all teachers for each indicator. The dash (-) represents the expected year for that competency to be reached. For example, Year 1 is the reference category in the first regression because the median expectation of teachers for students to recognize and say letter names was by the end of Year 1. The positive coefficient of 5.27 in Year 2 indicates that students whose teachers expected them to recognize and say letter names by Year 2, a year later than the median expected time, read 5.27 more words relative to students whose teachers expected them to achieve the competency by Year 1. This pattern was also observed in the second regression. Most teachers expected their students to understand stories read aloud by the teacher by the end of Year 1. Teachers who expected their students to achieve this competency by the end of Year 2 read 8.76 more words per minute, and for those with expectations for Year 3 students, the ORF score increased by 13 words per minute. When teachers had higher expectations than the median teachers, reading performance also increased, but by a smaller margin than for teachers with low expectations. This was observed for the last two indicators. Students were expected to read aloud to the teacher and to understand simple sentences by the end of Year 2. Teachers who expected students to achieve these competencies by the end of Year 1 saw an increase of 1-2 correct words per minute while teachers with expectations for Year 3 saw an increase of 3-7 words per minute. 52 Table 31: Association of teacher expectations to oral reading fluency (ORF) scores Teacher expectations of student Change in Rsqr reading performance ORF score (+/-) Grade students should recognize and say letter names Year 1 - 0.74 Year 2 5.27 Year 3 6.64 Grade students should understand stories read aloud by the teacher in the classroom Year 1 - 0.80 Year 2 8.76 Year 3 13.33 Grade students should sound out words using phonics Year 1 - 0.75 Year 2 7.41 Year 3 6.40 Grade students should read aloud to teacher and other students Year 1 1.28 0.73 Year 2 - Year 3 3.70 Grade students should understand simple texts that they read in class Year 1 1.79 0.75 Year 2 - Year 3 6.96 Based on the results in Table 31, it appears that students whose teachers expected them to meet the competencies later read more words per minute than students whose teachers expected them to achieve the competency before the expected grade level. The greatest effect was when teachers expected students to achieve the competency in Year 3. When teachers expected students to recognize and say letter names in Year 3, students read 6.64 more cwpm. When teachers expected students to understand stories read aloud by the teacher in the classroom by Year 3, students read 13.3 cwpm. Additionally, when teachers expected students to understand simple text that they read in class by Year 3, students read 6.9 more cwpm. Therefore, the overall results of the regression analysis suggest that when teachers have lower expectations for students’ achieving the reading competencies, students’ performance on the ORF sub-test was higher. 5.4. Association of teaching training and guides to student reading performance We also sought to explore the relationship between teaching resources and student reading performance. Teachers were asked whether they had a teaching syllabus and whether they 53 had received any training on how to teach reading in the last three years. As shown in Table 28, over 90 percent of teachers had a syllabus for teaching reading in Tuvalu. All teachers reported having an English syllabus. However, only a small percentage of teachers (15%) have received training on reading instruction. Table 32: Teacher training and guides % of cases N Teacher has a syllabus 94% 34 Teacher has an English syllabus 100% 34 Teacher have receive training on how to teach reading in 15% 34 the last three years Results of the regression analysis for teaching resources and its association with student reading performance, measured by a change in ORF score are presented in Table 33. Whether the teacher had a syllabus for teaching reading in Tuvalu or English did not have any effect on ORF scores. However, whether teachers attended training on early grade reading instruction had a significant negative relation with ORF scores. The mean ORF score of students whose teacher attended a training workshop was 8.99 words lower than students whose teacher did not. This result does not necessarily imply that the training was inefficient. A negative association could be explained by the fact that teachers who have less competent student received the training. The regression did control for teacher experience, so there are many other external factors that could have affected students’ performance. Table 33: Association of training and guides to student oral reading fluency (ORF) scores Change in Rsqr ORF score Teacher have a syllabus 0.72 0.72 Teacher have an English syllabus - Teacher have receive training on how to teach -8.99 0.76 reading in the last three years 5.5. Association of Classroom Environment to Student Performance Through classroom observations, we collected data on the classroom environment and types of resources available in the classroom. Assessors recorded whether they observed the following classroom displays: the alphabet, a calendar, days of the week, numbers and the weather. Classrooms observed had an average of 3.63 classroom displays. The observation also sought to determine the types of printed materials used in instruction, such as newspapers, magazines, flashcards, food wrappers and packaging, prepaid cards, objects in treasure boxes and any other materials. There was an average of 0.97 printed material used in the classroom. Table 34: Average Number of Classroom Displays and Materials Observed Classroom environment Mean (sd) N 54 Classroom displays 3.63 (1.5) 36 Print materials used in instruction 0.97 (1.3) 36 As shown in Table 35, 81 percent of classrooms have spelling/vocabulary words displayed, 19 percent have songs/hymns/stories written on the blackboard and 69 percent have them written on charts or posters, and 69 percent have student work displayed. Additional factors related to the classroom environment that were observed included whether there was sufficient space for organizing group activities, whether there was a reading corner in the classroom, whether teachers maintained folders with students’ work (student profiles) and the seating arrangement. The results showed that 75 percent of classrooms have a reading corner, 67 percent have sufficient space for organizing group work, 6 percent have student profiles and 100 percent of classroom students sit on the floor. Table 35: Frequency and Type of Classroom Displays/Resources Available Types of classroom displays/resources available % of classes N Spelling/vocabulary displayed 81% 36 Song/hymns/stories displayed on blackboard 19% 36 Song/hymns/stories displayed on charts/posters 69% 36 Student work displayed 69% 36 Sufficient classroom space for organized group activities 67% 36 Reading corner in the classroom 75% 36 Student profiles (folder with student work and student info) 6% 36 Students sitting on floor 100% 36 Among the different classroom environment variables, the presence of student profiles had the most positive relation with ORF scores. When teachers maintained student profiles, the students read an average of 10.43 more words per minute. This relation is statistically significant. Having a reading corner in the classroom and sufficient space for organizing group activities was associated with an increase of 5 cwpm. Table 36: Association of Classroom Environment to Student ORF Scores Classroom Environment Change in Rsqr ORF Score (+/-) Classroom displays 0.70 0.74 Spelling/vocabulary displayed 1.76 0.74 Song/hymns/stories displayed on blackboard 3.29 0.74 Song/hymns/stories displayed on 0.38 0.74 charts/posters Student work displayed 3.47 0.75 Print materials used in instruction 1.50 0.75 Sufficient classroom space for organized group 4.99 0.76 activities Reading corner in the classroom 5.22 0.76 Student profiles (folder with student work and 10.43 0.76 student info) 55 Students sitting on floor n/a n/a 5.6. Association of Reading Materials to Student Performance The classroom observation further aimed to identify reading materials used for reading instruction. The instrument included 8 different types of reading materials that could be observed in the classroom (Spark readers, Tuvaluan readers, school journals, Tulip reader, Sunshine readers, Pacific folk tales, Big books, and any other reading material). The assessors reported a mean of 4.11 different reading materials used in the classrooms. Additionally, there was an average of 1.97 reading books observed during the lesson. Table 37:Average Number of Reading Materials and Books in Classroom Resources Mean (sd) N Reading materials in classroom 4.11 (1.1) 36 Number of reading books used during the lesson 1.97 (4.7) 35 According to the regression results in Table 38, the relation between the reading materials or number of books used during the lesson and ORF scores is not very important. Table 38: Association of Reading Instructional Resources to Student ORF Scores Language use in classroom Change in Rsqr ORF Score (+/-) Reading materials in classroom 1.09 0.74 Number of reading books used during the 0.02 0.74 lesson 5.7. Association of Teaching and Assessment Methods to Student Performance The final set of regression analysis observed the relationship between teaching and assessment methods and student performance in ORF. Table 40 presents results of regressions analysis of the frequency in which students and teachers performed 13 teaching and assessment methods within the course of the week. The dependent variable is the mean ORF score of the students and the independent variables are the instructional methods. The reference category used for every regression model is zero frequency, meaning the task never occurred. The mean ORF score of students who were never exposed to an instructional or assessment method was compared to the mean ORF score of students who were exposed 1-2 days, 3-4 days or 5 days a week. Results from those regression analyses were analyzed to determine if being exposed to a specific method is associated with increased or decreased ORF scores and if so, whether the effect is the same for all rates of exposure (e.g., 1-2 days, daily, etc.). All regression models include gender and Year for controlling for those two characteristics. Table 39 shows the teaching methods used during reading instruction and the frequency in which it is applied. The instructional methods used most frequently were teaching new letter sounds and the meaning of new words/vocabulary. The most common classroom activities were whole class reading, listening comprehension and reading comprehension 56 activities. Less than 10 percent of teachers evaluate students’ oral reading fluency on a regular basis (3-5 days a week). 57 Table 39: Frequency of methods used during reading instruction Teaching and assessment methods Never 1–2 3–4 5 days N days days Teaching of Listening Comprehension 12% 38% 18% 32% 34 Low order book orientation21 24% 47% 18% 12% 34 Asking children to orally retell a story 26% 35% 15% 24% 34 that they have read Teaching new letter sounds 12% 12% 15% 62% 34 Asking children to sound out unfamiliar 29% 26% 18% 26% 34 words using knowledge of letter sounds Teaching the meaning of new 9% 32% 15% 44% 34 vocabulary words Whole class reading 9% 32% 18% 41% 34 Guided reading 35% 41% 15% 9% 34 Listening to a child read aloud 21% 41% 26% 12% 34 Students reading silently 35% 29% 9% 26% 34 Reading comprehension activities 21% 32% 15% 32% 34 Children take books home to read with 59% 29% 3% 9% 34 their parents Evaluating student’s oral reading with 68% 15% 9% 9% 34 running records or any other method Table 40: Association of teacher instructional and assessment methods to student performance Teaching instructional and assessment Change in Rsqr methods ORF score Teaching of Listening Comprehension 0.78 1-2 days 7.34 3-4 days -2.63 Daily 0.68 Low order book orientation 0.72 1-2 days 2.23 3-4 days 0.27 Daily 0.35 Asking children to orally retell a story that 0.74 they have read 1-2 days 3.67 3-4 days -2.53 Daily -0.84 Teaching new letter sounds 0.72 1-2 days 13.79 3-4 days 3.50 21 Low Order Book Orientation means being able to identify the parts of a book: a cover, top/bottom, title, author, words, and sentences. 58 Daily 3.55 Asking children to sound out unfamiliar 0.79 words using knowledge of letter sounds 1-2 days 5.72 3-4 days -4.75 Daily 7.25 Teaching meanings of new 0.75 words/vocabulary 1-2 days 14.7 3-4 days 6.59 Daily 8.58 Whole class reading 0.74 1-2 days 7.25 3-4 days 3.08 Daily 2.39 Guided reading 0.75 1-2 days 6.57 3-4 days 4.36 Daily 6.56 Listening to a child read aloud 0.76 1-2 days 4.47 3-4 days 1.18 Daily 10.87 Students readings on their own silently 0.78 1-2 days 9.72 3-4 days 6.42 Daily 2.57 Reading comprehension activities 0.72 1-2 days 1.45 3-4 days 3.18 Daily 2.79 Children take books home to read with 0.73 their parents 1-2 days 1.32 3-4 days 10.65 Daily -2.74 Evaluating student’s oral reading with 0.72 running records or any other method 1-2 days -0.89 59 3-4 days 1.30 Daily -2.46 Results of the regression analysis presented in Table 40 show that some instructional methods used in reading classroom are associated with higher levels of ORF scores. Teaching new vocabulary words and letter sounds produced the highest gains in students’ ORF score. Students who learn new vocabulary words 1-2 days a week read an average of 14.7 more words. Students who were taught letter sounds 1-2 days a week read an average of 13.79 more words. Reading aloud daily or silently improved scores by 6-10 words per minute. Students who read aloud to the teacher or class on a daily basis read an average of 10.87 more words. Students who read silently in the classroom at least 1-2 days a week read an average of 9.72 more words. Two methods when used on a daily basis produced a negative result on ORF scores: children taking books to read with their parents and evaluating students’ oral reading. If students brought books home to read 3-4 days a week, students read 10 more words; however, if they brought books home daily, they read -2.74 words. This suggests that the more often they bring books home, the less they read. Assessing oral reading fluency on a daily basis resulted in a decreased ORF score of 2.4. There was a slight increase in the score (1.3) if assessment was conducted 3-4 days. 60 Chapter 6 - Conclusions and Recommendations The TuEGRA census study confirm that students are lagging behind in literacy (as reflected in the PILNA) and point to a dire need to improve reading skills throughout the country. Results indicate that while students have good listening comprehension, dictation and letter recognition skills, they struggle with decoding familiar words and nonwords. As a result, an overwhelming percentage of students are unable to read fluently with comprehension. The majority of students in Year 1 (90%) and Year 2 (80%) have zero reading comprehension skills. In Year 3, comprehension improves where about one-third of students could comprehend at least 60 percent of text. However, the majority of students in Years 1-3 are reading below the recommended 80 percent comprehension benchmark. Only 9 percent of all students met the benchmark and about 20 percent in Year 3. Thus, the breakthrough point for reading fluency and comprehension is likely much later than Year 3 for the majority of students. An analysis of the relationship between oral reading fluency and reading comprehension scores to determine fluency ranges and percentage of students reaching the 80 percent reading comprehension benchmark found that greater oral reading fluency is associated with higher levels of reading comprehension. However, there is great variability in fluency ranges that fall within the 80 percent benchmark. Scores ranged from 40 to 129 words per minute. Most students meeting the benchmark read 50-79 correct words per minute. Only 9 percent of students across all years were able to comprehend 80 percent of what they read. One potential explanation for low reading comprehension could be the high levels of stunting in Tuvalu, which affects short-term memory. Additional research (e.g., correlations with children’s height) is necessary to validate this conclusion. Throughout all sub-tests, girls tended to read better than boys. More females than males reached the 80 percent reading comprehension benchmark. Students in the northern and southern regions performed better than other regions; however, the differences were not statistically significant. The analysis identified a number of student characteristics, instructional methods, classroom environmental factors, and teacher expectations that are associated with better reading outcomes. Students who attended pre-school, read to himself/herself at home, and liked to read scored an average of 10 more words on the ORF sub-test. Instructional methods that seem to have a positive influence on students’ ORF scores were teaching new vocabulary words and teaching letter sounds. When teachers maintained records of students’ work, ORF scores increased by 10 words per minute. Students’ reading silently or aloud was found to be very important for increasing students’ fluency rates. In terms of classroom environmental factors, having a reading corner and sufficient space for organizing group work activities were considered important for increasing reading fluency. Interestingly, students tended to have higher ORF scores when teachers had low expectations for achievement of reading competencies. This could be because teachers provided more time for practice or reviewed lower-level skills with students. Teachers with 61 high expectations yielded a small increase in ORF scores. This could be because they spent less time reviewing lessons. In this analysis teacher expectations were measured according to whether they were higher or lower than the median expectation of teachers. In the future, teacher expectations should be measured against curricular standards. Remarkably, teacher training and assessment methods were negatively related to student ORF scores. Teacher training could negatively impact student performance if teachers are frequently absent to attend trainings or if they are struggling to implement new teaching methods. Likewise, overemphasizing assessment and not utilizing results to inform instruction can hamper student performance. More research is necessary to identify the true cause behind these results. Based on the findings and conclusions, the following recommendations are presented to improve the quality of early grade reading instruction in Tuvaluan schools: 1. Set reading fluency and comprehension benchmarks to guide and support teacher professional development to reach reading targets: Reading benchmarks bring attention to the learning expectations both students and teachers should be held accountable to support children in becoming readers. They serve a useful function when they are coupled with strategies that will strengthen the capacity of schools and teachers to help students reach them. Without benchmarks, there is no pressure in the system to achieve learning outcomes and no fear of failure. At the same time, simply setting standards and benchmarks without evidence that schools and teachers are failing in their job and the support mechanisms to improve, benchmarks will create fear and frustration among school administrators, teachers and parents. Reading benchmarks should be established to guide the development of basic reading skills in each of the early grades both in terms of oral reading fluency and comprehension; and to guide the mechanisms that are required for schools to succeed in achieving them. As a first step, stakeholders in Tuvalu should decide on the level of comprehension required to understand grade level text (e.g., 60%, 80%) and then review the fluency scores that fall within that range. Tonga, for instance, has set the reading fluency benchmark at 50 cwpm and a reading comprehension of at least 75%. Students achieving these levels can be considered to have learned to read well, have the basic reading skills needed to develop their literacy skills, and have the ability to comprehend more complex text in upper grades. If stakeholders in Tuvalu agree with the 80 percent benchmark, then looking at the TuEGRA data, an acceptable fluency range may be 70-75 cwpm. Given that the current mean ORF score is 21 cwpm, that range may seem too high for students to achieve. Thus, policymakers may decide to lower the benchmark to 60 percent and the fluency benchmark to 57 cwpm, although this decision will carry important educational implications that ought to be discussed in country. Once the benchmark is decided, the next step is to consider the targets. Currently, 9 percent of students are meeting the 80 percent reading comprehension benchmark. How many should meet the benchmark in one year or five years? Once the targets are set up, schools and educators should be supported to achieve these benchmarks. The Tuvaluan MOE should then conduct a national reading campaign to sensitize all 62 stakeholders of the new benchmarks, inform educators and schools of the support mechanisms available to achieve them, and then monitor and report progress towards achieving the targets at all levels (national/regional MOE, school and communities). 2. Conduct a follow-up study on teaching practices and assessment methods to identify strengths and weaknesses. Teacher training and assessment were associated with lower ORF scores. It is important to understand why these interventions have an adverse relationship. It could be that so few teachers were trained (15%) so there was lack of support to master the new methodologies or that training was not aligned with best practices in reading instruction. Additionally, teachers may not understand how to accurately assess students and utilize the results for reflection and lesson planning. The analysis also showed that classrooms with teachers who maintained a record of students’ work had higher ORF scores. A follow-up study is recommended to investigate the strengths and weaknesses of current assessment and teaching methods vis-à-vis the results observed in different classrooms. It is also important to understand what types of information the teacher records and whether it is related to tracking student progress. 3. Strengthen teacher training for reading instruction with a focus on vocabulary, phonics and reading comprehension. Students whose teachers spent more time on teaching vocabulary, letter sounds, decoding skills and reading comprehension had higher ORF scores. Hence, instruction in these areas should be strengthened to increase overall reading scores. In the case of Tuvalu, the large shares of zero scores observed in various reading subtasks included in TuEGRA suggest inadequate instructional time in the classroom due to student absenteeism, school closures, or even inadequate use of instructional time -i.e. pedagogy or content-- during regular school days. Teacher training should incorporate the curriculum expectations and provide teachers with the specific methods, classroom activities and assessment methods required to achieve results. In addition, classrooms should be equipped with reading corners to provide students with sufficient resources and opportunities to read. 4. Increase the number of books in the classroom library/reading corner and the use in teachers’ instruction. In Tuvalu, 80 percent of classrooms have reading corners with an average of four books, but only 1.97 books are used during classroom instruction. One possible explanation is that there are not enough books in the classroom. The International Reading Association (IRA) recommends that classroom libraries start with at least seven books per child and purchase two additional new books per year. The optimal number of books in a classroom library is 300-600, depending on the grade level and number of copies22. The number of books teachers should expect children to read during the school year is 100-125 picture books by the end of Year 1 and 50-75 chapter books by the end of Year 2. There is strong evidence that reading skills grow and develop to the extent that 22 Neuman, S. (undated). The importance of the classroom library. Scholastic. Available at: http://teacher.scholastic.com/products/paperbacks/downloads/library.pdf 63 children get actual practice reading. This is particularly difficult in countries with a modest literary tradition where the amount of titles in the local language may be limited. Recent innovations in technology now enable the production of reading materials in almost any language, provided there are guidelines for avid writers to follow. Tuvaluan educators are already producing additional reading materials for children and their efforts should be further supported to ensure beginning readers have a reasonable variety of stories to practice and enjoy. Additionally, in low- resource countries where publishing may be expensive and there are high teacher- pupil ratios, a low-cost option is to provide e-readers. E-readers allow students and teachers to choose from a variety of genres, it is portable so students can read from home or school, and its read aloud features provides additional support for emergent readers2324. In addition to provision of an increased number of hard and soft copy books, teachers should be trained on how to better integrate materials into their instruction and on how to develop attractive reading corners25. 5. Increase time-spent reading. The more time children spend reading the better and more fluent readers they become. In order to increase students’ reading fluenc y skills, teachers should ensure that students are spending sufficient time reading every day through teacher-led, parent-led or self-guided reading activities. Research recommends that children read between 20-40 minutes per day. If more titles are produced to assist children in their path to literacy, teachers and parents should ensure adequate time is given to read aloud activities at school and at home even among students considered to be reading fluently and with comprehension. 6. Address the issue of non-readers in Years 1 and 2. Stakeholders should conduct classroom level assessments to identify non-readers in Years 1 and 2, diagnose the causes, and design specific activities to address deficiencies before children complete Year 2. Teachers in Years 1 and 2 have the greatest responsibility in ensuring children finish the first two grades of primary education with the necessary skills to read and understand text. In this sense, identification of struggling readers and knowledge of ways in which they need to be supported should make the core material of in-service training for Year 1 and Year 2 teachers. For instance, teachers may group students according to ability and provide remedial activities and appropriately levelled text. This “catch-up approachâ€? is being used by UNICEF in Zambia based J-Pal’s research in India, which demonstrated that students grouped by ability is more effective than mixed-ability grouping. UNICEF will assess students in all grades and group them according to reading levels (non-readers, those who can read letter sounds, syllables, words, passages, etc.). Grouping students by level rather than grades has produced dramatic results in India, Kenya and Ghana. Teachers and school administrators should further determine whether 23 Adams, A. & van der Gaag, J. (2011). First Step to Literacy: Getting Books in the Hands of Children Available at: https://www.brookings.edu/research/first-step-to-literacy-getting-books-in-the-hands-of- children/ 24 UNESCO (2014). Reading in the mobile era: A study of mobile reading in developing countries. http://unesdoc.unesco.org/images/0022/002274/227436E.pdf 25 Neuman, S. (undated). The importance of the classroom library. 64 non-readers have learning disabilities (e.g., dyslexia) and design relevant interventions for special needs students. 7. Teach students how to develop reading comprehension skills in the early grades. Research shows that teachers are often the ones to ask questions orally and that students are expected to respond, but rarely are students asked to develop their own questions as they read (think aloud) or to make predictions and then check their predictions. Students need to be empowered to make meaning from text and to use an inquiry-based approach to reading. Teachers could ask students to make predictions before, during and after reading, to retell stories, or to identify problems within the story and provide potential solutions. Teachers should use various strategies and frameworks to develop students’ reading comprehension skills from as early as kindergarten or Year 126. 8. Develop activities that specifically focus on raising boys’ performance and interest in reading. The results illustrated that boys consistently performed lower than girls. There may be cultural or gender barriers that affect boys’ interest and engagement in reading activities. In addition to designing strategies to address low competencies of boys and girls, stakeholders at all levels should discuss the potential challenges specific to boys and design strategies to improve boys’ reading achievement. Successful strategies that have worked in other countries include developing gender-sensitive materials that attract boys’ attention (such as sports, science fiction, fantasy, comic books, digital text, and stories that are humorous) and integrating reading into extracurricular activities (e.g., sports, health clubs, student government)27. ANNEX 1 / TABLES Annex 1.A: Reading skills tested Annex 1.B: Correlations between tasks Annex 1.C: Tuvalu- Regression Analysis with Reading Comprehension as Outcome 26 Sample reading comprehension activities can be found in the following guides: Ontario Ministry of Education. (2003). A Guide to Effective Instruction in Reading: Kindergarten to Grade 3 (Available at: http://eworkshop.on.ca/edu/resources/guides/Reading_K_3_English.pdf) Institute of Education Sciences (IES). (2010). Improving Reading Comprehension in Kindergarten through 3rd Grade. What Works Clearninghouse (Available at: https://education.ohio.gov/getattachment/Topics/Early-Learning/Third- Grade-Reading-Guarantee/Third-Grade-Reading-Guarantee-Teacher-Resources/Improving-Reading- Comprehension-in-Kindergarten-Through-3rd-Grade.pdf.aspx) 27 Sample guide for improving boys’ literacy: Ontario Ministry of Education. (2004). Me Read? No Way! A practical guide to improving boys’ literacy skills. Available at: http://www.edu.gov.on.ca/eng/document/brochure/meread/meread.pdf 65 Annex 1.A: Reading Skills Tested Test Measured Timed Letter Name Knowledge Ability to read alphabet letters with accuracy and Y fluency Letter Sound Knowledge Phonics – the ability to identify sounds of letters with Y accuracy and fluency Initial Sound Phonemic awareness – the ability to identify sounds in N Identification spoken words Familiar Word Reading Ability to read familiar words with fluency and Y accuracy Nonword Reading Ability to decode linguistically sound invented words Y Oral Passage Reading Ability to read a short passage with fluency and Y accuracy Reading Comprehension Ability to respond to several comprehension questions N based on passage Listening Comprehension Ability to comprehend a short story read aloud N Dictation Alphabet knowledge, listening and writing skills N 66 ANNEX 1.B: Correlation between tasks Letters Phonemic Sound Correct Correct Oral Reading Listening Dictatio Correc awarenes s words Invente Readin Comprehensio Comprehensio n t Per s Correc Per d Words g n n (D) Minute (PA) t Per Minute Per Fluenc (RC) (LC) (LCPM Minute (CWPM Minute y ) (SCPM ) (CIWPM (ORF) ) ) LCPM 1 0.471* PA 1 * 0.718* SCPM 0.689** 1 * 0.575* 0.656* CWPM 0.553** 1 * * CIWP 0.514* 0.583* 0.499** 0.955** 1 M * * 0.561* 0.630* ORF 0.536** 0.964** 0.946** 1 * * 0.481* 0.532* RC 0.454** 0.870** 0.852** 0.902** 1 * * 0.293* 0.358* LC 0.403** 0.345** 0.312** 0.338** 0.338** 1 * * 0.473* 0.536* D 0.460** 0.762** 0.722** 0.728** 0.661** 0.312** 1 * * 67 Annex 1.C: Tuvalu – Regression Analysis with Reading Comprehension as Outcome Table 41: Impact of student characteristics on Oral Reading Fluency (ORF) scores Student Characteristics Change in RC Rsqr score (+/-) Student attends preschool before Year 1 0.76 0.25 Student eats breakfast before arriving to school 0.13 0.23 Student speaks Tuvaluan at home 0.10 0.23 Student has a literate mother 0.29 0.24 Student has a literate father 0.40 0.25 Student has both parents literate 0.26 0.26 Student has a literate sibling 0.10 0.23 Student receives help with homework from the 0.24 0.24 mother Student receives help with homework from 0.14 0.23 father Student receives help with homework from -0.10 0.23 sibling Someone asks student about what he/she did in 0.15 0.23 school Student tells someone at home when he/she gets 0.16 0.23 good marks Student has books, newspapers or other things 0.05 0.23 to read at home Student has books, newspapers or other things 0.06 0.23 to read at home in Tuvaluan Student has books, newspapers or other 0.37 0.24 things to read at home in English Student reads aloud to someone at home 0.10 0.23 Student reads to himself/herself at home 0.52 0.25 Someone reads to student at home 0.18 0.23 Student reads on a computer or mobile device at -0.26 0.21 home Student likes to read 0.51 0.24 Table 42: Effect of teachers' characteristics on Oral Reading Fluency Score (ORF) Teacher Characteristics Change in RC Rsqr Score (+/-) Has a primary teaching certificate -0.04 0.45 Has a reading corner in the classroom 0.33 0.18 Has not been absent from school in the last -0.08 0.40 term Has met with the parents of his/her students -0.07 0.14 Age of the teacher -0.002 0.42 68 Number of year of experience in teaching -0.000 0.45 Number of year of experience in teaching on -0.004 0.29 this island Number of minute from home to school -0.001 0.45 Table 43: Effect of teacher expectations on oral reading fluency (ORF) scores Teacher expectations of student Change in RC Rsqr reading performance score (+/-) Grade students should recognize and say letter names Year 1 - 0.38 Year 2 0.11 Year 3 0.36 Grade students should understand stories read aloud by the teacher in the classroom Year 1 - 0.34 Year 2 0.25 Year 3 0.61 Grade students should sound out words using phonics Year 1 - 0.28 Year 2 0.45 Year 3 0.43 Grade students should read aloud to teacher and other students Year 1 -0.07 0.20 Year 2 - Year 3 0.30 Grade students should understand simple texts that they read in class Year 1 0.01 0.39 Year 2 - Year 3 0.22 Table 44: Effect of training and guides on student oral reading fluency (ORF) scores Change in RC Rsqr score Teacher have a syllabus 0.13 0.38 Teacher have an English syllabus - Teacher have receive training on how to teach -0.27 0.15 reading in the last three years 69 Table 45: Effect of Classroom Environment on Student ORF Scores Classroom Environment Change in RC Rsqr Score (+/-) Classroom displays 0.11 0.13 Spelling/vocabulary displayed 0.13 0.06 Song/hymns/stories displayed on blackboard 0.14 0.02 Song/hymns/stories displayed on -0.11 0.04 charts/posters Student work displayed 0.16 0.04 Print materials used in instruction 0.06 0.02 Sufficient classroom space for organized group 0.09 0.05 activities Reading corner in the classroom 0.29 0.11 Student profiles (folder with student work and 0.17 0.02 student info) Students sitting on floor n/a - Table 46: Effect of Reading Instructional Resources on Student ORF Scores Language use in classroom Change in RC Rsqr Score (+/-) Reading materials in classroom -0.01 0.01 Number of reading books used during the 0.004 0.01 lesson Table 47: Effect of teacher instructional and assessment methods on student performance Teaching instructional and assessment Change in Rsqr methods ORF score Teaching of Listening Comprehension 0.52 1-2 days 0.05 3-4 days -0.18 Daily 0.05 Low order book orientation 0.35 1-2 days 0.09 3-4 days -0.22 Daily 0.14 Asking children to orally retell a story that 0.47 they have read 1-2 days -0.02 3-4 days -0.28 Daily -0.07 Teaching new letter sounds 0.34 70 1-2 days -0.25 3-4 days 0.21 Daily 0.08 Asking children to sound out unfamiliar 0.64 words using knowledge of letter sounds 1-2 days 0.03 3-4 days -0.14 Daily 0.03 Teaching meanings of new 0.48 words/vocabulary 1-2 days 0.20 3-4 days 0.36 Daily 0.13 Whole class reading 0.09 1-2 days -0.02 3-4 days -0.08 Daily -0.11 Guided reading .21 1-2 days 0.22 3-4 days -0.16 Daily 0.12 Listening to a child read aloud 0.17 1-2 days 0.22 3-4 days 0.26 Daily 0.39 Students readings on their own silently 0.23 1-2 days 0.01 3-4 days -0.54 Daily -0.08 Reading comprehension activities 0.10 1-2 days 0.07 3-4 days 0.17 Daily 0.12 Children take books home to read with 0.23 their parents 1-2 days 0.10 3-4 days -0.78 Daily 0.07 71 Evaluating student’s oral reading with 0.14 running records or any other method 1-2 days -0.07 3-4 days -0.35 Daily -0.08 72 ANNEX 2 / INSTRUMENTS Annex 2.A: EGRA Instrument Annex 2.B: Student Questionnaire Annex 2.C: Head Teacher Questionnaire Annex 2.D: Teacher Questionnaire Annex 2.E: Classroom Observation 73 Annex 2.A: EGRA Instrument Early Grade Reading Assessment: Student Response Form Administrator Instructions and Protocol General Instructions It is important to establish a playful and relaxed rapport with the children to be assessed, via some simple initial conversation about topics of interest to the child (see example below). The child should perceive the following assessment almost as a game to be enjoyed rather than a severe situation. It is important to read aloud ONLY the sections in boxes slowly and clearly. Talofa. A toku igoa ko __________ a ko au e nofo i ____________. E fia fai atu ne au se fakamatalaga toetoe e uiga mo au. (Fakaasi a mea e fiafia koe o fai, pela mo tafaoga, meakai, manu fagai.) ____________ A ko koe? 1. Sea a te mea e fiafia koe o fai ma fai koe e se nofo i te akoga? 2. Nea a tafaoga e fia tafao koe i ei? Verbal Consent • Ke fai atu ne au a te pogai e nofo i ei au i konei i te aso nei. Au e galue fakatasi mo te Matagaluega o Akoakoga, kae e taumafai matou ke malamalama me e tauloto pefea a tamaliki ke iloa ne latou o faitau. • E manakogina a tou fesoasoani i te feitu tenei. Kae e seafaina ma fai e se manako koe o kau mai. • A taua ka tafao i se tafaoga faitau. Fakamolemole faitau mai ne koe ki luga a mataimanu, muna mo se tala toetoe. • Ka fakaoga ne au a te 'tablet' tenei ke lavea ne au a te leva o tau faitau. • A te mea tenei e SEAI se sukega kae ka se pokotia foki i ei ou kai i te akoga. • Ka sili atu foki ne au ne nisi fesili e uiga mo tou kaiga, e pela mo te gana e fakaoga ne latou i te fale mo nisi mea i ei kola e isi i te otou kaiga. • Toe fakamasaua atu me e mafai ne koe o se kau mai ki te mea tenei ma fai e se manako koe ki ei. Ka fai taua ko kamata, e lei fua ma fai e se fia tali ne koe se fesili. • E isi ne au fesili? Ko toka koe o kamata? Check box if verbal consent is obtained: YES (If verbal consent is not obtained, thank the child and move on to the next child, using this same form) A. Date of assessment : G. Student’s grade: â—‹ 1 = 1st Grade B. Enumerator’s name : â—‹ 2 = 2nd Grade C. Location of school : â—‹ 3 = 3rd Grade D. School name : I. Student’s age: E.Teacher name J. Student’s gender: â—‹ male â—‹ female F. Multigrade class? â—‹ 0 = No â—‹ 1 = Yes K. Time started: ____ : _____ am / pm 74 Early Grade Reading Assessment: Student Response Form MOD 1 Section 1. Letter Name Knowledge Show the child the sheet of letters in the student stimuli booklet. Say: A te laupepa tenei e fonu i mataimanu fakaTuvalu. Fakamolemole, fai mai a IGOA o mataimanu katoa kola e iloa ne koe. E se ko lotou fakatagiga, kae ko olotou igoa. [point to the letter o] E fai penei, a te igoa o te mataimanu tenei [O] ko te /o/.[A taua ka fakaakoako muamua. Fai mai a te igoa o te mataimanu tenei [point to the letter n ]: [If the child responds correctly, say]: Lei, a te igoa o te mataimanu tenei ko te /nu/. [If the child does not respond correctly, say]: A te igoa o te mataimanu tenei ko te /nu/. Toe tasi. Fai mai a te igoa o te mataimanu tenei [point to the letter f ]: [If the child responds correctly, say]: Lei, a te igoa o te mataimanu tenei ko te /fa/. [If the child does not respond correctly, say]: A te igoa o te mataimanu tenei ko te /fa/. Ka fai au pela "Kamata," ko koe loa ko kamata i konei [point to the first letter] o fakasolo pela i te laupepa [point to the last letter ]. Tusi ki mataimanu takitasi kae fai mai a te igoa o te mataimanu tena i se leo maluga. Faitau fakalei kae fakavave ki te mea e mafai ne koe. Ka oko koe ki se mataimanu e se iloa ne koe, fano ki te sua mataimanu. Tusi a tou matikao i luga i te mataimanu muamua. Ko toka. Taua ka kamata. Start the timer when the child reads the first letter. Follow along with your pencil and clearly mark any incorrect letters with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self- corrected letter as incorrect, circle the letter and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the name of the letter, point to the next letter and say “Please go on.â€? Mark the letter you provide to the child as incorrect. If the student gives you the letter sound, rather than the name, provide the letter name and say: [“Please tell me the NAME of the letterâ€?]. This prompt may be given only once during the exercise. AFTER 60 SECONDS SAY, “stop.â€? Mark the final letter read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: O n f 1 2 3 4 5 6 7 8 9 10 G T h m k a L s f o (10) V I u E N i P n O e (20) p g U L M H v A K F (30) S m n U a K t e U A (40) t A T k M L O a i E (50) A I p e L i N o T S (60) e O a g o E L E s v (70) i h m i e a k f a I (80) U a O E K t F T f A (90) t I u A L e o N a n (100) Time remaining on stopwatch at completion (number of SECONDS) : Check this box if the exercise was discontinued because the child had no correct answers in the first line. Did the child respond with some of the letter names in English? yes no Malo taumafai! Ka fakasoko tatou ki te sua vaega. 75 Early Grade Reading Assessment: Student Response Form MOD 2 Section 2. Initial Sound Identification This is NOT a timed exercise and THERE IS NO STUDENT SHEET. Read aloud each word twice, and have the student say the sounds. Remember to model the “pureâ€? sounds: /p/, not “puhâ€? or “pay.â€? Say: A te mea tenei se vaega fakalogologo. Au e manako ke fai mai ne koe a te tagi o mataimanu muamua o pati takitasi. E fai penei, i te pati tenei “haluâ€? a te tagi muamua loa o te mataimanu /hhh/. Au e manako ke fai mai ne koe a te tagi muamua loa o te mataimanu tela e lagona i pati takitasi. Fakalogologo ki te pati, ko fai mai i ei ne koe a te tagi muamua loa o te mataimanu tena. A taua ka fakaakoako muamua. Sea te tagi o te mataimanu muamua loa i te pati "lakau"? "lakau"? [If the child responds correctly, say] : Lei kii, a te tagi tenei muamua i te pati "lakauâ€? ko te /lll/. [If the child does not respond correctly, say] : Toe fakalogologo: "lakau". A te tagi muamua i te pati "lakau" ko te /lll/. Toe tasi. Sea te tagi o te mataimanu muamua loa i te pati "taina"? "taina"? [If the child responds correctly, say] : Lei kii, a te tagi tenei muamua i te pati "taina" ko te /t/. [If the child does not respond correctly, say] : Toe fakalogologo: "taina". A te tagi muamua i te pati "taina" ko te /t/. Ko toka. Taua ka kamata. Read the prompt and then pronounce the target word a second time. Accept only as correct the isolated sound (without an other vowel). If the child does not respond after 3 seconds, mark as “No responseâ€? and say the next prompt. Enunciate clearly, but do not overemphasize the beginning sound of each word. Early stop rule: If the child responds incorrectly or does not respond to the first five words, say “Thank you!â€?, discontinue this exercise, check the box at the bottom of the page, and go on to the next exercise. Sea a te tagi muamua loa i te pati “ â€?? “ â€?? [Repeat the word twice] fitu /ffff/ o Correct o Incorrect o No Response ato /aaa/ o Correct o Incorrect o No Response kuli /k/ o Correct o Incorrect o No Response pula /p/ o Correct o Incorrect o No Response nonu /nnn/ (5 words) o Correct o Incorrect o No Response uila /uuu/ o Correct o Incorrect o No Response mate /mmm/ o Correct o Incorrect o No Response gutu /ggg/ o Correct o Incorrect o No Response selau /sss/ o Correct o Incorrect o No Response vela /vvv/ o Correct o Incorrect o No Response Check this box if the exercise was discontinued because the child had no correct answers in the first five words: Malo taumafai! Ka fakasoko tatou ki te sua vaega. 76 Early Grade Reading Assessment: Student Response Form MOD 3 Section 3. Letter Sound Knowledge Show the child the sheet of letters in the student stimuli booklet. Say: A te laupepa tenei e fonu loa i mataimanu fakaTuvalu. Fakamolemole, fakatagi mai a tagi o mataimanu katoa kola e iloa ne koe. E se ko lotou igoa, kae ko olotou tagi. [point to the letter a] E fai penei, a te mataimanu tenei [a] e tagi penei /aaa/ A taua ka fakaakoako muamua. Fai mai a te tagi o te mataimanu tenei [Point to the letter T ]: [If the child responds correctly, say ] : Lei, a te mataimanu tenei e tagi penei /t/. [If the child does not respond correctly, say ] : A te mataimanu tenei e tagi penei /t/. Toe tasi. Fai mai a te tagi o te mataimanu tenei [Point to the letter m]: [If the child responds correctly, say ] : Lei, a te mataimanu tenei e tagi penei /mmm/. [If the child does not respond correctly, say] : A te mataimanu tenei e tagi penei /mmm/. Ka fai au pela "Kamata," ko koe loa ko kamata i konei [point to the first letter] o fakasolo pela i te laupepa [point to the last letter]. Tusi ki mataimanu takitasi kae fai mai a te tagi o te mataimanu tena i se leo maluga. Faitau fakalei kae fakavave ki te mea e mafai ne koe. Ka oko koe ki se mataimanu e se iloa ne koe, fano ki te sua mataimanu. Tusi a tou matikao i luga i te mataimanu muamua. Ko toka. Taua ka kamata. Start the timer when the child reads the first letter. Follow along with your pencil and clearly mark any incorrect letters with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self- corrected letter as incorrect, circle the letter and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the sound of the letter, point to the next letter and say “Please go on.â€? Mark the letter you provide to the child as incorrect. If the student gives you the letter name, rather than the sound, provide the letter sound and say: [“Please tell me the SOUND of the letterâ€?]. This prompt may be given only once during the exercise. AFTER 60 SECONDS SAY, “stop.â€? Mark the final letter read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: a T m 1 2 3 4 5 6 7 8 9 10 (10) T G s H N u L e o A (20) s U t A m E S a P i (30) k e K O A p I h A E (40) M h U e f A T N m A (50) E F L I u E i T e o (60) I a e T a P n a k I (70) o L n a t K a v a o (80) N g K n L e M u L S (90) v O f k O a L A U m (100) i F i V g i A E t O Time remaining on stopwatch at completion (number of SECONDS) : Check this box if the exercise was discontinued because the child had no correct answers in the first line. Malo taumafai! Ka fakasoko tatou ki te sua vaega. 77 Early Grade Reading Assessment: Student Response Form MOD 4 Section 4. Familiar Word Reading Show the child the sheet of familiar words in the student stimuli booklet. Say: Konei a nisi pati masani i te gana Tuvalu kola e manako au ke faitau mai ne koe a sose pati tela e iloa ne koe. Sa "spell" mai ne koe a pati, kae faitau. [Point to the word “neâ€?] E fai penei, a te pati tenei ko te : "ne". A taua ka fakaakoako muamua: Fakamolemole, faitau mai a te pati tenei [Point to the word "koe"] [If the child responds correctly, say]: Lei, a te pati tenei ko te "koe" [If the child does not respond correctly, say]: A te pati tenei ko te "koe" Toe fai mai a te sua pati tenei: Fakamolemole faitau mai a te pati tenei [Point to the word "eva"] [If the child responds correctly, say]: Lei, a te pati tenei ko te "eva" [If the child does not respond correctly, say]: A te pati tenei ko te "eva" Ka fai au pela "Kamata," ko koe loa ko kamata i konei [ point to the first word ] o fakasolo pela i te laupepa [ point to the last word]. Tusi ki pati takitasi kae fai mai a te pati tena i se leo maluga. Faitau fakalei kae fakavave ki te mea e mafai ne koe. Ka oko koe ki se pati e se iloa ne koe, fano ki te sua pati. Tusi tou matikao i luga i te pati muamua. Ko toka. Taua ka kamata. Start the timer when the child reads the first word. Follow along with your pencil and clearly mark any incorrect words with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self-corrected word as incorrect, circle the word and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the word, point to the next word and say “Please go on.â€? Mark the word you provide to the child as incorrect. AFTER 60 SECONDS SAY, “stop.â€? Mark the final word read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: ne koe eva 1 2 3 4 5 (5) lua it i mo au lalo (10) mama mai fiafia ka luga (15) t ala ta kae fale te (20) kaiga ia t enei fait au ma (25) t usi lot o lomigina fuli ki (30) fano t ele mafai at a kuli (35) ko laua lavea se lakau (40) t oku t usigina fia mea loa (45) ei ikai t aku manako t aimi (50) akoga fakat oka t ena aso kai Time remaining on stopwatch at completion (number of SECONDS) : Check this box if the exercise was discontinued because the child had no correct answers in the first line. Malo taumafai! Ka fakasoko tatou ki te sua vaega. 78 Early Grade Reading Assessment: Student Response Form MOD 5 Section 5. Non-Word Decoding Show the child the sheet of invented words in the student stimuli booklet. Say: Konei a nisi pati faite i te gana Tuvalu kola e manako au ke faitau mai ne koe a sose pati kola e iloa ne koe. Sa "spell" mai ne koe a pati, kae faitau. [Point to the word “seteâ€?] E fai penei, a te pati faite tenei ko te : "sete". A taua ka fakaakoako muamua: Fakamolemole, faitau mai a te pati faite tenei [Point to the word "nile"] [If the child responds correctly, say] : Lei, a te pati faite tenei ko te "nile" [If the child does not respond correctly, say] : A te pati faite tenei ko te "nile" Toe fai a te sua pati tenei: fakamolemole faitau mai a te pati faite tenei [Point to the word "kei"] [If the child responds correctly, say] : Lei, a te pati faite tenei ko te "kei" [If the child does not respond correctly, say] : A te pati faite tenei ko te "kei" Ka fai au pela "Kamata", ko koe loa ko kamata i konei [Point to the first word] o fakasolo pela i te laupepa [point to the last word]. Tusi ki pati takitasi kae taku mai a te pati tena i se leo maluga. Faitau fakalei kae fakavave ki te mea e mafai ne koe. Ka oko koe ki se pati e se iloa ne koe, fano ki te sua pati. Tusi tou matikao i luga i te pati muamua. Ko toka. Taua ka kamata. Start the timer when the child reads the first word. Follow along with your pencil and clearly mark any incorrect words with a slash ( / ). Count self-corrections as correct. If you’ve already marked the self-corrected word as incorrect, circle the word and go on. Stay quiet, except when providing answers as follows: if the child hesitates for 3 seconds, provide the word, point to the next word and say “Please go on.â€? Mark the word you provide to the child as incorrect. AFTER 60 SECONDS SAY, “stop.â€? Mark the final word read with a bracket ( ] ). Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this exercise, check the box at the bottom, and go on to the next exercise. Example: sete nile kei 1 2 3 4 5 pasu ogi kue muvu enu (5) neno sifua gat o ifa nili (10) giviu t usu vola miga numuga (15) sogeu ugou veavea ahe sot u (20) sunikimo lanai seseu hela gou (25) t eae vesu sat ikemo panu neue (30) pesa gue ogeka sinie et e (35) geuke losu veine logusigu age (40) ut e lupa sut u alu kit o (45) t oukou simo vuo t igi vousoi (50) Time remaining on stopwatch at completion (number of SECONDS) : Check this box if the exercise was discontinued because the child had no correct answers in the first line. Malo taumafai! Ka fakasoko tatou ki te sua vaega. 79 Early Grade Reading Assessment: Student Response Form MOD 6 Section 6a. Oral Passage Reading Section 6b. Oral Passage Reading Show the child the story in the student stimuli booklet. Say: When 60 seconds are up or if the child finishes reading the passage in less than 60 seconds, REMOVE the passage from Tenei se tala toetoe. Au e manako ke faitau fakalei mai ne koe fakavave i se leo in front of the child, and ask the first question below. maluga. Ka fai koe ko oti, ka sili atu ne au ne fesili e uiga mo te tala tena. Ka fai au pela "Kamata", ko koe loa ko faitau ki te mea tela e mafai ne koe. Ka fai e oko koe Give the child at most 15 seconds to answer the question, ki se pati e se iloa ne koe, fano ki te sua pati. Tusi tou matikao i luga i te pati mark the child’s response, and move to the next question. muamua. Ko toka. Taua ka kamata. Read the questions for each line up to the bracket showing where the child stopped reading. Start the timer when the child reads the first word. Follow along with your pencil and clearly mark any incorrect words with a slash (/). Count self-corrections as correct. Stay quiet, unless the child hesitates for 3 seconds, in which case provide the word, point to the next word and say “Please go on.â€? Mark the word you provide to the child as incorrect. Nei, ka sili atu ne au ne naai fesili e uiga mo te tala ko oti ne faitau ne koe. AFTER 60 SECONDS SAY, “stop.â€? Mark the final word read with Taumafai o tali fakalei mai a fesili ki te mea e mafai ne koe. E mafai o tali mai a bracket ( ] ). ne koe a fesili ki sose gana. Ko toka. Taua ka kamata. Early stop rule: If the child does not give a single correct response on the first line, say “Thank you!â€?, discontinue this No Correct Incorrect exercise, check the box at the bottom, and go on to the next exercise. Response I te tala, sea te mea e fiafia ki ei a Tui? A Tui e fiafia o futipolo. 6 [Futipolo] Ko oi te igoa o te taugasoa o Tui? A ia e kau tasi mo tena taugasoa ko Seti. 16 [Ko Seti; Seti] Ko te taimi fea ne olo i ei laua o futipolo? I te afiafi e tasi, a laua ne olo o futipolo 27 [I te afiafi; Te afiafi] i te malae. A Tui ne galo ana taka. Ne tuku ne Kaia a Tui ne tafao ki taka o Seti? Seti ki ei ana nisi taka. 45 [Ne galo ana taka] I ai te kau ne sili? Ne tafao laua kae ne sili te olotou kau. 54 [Te kau a Tui mo Seti, Te kau a Tui, Te kau a Seti,Lauto, laua ] Time remaining on stopwatch at completion (number of SECONDS) : Which language did the child use to answer the questions? Check this box if the exercise was discontinued because the child had Tuvaluan Nuian no correct answers in the first line. Malo taumafai! Ka fakasoko taua ki te sua vaega. 80 81 CODING FOR DATA ENTRY PERSONNEL ONLY—DO NOT CODE AT THE SCHOOL. Usually 2 = Correct; 1 = Partially correct; 0 = Incorrect; 9 = No Evaluation Criteria Score response For spaces 1 = Correct; 0 = no space in correct position 2 – i, I Tusi ‘I’ ke sao I 0 – any other response; 9 – no response, they left it blank 2 – taeao, Taeao Tusi ‘taeao’ ke sao taeao 1 – tao, aeo, taeo, tae, ao, teao 0 – any other response; 9 – no response, they left it blank 2 – katoa, Katoa Tusi ‘katoa’ ke sao katoa 1 – ktoa, kto, kta, atoa, kato, toa 0 – any other response; 9 – no response, they left it blank 2 – au, Au Tusi ‘au’ ke sao au 1 – a, A, aau, auu, aauu, u, U 0 – any other response; 9 – no response, they left it blank 2 – e, E Tusi ‘e’ ke sao e 0 – any other response; 9 – no response, they left it blank 2 – masani, Masani Tusi ‘masani’ ke sao masani 1 – maasani, msani, sani, msni, masni, masan, ma sani 0 – any other response; 9 – no response, they left it blank 2 – o, O Tusi ‘o’ ke sao o 0 – any other response; 9 – no response, they left it blank 2 – teu, Teu Tusi ‘teu’ ke sao teu 1 – te, tu, eu 0 – any other response; 9 – no response, they left it blank 2 – a, A Tusi ‘a’ ke sao a 0 – any other response; 9 – no response, they left it blank 2 – kaiga, Kaiga Tusi ‘kaiga’ ke sao kaiga 1 – kga, kaga, kiga, kaig, kig, iga, aiga 0 – any other response; 9 – no response, they left it blank Fakavasaga ‘I taeao’ 1 – space between ‘I’ and ‘taeao’; 0 – no space Fakavasaga ‘taeao 1 – space between ‘taeao’ and ‘katoa’; 0 – no space katoa’ Fakavasaga ‘katoa, au’ 1 – space between ‘katoa,’ and ‘au’; 0 – no space Fakavasaga ‘au e’ 1 – space between ‘au’ and ‘e’; 0 – no space Fakavasaga ‘e masani’ 1 – space between ‘e’ and ‘masani’; 0 – no space Fakavasaga ‘masani o’ 1 – space between ‘masani’ and ‘o’; 0 – no space Fakavasaga ‘o teu’ 1 – space between ‘o’ and ‘teu’; 0 – no space Fakavasaga ‘teu a’ 1 – space between ‘teu’ and ‘a’; 0 – no space Fakavasaga ‘a kaiga’ 1 – space between ‘a’ and ‘kaiga’; 0 – no space 2 – correct capitalization throughout the sentence (only capitalized letter is the initial word I) Fakaaogaga o 1 – if ‘I’ is capitalized in first word of the sentence, but other words mataimanu lasi incorrectly capitalized; OR other words are lower case, but I not [Correct capitalized capitalization] 0 – if I is not capitalized and other words are incorrectly capitalized 9 – no response, they left it blank Fakaaoga a te fakailoga koma [Use of , 2 – ‘,’; 0 – no correct comma (if student writes ‘koma’ = 0) comma.] Fakaaoga a te fakailoga piliota/poini i te fakaotiga o te ‘ .’ 2 –‘.’; 0 – no correct full stop (if student writes ‘poini’ = 0) fuaiupu. [Use of full stop at end of sentence] 82 Annex 2.B: Student Questionnaire Tuvalu Student Questionnaire : Student context interview Fakafetai! Fakamolemole tali mai te vaega fakaoti ki te mea e mafai ne koe. Please interview the child in a calm and friendly manner. Please circle the answer the child gives without giving them the choices shown in the Student Response Form. Remember to record only the answers the child gives you. 1 What grade were you in last year? Did not attend school……………………………............. ..0 Koe ne vasega fia i te tausaga ko teka? Class 1………………………………………………………………….1 Class 2………………………………………………………………….2 Class 3………………………………………………………………….3 Pre-school……………………………………………………………7 Do not know/ No response……………………………….88 2 Did you go to a preschool (center-based) before class 1? No…………………………………………..…………………………..0 A koe ne akoga i se akoga kamata [preschool] koi tuai Yes………………………………………………………………………1 koe o Vasega 1? Do not know/ No response…………………………….…88 3 Did you eat any food before you arrived at school today? No…………………………………………..…………………………..0 A koe ne kai koi tuai koe o vau ki te akoga i te aso nei? Yes………………………………………………………………………1 Do not know/ No response…………………………….…88 4 What language(s) do you speak at home? Tuvaluan Nui Ne a gana e fakaoga ne koe i te fale? English Samoan [Multiple responses are allowed] Fijian Kiribati Chinese 5. Other: ____________________ 6 What language(s) does your teacher speak in the Tuvaluan classroom? Nui English Ne a gana e fakaoga ne tau faiakoga i te faleakoga? Samoan [Multiple responses are allowed] Fijian Kiribati Chinese Other: ____________________ 7 What does the teacher say or do when someone Neutral response: answers a question correctly in class? Teacher says or does nothing................................. 0 Nea a muna/mea a te faiakoga e fai mafai se tamaliki e lei ana mea akoga? Positive response: Teacher praises student (good job/well done)....... 1 Do NOT read the responses to the pupil. You can use ’fai’ Teacher asks the class to clap …………….…….……...… 2 do or ‘muna’ say when asking this question. Teacher is happy………… ……………………….……....….. 3 [Multiple responses are allowed] Teacher smiles…………………………….…….………….….. 4 Other .................................................................... 5 Don’t know/No response .................................. 88 1 83 15 When you get a good mark in school, do you tell No ....................................................................... 0 someone at home? Yes………………. ……………………............................... 1 Ka fai ne lei au mea akoga, e mata e fakailoa ne koe ki Don’t know/No response ................................. 88 se tino i te fale? 16 [If yes to #15, then ask:] They do nothing …..………………………………. 1 What do they do? They congratulate or encourage me…….. 2 Mafai tau tali e ao, ne a la ana mea e fai? They give me a treat …………………………… 3 [Multiple responses are allowed] Other …….………………………………………… 4 Don’t know/No response .………………… 88 17 Do you have time to read books on your own in your No…………………………………………..…………………….……0 classroom? Yes……………………………………………………………….…….1 E isi sou taimi o faitau a tusi i loto i tou faleakoga? Do not know/ No response……………………………...88 18 Do you have a school library here? Point it out for me. No…………………………………………..………………………….0 Ok, good. Yes……………………………………………………………………..1 E isi se fale tusi i te akonga? Fakasino mai me te fea. Do not know/ No response………………………….…..88 Lei. 19 [If yes to #18, then ask:] No…………………………………………..………………………….0 Do you have time to read books in your school library? Yes……………………………………………………………………..1 E isi sou taimi o faitau a tusi i te fale tusi? Do not know/ No response………………………….…..88 20 Do you take books home from school to read? No…………………………………………..………………………….0 Yes………………………………………………………….…..……..1 E isi ne tusi e fano mo koe ki te fale mai te akoga o Do not know/ No response………………………….…..88 faitau? 21 Are there other books, newspapers or other things to No…………………………………………..………………………….0 read at your house? Yes……………………………………………………………………..1 E isi aka foki ne nisi tusi pela mo nusipepa io me ne nisi Do not know/ No response………………………….…..88 tusi mo ou faitau i te fale? 22 [If yes to 21], What language(s) is used in these books? Tuvaluan………………………………………………….....……..1 Ne a gana e fakaoga i tusi konei? English…………………………………………………….....……..2 Any other language………………………….………...……..3 [Multiple responses are allowed] (List) ------------------------------------------------ Do not know/ No response………………………………88 23 Do you read aloud to someone at home? No…………………………………………..………………………….0 Yes……………………………………………………………………..1 A koe e masani o faitau ki se tino i te fale i se leo maluga? 24 [If yes to 23] How often? Sometimes or every day? Sometimes .......................................................... 1 E fakafia taimi? I nisi taimi, io me i aso katoa? Every day ............................................................ 2 Don’t know/Refuse .......................................... 88 [Read the responses to the pupil.] 25 Do you read just by yourself and to yourself at home? No…………………………………………..……………………..….0 A koe e masani o faitau ia koe loa i te fale? Yes……………………………………………………………..……..1 26 [If yes to 25]How often? Sometimes or every day? Sometimes .......................................................... 1 E fakafia taimi? I nisi taimi, io me i aso katoa? Every day ............................................................ 2 Don’t know/Refuse .......................................... 88 27 Does someone read to you at home? No…………………………………………..…………….........….0 E mata e isi se tino e faitau kia koe i te fale? Yes………………………………………………………........……1 3 84 8 What does the teacher say or do when someone Positive response: answers a question incorrectly in class? Teacher says good effort/good try………………….…..1 Nea a muna/mea a te faiakoga e fai mafai se tamaliki Teacher corrects student .......................................2 e se lei ana mea akoga? Teacher explains question again/asks to try again……………………………………………………………………3 Do NOT read the responses to the pupil. You can use ’fai’ do or ‘muna’ say when asking this question. Neutral response: Teacher says or does nothing………………………………4 [Multiple responses are allowed] Teacher asks another student or says go sit down...................................................................... 5 Negative response: Teacher hits the student…………………………....……….6 Teacher insults or yells at student ........................ 7 Teachers says no………………………………………………....8 Teacher punishes student in some other way………9 Don’t know/No response...................................... 88 9 Does anyone know how to read at your home? No…………………………………………..…………………………..0 E isi aka ne tino i tou fale io me ko tou kaiga e iloa o Yes……………………………………………………………………….1 faitau? Do not know/ No response…………………………….…88 10 [If yes to Question 9] Who is that? Mother………………………………………….……………….......1 Father………………………………………………………........……2 Ko oi? Sister…………………………………………………………………….3 Brother………………………………………………………………...4 [Multiple responses are allowed] Any other person………………………………………………….5 Identify that person ---------------------------------- Do not know/ No response…………………..……………88 11 Does the teacher give you homework? No…………………………………………..…………………………..0 E isi ne au meaakoga [homework] e avatu sale ne te Yes……(sometimes counts)………………………………….1 faiakoga? Do not know/ No response…………………………….…88 12 [If yes to 11] Does anyone help you do your homework? No…………………………………………..…………………………..0 E isi se tino e fesoasoani atu ki a koe o fai au meaakoga Yes……………………………………………………………………...1 [homework]? Do not know/ No response…………………………….…88 13 [If yes to 12] Who is that? Mother…………………………………………………………...….1 Father……………………………………………………………...…2 Ko oi e fesoasoani atu? Sister………………………………………………………………….3 Brother………………………………………………………….…...4 Any other person………………………………………….…….5 [Multiple responses are allowed] Identify that person ---------------------------------- Do not know/ No response…………………..……………88 14 When you come home from school, does someone in No…………………………………………..…………………………..0 the home ask about what you did at school? Yes……………………………………………………………………….1 Ka fai e foki atu koe ki te fale ma oti te akoga, e mata e Do not know/ No response………………………….….…88 isi se tino i te fale e silisili atu ki au mea ne fai i te akoga i te aso tena? 2 85 28 [If yes to 27] How often? Sometimes or every day? Sometimes ........................................................ 1 E fakafia taimi? I nisi taimi, io me i aso katoa? Every day .......................................................... 2 Don’t know/Refuse ........................................ 88 [Read the responses to the pupil] 29 Do you have access to a computer or mobile device at No…………………………………………..………......……….. 0 home? Yes…………………………………………………….....………. 1 E fakaoga ne koe se computer io me se mobile i te fale? 30 [If yes to #29, then ask:] No…………………………………………..……….....………. 0 Do you read on it? Yes……………………………………………………....……… 1 Koe e faitau ki ei? 30 [If yes to #30, then ask:] Never………………………………………………….....….. 0 b How often do you read on it? Sometimes.…………………………………....……........ 1 Every day …………………………………………..…...... 2 E pefea te faitau soko koe kiei? Don’t know/ Refuse .…..……………………........ 88 31 Do you like reading? No……………………………………………………........... 0 Koe e fiafia o faitau? Yes……………………………………………………......... 1 Do not know/ No response………………........ 88 Time Finished: : ____:______am/pm 4 86 Annex 2.C: Head Teacher Questionnaire Tuvalu Ministry of Education, Youth and Sports Early Grade Reading Assessment (EGRA) Head Teacher Questionnaire The Ministry of Education, Youth, and Sports of Tuvalu is conducting a study to better understand how children learn to read. We would like to ask you some questions about your school. • The information obtained in this questionnaire will be used by the Ministry of Education, Youth and Sports to help identify areas where additional support may be needed. • The name of your school, the grade level and class you teach will be recorded, but only so that we can correctly link school, class and student data in order to analyze relationships between children’s learning and the characteristics of the schools in which they learn. Your school’s name will not be used in any report or presentation. • I will read you the consent statement below and mark ‘X’ in the “Yesâ€? box if you agree. Please respond to the questions I ask you as completely and accurately as you can. It should not take you more than 30 minutes for us to complete this questionnaire. • You do not have to complete the questionnaire if you do not want to. CONSENT STATEMENT: I understand and agree to participate in this reading study by filling out this questionnaire with you as completely and accurately as possible. YES SECTION 1: YOUR SCHOOL FACILITY AND THE COMMUNITY We would like your views on your school facility and the community’s role in your school. Name of Island Name of School Age Years____________ Male………………………………………….. 0 Gender Female………………………………………. 1 1 What is your highest level of None ………………………………………… 0 qualification? USP Foundation………………………..….. 1 PSSC (Pacific Senior Secondary Certificate)……………………………….…. 2 FSLC (Fiji School Leaving Certificate)…... 3 1 87 SPFSC (South Pacific Form Seven Certificate)……………………………..….… 4 Certificate in Primary Teaching...……….… 5 Diploma in Primary Education.................... 6 Diploma in Primary Education/In-service… 7 Bachelor of Education (primary) ……….… 8 Masters in Education …...........................… 9 Other (specify) ..……………………………. 10 2 What is your role? HT……………………….……………………… 1 Teacher ……………………….……………… 2 Other ……………………….……………...… 0 3 [If HT]: How many years have you been a head teacher? Years_______________ 4 [If HT]:How many years were you a teacher before becoming a head teacher? Years _______________ 5 No……………………………………………. 0 Is there a PTA at this school made up Yes…………………………………………... 1 of parents and teachers? Don’t know…………………………………… 3 6 Once a week………………………………….. 1 Once a month……………………………….. 2 How often did the PTA meet in the Once a term………………………………… 3 past year? Once a year………………………………… 4 Skip if #5= No or Don't knpw Other………………………………………… . 0 Don’t know…………………………………... 9 7 Is there a kaupole member on the No……………………………………………. 0 PTA? Yes…………………………………………... 1 Skip if #5= No or Don't knpw 8 No…………,,,,,……………………………….. 0 Is there a clean, safe water supply Yes…………..…………………………………. 1 available on the school premises? Sometimes…………………………………… 3 9 No…………,,,,,…………………………..…… 0 Does the school have electricity? Yes…………..……………………………..…. 1 Sometimes…………………………………… 3 10 No………………………………………..…… 0 Does the school have girls’ and boys’ Yes, separate toilets……………………..… 1 toilets? Yes to toilet, but same toilet for girls and boys……………………………………….…… 2 2 88 11- 11 Parent or community member checking on 15 student attendance……. Once a week………………………………… 1 Once a month……………………………… 2 Once a term………………………………… 3 Once a year…………………………………… 4 Never………………………………………..… 5 Other…………………………………………… 6 12 Parent or community member visiting you to resolve problems… Once a week………………………………… 1 Once a month……………………………… 2 Once a term………………………………… 3 Once a year…………………………………… 4 Never………………………………………..… 5 Other…………………………………………… 6 13 Parent or community member helping you at the school ………….. Once a week………………………………… 1 Once a month……………………………… 2 Once a term………………………………… 3 Once a year…………………………………… 4 How many times current school year Never………………………………………..… 5 did a parent or member of the Other…………………………………………… 6 community come to your school to do any one of the following things. 14 Parent or community member being guest [Please read each type of activity and speaker at your school indicate the response of the teacher.] Once a week………………………………… 1 Once a month……………………………… 2 Once a term………………………………… 3 Once a year…………………………………… 4 Never………………………………………..… 5 Other…………………………………………… 6 15 Parent or community helps clean up the school or the school grounds Once a week………………………………… 1 Once a month……………………………… 2 Once a term………………………………… 3 Once a year…………………………………… 4 Never………………………………………..… 5 Other…………………………………………… 6 3 89 16 Did a a parent or member of the No……………………………………........…. 0 community come to your school to Yes……………………………………............ 1 do for any other reason? Don’t know/No response………….......…. 99 16b if yes, specify 17 Semi-permanent (concrete walls, thatch roof OR iron roof, coconut walls)………………….… 1 Which description is most Made from all local materials (coconut walls, appropriate for your school? thatch roof)…………………………………… 2 Permanent (concrete walls, iron roof – all imported materials)………………………….. 3 18- How many teaching days ______ days in the last term 19 (instructional days in which teachers ______ teaching days that actually occurred were face to face with the students) last term (eliminate days such as sports, were there in the last term? cultural, exams, weather, and other days that were not instructional days in which you were Please refer to the school calendar to face to face with students) answer this question. 20 What are the most common reasons Weather (cyclone, rain, tsunami, etc.)…… 1 for school to be closed (apart from Official holidays…………………………..…. 2 school holidays already announced Village celebrations……………………...…. 3 on the school calendar)? (Multiple responses possible.) Sports and cultural events……………...…. 4 No electricity/power outage……………..… 5 No water…………………………………..…. 6 Fire………………………………………..…. 7 Other……………………………………….... 8 School doesn’t close……………………….. 9 4 90 Annex 2.D: Teacher Questionnaire Tuvalu Ministry of Education, Youth and Sports Early Grade Reading Assessment (EGRA) Teacher Questionnaire The Ministry of Education, Youth, and Sports of Tuvalu is conducting a study to better understand how children learn to read. We would like to ask teachers some questions about how they teach reading. • The information obtained in this questionnaire will be used by the Ministry of Education, Youth and Sports to help identify areas where additional support may be needed. • The name of your school, the grade level and class you teach will be recorded, but only so that we can correctly link school, class and student data in order to analyze relationships between children’s learning and the characteristics of the schools in which they learn. Your school’s name will not be used in any report or presentation. • I will read you the consent statement below and mark ‘X’ in the “Yesâ€? box if you agree. Please respond to the questions I ask you as completely and accurately as you can. It should not take you more than 30 minutes for us to complete this questionnaire. • You do not have to complete the questionnaire if you do not want to. CONSENT STATEMENT: I understand and agree to participate in this reading study by filling out this questionnaire as completely and accurately as possible. YES SECTION 1: GENERAL INFORMATION Please answer all questions honestly. Write each response in the space on the right across from each item. Where response options are given, clearly circle the number on the far right that corresponds most closely to your response. For example: 3 Name of Island Name of School Male…………………………………………..1 Gender Female……………………………………….2 Age Years_______________ Name the teacher 1 Class 1 ……….………………………...… 1 What class(es) are you teaching this Class 2………………….…………………. 2 year? [Circle ALL that apply] Class 3…………………………………….. 3 Class 4…………………………………….. 4 Class 5…………………………………..… 5 Class 6…………………………………….. 6 2 What is your highest level of None …………………………………....... 1 qualification? USP Foundation………………………..… 2 PSSC (Pacific Senior Secondary Certificate)……………………………...…. 3 FSLC (Fiji School Leaving Certificate).. 4 1 91 SPFSC (South Pacific Form Seven Certificate)………………………….....….. 5 Certificate in Primary Teaching...……… 6 Diploma in Primary Education..................7 Diploma in Primary Education/In- service…………………………………….. 8 Bachelor of Education (primary) ………. 9 Bachelor of Education (secondary)….... 10 Masters in Education…………............ 11 Other (specify) ..……………………… 12 3 How many years have you been teaching? Years_______________ 4 How many years have you been teaching on this island? Years _______________ 5 Normally, how long does it take you Record response: to get from your home to your (Answer in minutes) school? 6 Does your school provide you with No…………………………………………….0 housing? Yes…………………………………………...1 7 Your school………………………………….1 The ministry of education of Tuvalu……....2 Who provides your transport to Island council (kaupule)……………..……..3 school? You pay transport for yourself……………..4 You walk or bike to school…………………5 SECTION 2: LIBRARY, READING CORNERS AND RESOURCES 8 Does your school have a library? No…………………………………………….0 Yes…………………………………………...1 8a Is the library accessible to No…………………………………………….0 students? Yes…………………………………………...1 9 If yes, do you supervise your No…………………………………………….0 students when they use the library? Yes…………………………………………...1 Skip if #8 or #8a = No Don’t know/No response…………………..3 10 Do you have a reading corner (or No…………………………………………….0 classroom library) in your Yes…………………………………………...1 classroom? 11 Every day……………………………………1 If yes, how often do your students Two or three times a week………………...2 use the reading corner? Once a week………………………………..3 Skip if #10 = No Less than once a week…………………….4 Never…………………………………………5 12 What kind of reading books and Spark readers [EN]………………….....….1 classroom reading resources for Tuvaluan readers [TU]………………..... 2 students do you have at your School Journals[EN]……………………. 3 school? (Can be in a box or in use in Tulip readers[EN]……………………..… 4 the classroom). [Circle ALL that apply] Sunshine readers[EN/TU]……..……….. 5 Pacific folk tales [EN]… ………………… 6 Big Books [EN]…………………………… 7 Others (specify) ………………………… 8 None ……………………………………… 0 2 92 How often do you use the following books in Tuvaluan reading lessons? Book type Never Occasion About Most but Every ally half the not lesson time every lesson 13 Spark readers 1 2 3 4 5 14 Tuvaluan readers 1 2 3 4 5 15 School Journals 1 2 3 4 5 16 Tulip readers 1 2 3 4 5 17 Sunshine readers 1 2 3 4 5 18 Pacific folk tales 1 2 3 4 5 19 Big Books 1 2 3 4 5 20 Others (please specify): 1 2 3 4 5 _____________________ SECTION 3: TEACHING AND LEARNING I will ask you some questions about the activities you did with your students over the last five days of school. Please tell me if you did each activity daily, 3 or 4 days a week, 1 or 2 days a week, not in the past 5 days, but you do it sometimes, or never Activity Never Not in the On 1 or On 3 or Daily last 5 2 days a 4 days a days, but I week week do it sometimes 21 Listening 1 2 3 4 5 comprehension (children listen to a story and answer questions) 22 LOBO – low order book 1 2 3 4 5 orientation 23 Children orally retell a 1 2 3 4 5 story that they have read 24 Children learn new 1 2 3 4 5 letter sounds 25 Children sound out 1 2 3 4 5 unfamiliar words using knowledge of letter sounds 26 Children learn 1 2 3 4 5 meanings of new words/vocabulary 27 Whole class reading 1 2 3 4 5 28 Guided reading (teacher 1 2 3 4 5 listens to children read in small groups) 3 93 29 Listening to a child read 1 2 3 4 5 aloud to you one on one 30 Students reading on 1 2 3 4 5 their own silently 31 Reading 1 2 3 4 5 comprehension activities (orally or in writing) 32 Children take books 1 2 3 4 5 home to read with their parents 33 Evaluating students’ 1 2 3 4 5 oral reading with running records or any other method 34 Other activities (please 1 2 3 4 5 describe): SECTION 4: READING IN THE CURRICULUM Listed below are five important early reading skills which students must learn. For each reading skill, circle the class level when you think students are able to: Reading skill Class 1 Class Class 3 Class 4 Class 5 Not 2 important 35 Recognize and say letter 1 2 3 4 5 0 names 36 Understand stories read 1 2 3 4 5 0 aloud by the teacher in the classroom 37 Sound out words using 1 2 3 4 5 0 phonics 38 Read aloud to teacher 1 2 3 4 5 0 and other students 39 Understand simple texts 1 2 3 4 5 0 that they read in class SECTION 5: IN-SERVICE TRAINING AND TEACHER SUPPORT MATERIALS We would like your views on teacher support for reading. 40 No…………………………………………….0 Do you have a Fakavaetaki Yes, paper copy………………….………...1 [syllabus] for Gana Tuvalu? Yes, copy on computer…………………….2 Don’t know/no response………………3 41 If yes to Question 40, how useful do Not very useful……………………………...1 you find the Fakavaetaki [syllabus] Quite useful…………………………………2 for Gana Tuvalu? Very useful………………………………….3 4 94 42 No…………………………………………….0 Do you have an English language Yes, paper copy………………….………...1 syllabus? Yes, copy on computer…………………….2 Don’t know/no response …………………3 43 If yes to Question 42, how useful do Not very useful……………………………...1 you find the English language Quite useful…………………………………2 syllabus? Very useful………………………………….3 44 Have you received any training on No…………………………………………….0 how to teach reading in the last Yes…………………………………………...1 three years? 45- If yes to Question 44, indicate the Name of training_________ 46 name of the training and how many hours of reading training you have Total hours__________ received in total (approximately). 47 If yes to Question 44, what was the most useful aspect of this training? __________________________ SECTION 5: PARENTS AND THE COMMUNITY We would like your views on the community’s role in your school. 48- 48 Have a parent-teacher meeting with 54 you……... Once a week……………………………........ 1 Once a month………………………………… 2 Once a term…………………………………… 3 Once a year…………………………………… 4 Never………………………………………..…. 5 Other…………………………………………… 6 49 Parent or community member checking on student attendance……. Once a week……………………………........ 1 Once a month………………………………… 2 Once a term…………………………………… 3 How many times during the last Once a year…………………………………… 4 term did a parent or member of Never………………………………………..…. 5 the community come to your Other…………………………………………… 6 class to do any one of the following things. 50 Parent or community member visiting [Please read each type of classroom to resolve problems… activity and indicate the Once a week……………………………........ 1 response of the teacher.] Once a month………………………………… 2 Once a term…………………………………… 3 Once a year…………………………………… 4 Never………………………………………..…. 5 Other…………………………………………… 6 51 Parent or community member helping you in the classroom………….. Once a week……………………………........ 1 Once a month………………………………… 2 Once a term…………………………………… 3 Once a year…………………………………… 4 Never………………………………………..…. 5 Other…………………………………………… 6 5 95 52 Parent or community member being guest speaker in your class Once a week……………………………........ 1 Once a month………………………………… 2 Once a term…………………………………… 3 Once a year…………………………………… 4 Never………………………………………..…. 5 Other…………………………………………… 6 53 Parent or community helps clean up the school or the school grounds Once a week……………………………........ 1 Once a month………………………………… 2 Once a term…………………………………… 3 Once a year…………………………………… 4 Never………………………………………..…. 5 Other…………………………………………… 6 54 Did a parent or member of the Yes community come to your No school to do for any other Does not know reason? 54b if yes, specify SECTION 6: TEACHER ATTENDANCE We would like get a few information about your attendance. 55 How many days were you absent None…………………………………….... 1 from school in the last term? 1-5 days………………………………...... 2 6-10 days………………………….……….3 Over 11 days……………………….….… 4 Don’t remember……………………....… 5 56 What are the most common Weather (cyclone, rain, tsunami, etc.)…. 1 reasons for you to be absent You are sick …………………………….. 2 from school? (Apart from official Caring for a sick relative………………… 3 holidays) Family functions (Funeral, Wedding, Birthdays)…………………………………. 4 Tick all that apply Travel……………………………………… 5 Other (specify)………….………………… 6 56b If other, please specify 57-58 How many boys and how many Number of boys……….. girls are in your class? Number of girls………. [Indicate numbers by gender] 6 96 59 What are the most common Weather (cyclone, rain, tsunami, etc.)… 1 reasons for students to miss Student sickness………………………… 2 school? Caring for siblings……………………… 3 Family problems………………………… 4 Tick all that apply Family finances………………………… 5 Family function (Funeral, Wedding, Birthdays)………………………………… 6 Travel……………………………………… 7 No clean uniform………………………… 8 No lunch………………………………..… 9 No money for school materials…………10 Internet addiction (went to Internet café)……………………………………… 11 Student didn’t want to go……………… 12 Parent didn’t care……………….....…… 13 Student scared of the teacher………… 14 Haven’t done their homework………… 15 Bullying……………………………………16 No transport/nobody brings them to school…………………………………… 17 Other…………………………………… 18 60 Is there any other comment or information you would like to provide me? Thank you very much for taking the time to complete this questionnaire! 7 97 Tuvalu Early Grade Reading Assessment Classroom Observation A. Date of observation: B. School name C. School Code: D. location North Central E. region South Funafuti F. Observer Name G. Year observed: If combined classes, indicate which classes have been combined o 1= o 2= o 3= o 4= o 5= o 6= H. For Nauti P.S Indicate name of teacher: ___________________________________ J. Subject observed o 1 = Language (Reading) o 2 = Language (Other) o 3 = Other subject K. Time observation started: L. Time observation finished: _____ : ____ (circle AM or PM) AM / PM ____ : ___ (circle AM or PM) AM / PM INSTRUCTIONS 1. Have the required number of observation guides on your clipboards before you travel to the schools. 2. Sharpen two pencils and attach to your clipboard before you travel to the schools. 3. Carry a pencil sharpener at all times. 4. Observe a class that has a teacher present on the day of the classroom observation. 5. Make a point of introducing yourself to the teacher before the observation session. 6. Ask the teacher if you can observe a reading lesson. If this is not possible and this is the only classroom to observe, proceed with the observation. 7. The whole classroom observation should last one hour. Keep to this time allocation at all times. 8. Collect a random sample of 5 language exercise books to assess. Take about 10 minutes at the end of the observation and return the books to the teacher before you leave. 9. Blend into the classroom environment during the observation sessions. 10. Please complete ALL sections of this form, including the comments section. It is important to provide as much detail as possible. 11. Thank the teacher at the end of the lesson. 1 98 Fill the following 2 tables according to the number of levels present in the classroom at the time of observation CLASS ENROLLMENT (asking each teacher to prepare a summary so may not need this) Write down the total number of students enrolled in the class observed (look in the roll book for this information). If combined classes, record the enrollment for each class. Class Male Female Total 1 2 3 4 5 6 Comments: 2. ATTENDANCE (Count students when you sit down to begin the observation quietly from your seat). Write down the number of students who are actually in class today (count the children in class). If combined classes, record the attendance for each class. Class Male Female Total 1 2 3 4 5 6 Comments: 3. CLASSROOM ENVIRONMENT Tick the appropriate response to show what you can actually see in the classroom. Tick ALL sections. Resources Yes No 1. Tuvaluan alphabet displayed in classroom 2. Classroom has Tuvaluan print in common teaching aids and wall displays ¨ calendar ¨ days of week ¨ alphabet ¨ numbers ¨ weather ¨ None of above 3. List of Tuvaluan spelling/vocabulary words written on black board 2 99 4. List of Tuvaluan spelling/vocabulary words written on charts/posters 5. Tuvaluan stories written on the black board 6. Tuvaluan hymns/songs written on the black board 7. Tuvaluan stories written on charts/posters 8. Tuvaluan hymns/songs/prayers written on charts/posters 9. Recent students’ written work (from this school year) displayed around classroom 10. Student written work from previous years displayed around classroom 11. Sufficient classroom space for organized group activities 12. Other print materials are used in instructional activities ¨ newspapers ¨ magazines ¨ flash cards ¨ food wrappers and packages (for example, sugar, biscuits, rice) ¨ prepaid cards ¨ objects in treasure boxes in classroom ¨ Other materials ¨ None 12b. Other, specify ______________________________________ 13. Student reading corner in the classroom 14. Student Profiles (folder with student work and student information) 15. Students sitting on floor 16. Students sitting in desks or chairs Comments: 4. TEACHING AND LEARNING Tick the appropriate response to show what you actually see going on in the classroom. Include as much detail as you can in the comments section. Activity Yes No 17. Teacher having students practice letter sounds (for example: m, b, k, t, a, e, etc.) 18. Teacher having students practice syllable sounds (for example: ma, me, mi, mo, mu) 19. Students learning the meaning of new words/vocabulary 20. Students singing song with the teacher. 21. Teacher leading students in activity or song with letters and words in it and body movements that go with it 22. Teacher reading story/text aloud to students 23. Students reading story/text aloud with teacher 24. Teacher asking comprehension questions when reading story/text 25. Students assisting peers to read (buddy reading) 26. Students reading to each other in groups 27. Students reading silently on their own 3 100 28. Varied reading activities during lesson. ¨ Spelling ¨ Students reporting out about a story or text ¨ general discussions of teacher and students about a book or story ¨ building new words ¨ creating new sentences & stories ¨ Other ¨ None 28b Other activities, specify Comments: 5. LANGUAGE USED BY TEACHER AND STUDENTS IN CLASSROOM Tick the boxes to show what you observed about the language used in the classroom. Language Tuvaluan Nui English Other 29. Language(s) of instruction used by teacher 30. Language(s) students use to communicate with each other during classroom activities 31. Language(s) students use to communicate with the teacher. 32. Language(s) written on the black board to teach content 33. Language(s) written on charts, posters, classroom displays and other teaching aids. Comments: 6. TEACHER’S USE OF TUVALUAN Tick the appropriate response. Activity Yes No 34. Does the teacher switch languages when teaching in Tuvaluan class? (Write examples in the Comments section below.) 35. Are there errors in the teacher’s written text in Tuvaluan on the blackboard? (Write examples in the Comments section below.) 36. Two different writing systems have been used for Tuvaluan. One writing system uses the standard letters: a, e, i, o, u, f, g, etc. Does the teacher use this writing system when teaching or writing Tuvaluan? 37. Tick here the letters ¨ o ¨ h ¨ f ¨ n the teacher is teaching ¨ u ¨ m ¨ s ¨ p today: ¨ g ¨ k ¨ v ¨ Not ¨ A ¨ t ¨ l teaching ¨ e letters ¨ i 38. The other writing system includes other symbols like: Ä?, Ä“, Ä«, Å?, Å«, ‘p, ‘n or pp, nn. Does the teacher use these additional symbols when teaching or writing Tuvaluan? 4 101 Comments: 7. RESOURCES Write the number of sets of different titles in the appropriate column to show what resources you actually see in the classroom. Please comment if there are books stored elsewhere and not in the classroom. Resources in classroom None 1 to 5 6 to 10 11 to 20 More than 20 39. Number of Spark readers 40. Number of Tuvaluan readers 41. Number of School Journals 42. Number of Tulip readers 43. Number of Sunshine readers 44. Number of Pacific folk tales 45. Number of Big Books 46. Other Comments: 47. Count up all the reading books for students that you can see being used during the lesson. Write the total in the box below: 5 102 8. DO THE FOLLOWING AT THE END OF THE OBSERVATION. Ask the teachers for a sample of five or six students’ language exercise books. Areas to take note of in students’ language exercise books (tick ALL sections): Activity Ex book 1 Ex book 2 Ex book 3 Ex book 4 Ex book 5 48. Filling in missing words, sentence beginnings and endings, etc. 49. Short sentences 50. Short stories (2-5 sentences) 51. Reading comprehension activities 52. Labeling things or matching 53. Read and draw exercises 54. Regular written work in students’ language exercise books 55. Exercises in students’ language books marked by the teacher regularly 56. Written corrections and feedback given by teacher 6 103 Tick the appropriate response to show the language(s) written in the students’ exercise books: Ex Book 1 Ex Book 2 Ex Book 3 Ex Book 4 Ex Book 5 Tuv Eng Nui Oth Tuv Eng Nui Oth Tuv Eng Nui Oth Tuv Eng Nui Oth Tuv Eng Nui Oth 57. Language(s) written in children’s exercise books 9. Additional Comments/Observations 7 104