89472 [Book Section] 1 R U S S I A E D U C AT I O N A I D F O R D E V E L O P M E N T T R U S T F U N D READ ANNUAL REPORT 2013 2 READ Trust Fund 2013 Annual Report This report was prepared by Emily Gardner under the guidance and direction of Marguerite Clarke, the READ Trust Fund Program Manager. Valuable inputs and comments were provided by members of the READ Trust Fund Global and Country Teams, including Cristian Adeo, Saodat Bazarova, Chelsea Coffin, Michael Crawford, Dingyong Hou, Ana Jurema, Lorelei Lacdao, Julia Liberman, Diana Manevskaya, Sophie Naudeau, Jem Heinzel Nelson, Vidya Putcha, Maria Jose Ramirez, Cristina Panasco Santos, Gulmira Sultanova, An Thi My Tran, Stephanie von Wogau, and Girma Woldetsadik. Design: Elizabeth Salud   Front Cover Photo Credit: World Bank/Zambia Message from the READ Council For both the Russian Federation and the World Bank, the partnership The READ through the READ Program continues to be groundbreaking. As a result of this innovative program, many developing countries have received valuable program continues support in the area of student assessment, many Russians have strength- to increase global ened their expertise in assessment, and several highly useful knowledge tools have been created. attention on the At the global level, the READ Trust Fund continues to contribute to the importance of development of an impressive body of knowledge products on student education quality assessment systems with the result that there is now a much greater understanding of what it takes to develop an effective assessment system. and learning At the country level, many of the countries receiving support under the outcomes. READ Trust Fund have been able to develop parts of their assessment systems in significant ways and lay a strong foundation for future work in this area. Through the joint efforts of Russia and the World Bank, the READ program continues to make significant and recognized contributions to the Learning for All agenda and to increase global attention on the importance of educa- tion quality and learning outcomes. We are pleased to receive the READ Trust Fund Annual Report for 2013, which clearly tracks the progress and impact of the program in its fifth year. Andrei Bokarev Luis Benveniste Head of the Department for Education Sector Manager for East International Financial Affairs, Asia and the Pacific, World Bank Ministry of Finance, Russian Alberto Rodriguez Federation Education Sector Manager Vladimir Mau for Europe and Central Asia, Rector, Academy of National World Bank Economy under the Government of the Russian Federation Andrei Volkov Dean, Moscow School of Management (SKOLKOVO) LIST OF ACRONYMS AEAA Association for Educational Assessment in Africa AIR American Institutes for Research ATC Assessment Testing Center CICED Center for International Cooperation in Education Development CNECE Examinations Council of Mozambique EAOKO Eurasian Association for Educational Assessment ECZ Examinations Council of Zambia EGRA Early Grade Reading Assessment IAEA International Association for Educational Assessment ICT Information and Communications Technology ILSA International Large-Scale Assessment INDE National Institute of Education IT Information Technology NEAEA National Educational Assessment and Examinations Agency NLSA National Large-Scale Assessment NTC National Testing Center OECD Organization for Economic Co-operation and Development PISA Program for International Student Assessment READ Russia Education Aid for Development SABER Systems Approach for Better Education Results SAM School Achievements Monitoring TIMSS Trends in International Mathematics and Science Study UEE University Entrance Examination UIS UNESCO Institute for Statistics TABLE OF CONTENTS Background ........................................................................................................................ 2 2013 Highlights .................................................................................................................. 6 Program Achievements.............................................................................................. 7 Financial Summary ..................................................................................................10 Global-level Work.............................................................................................................. 12 SABER–Student Assessment ................................................................................. 13 SABER–Learning Standards ................................................................................... 19 Global Knowledge Sharing ..................................................................................... 20 Program Coordination ............................................................................................. 24 Country-level Work ........................................................................................................... 26 Angola ..................................................................................................................... 29 Armenia.................................................................................................................... 34 Ethiopia.................................................................................................................... 38 Kyrgyz Republic ...................................................................................................... 42 Mozambique ........................................................................................................... 46 Tajikistan ................................................................................................................. 50 Vietnam ................................................................................................................... 54 Zambia .................................................................................................................... 58 Looking Ahead ................................................................................................................. 63 Appendices....................................................................................................................... 64 Glossary of Assessment Terms............................................................................... 64 2013 AEAA and IAEA Conferences ......................................................................... 66 World Bank Symposium—Assessment for Global Learning .................................. 67 2 READ Trust Fund 2013 Annual Report Background Established in October 2008, the Russia Education Aid for Development (READ) program is a collaboration of the Government of the Russian Federation and the World Bank that focuses on improving education quality in low-income countries. The READ Trust Fund is a part of this program, with an amount of US$32 million to be executed over a six-year period, 2008 to 2014. Better Assessment for Better Learning The World Bank’s Education Sector Strategy 2020 highlights the impor- tance of “Learning for All.” The proven economic gains and poverty reduc- tion tied to education are only obtainable when children actually learn. All actors in an education system need to know whether or not learning is taking place so that they can use this information to improve education quality. The READ Trust Fund’s main purpose is to help low-income countries improve their student learning outcomes through the design, implementa- tion, and use of robust systems for student assessment. It supports analytical work and technical assistance to help countries: àà establish systems or institutions—or strengthen existing ones—that formulate learning goals and carry out assessments of student learning; àà improve existing or develop new instruments to measure student learning outcomes; and àà strengthen existing or develop new mechanisms (policies) to use learning outcomes data to improve teaching and learning. Background 3 READ Trust Fund Organization and Structure Under the guidance of the READ Council, a group comprised of key Russian and World Bank officials, the READ Trust Fund supports work at both the global and country levels. At the global level, the focus is on generating and sharing knowledge and good practices in the form of tools, analytical reports, and case studies. At the country level, World Bank teams and country stakeholders work together to develop and implement a set of READ Trust Fund–supported activities that address gaps in the country’s current learning assessment system. For both the global- and country-level work, an international group of assessment experts provides advice and support for product development, program design, and program implementation (see Figure 1). Figure 1. Roles and Responsibilities of Key Actors under the READ Trust Fund Program READ Council (Russia, World Bank) Guidance on overall strategic direction, priorities, and rules of engagement for the READ Trust Fund, and monitoring of program performance READ Trust Fund Management (World Bank) Program management and administration, i.e., strategic planning, allocation of funds, program coordination, quality assurance, and monitoring Global Team Country Teams (World Bank, (World Bank) Education Officials) Development of the conceptual Focal point for all READ Trust framework for work on assess- Fund assistance in target ment systems, benchmarking countries, working together to tools, and other global knowledge design and implement READ products Trust Fund-supported activities Technical Group (Assessment Experts) Support for development of global knowledge products, program quality assurance, and direct technical assistance at the country level 4 READ Trust Fund 2013 Annual Report Targeted Country Support At the country level, programs are underway in eight countries—Angola, Armenia, Ethiopia, the Kyrgyz Republic, Mozambique, Tajikistan, Vietnam, and Zambia. In each country, World Bank teams have been working closely with government counterparts to thoroughly review the existing assess- ment system, decide on activities to address any gaps, and implement and evaluate the effectiveness of those activities. Some countries, like Ethiopia and Zambia, have opted to address gaps in multiple areas, while others, like Tajikistan, have chosen to concentrate their efforts on a specific issue. All country programs include a large portion of capacity-building activities to strengthen in-country skills in assessment. Expected Outcome: Improved Learning Assessment Systems The main outcome of the READ Trust Fund will be increased institutional capacity of countries to develop, carry out, and effectively use data from student assessments to improve education quality and student learning. Armed with information on how well students are performing, teachers, policy makers, and international donors alike will be better able to deter- mine where to focus their energy and resources for the greatest improve- ment in learning outcomes. READ Trust Fund (2009–2012) Prior to 2013, the READ Trust Fund had been operating for four years (2009–2012). During this time, the program went through various phases, including a start-up phase to establish the program, a design phase to plan and agree on various program activities, and an implementation phase to carry out the initial set of activities. For example: àà In 2009, the focus was on establishing the program at the global and country levels. The trust fund structure was created, and a self- diagnosis exercise was carried out in each of the target countries in order to identify gaps in their existing assessment systems and create action plans for READ Trust Fund assistance. àà During 2010 and 2011, work at the global level centered on the design and validation of a conceptual framework and tools for analyzing and strengthening assessment systems. At the country level, several countries began implementing their first sets of activities, with a concentration on capacity building. àà In 2012, program implementation at the country level accelerated and the program began to see concrete, measurable results according to the key indicators being tracked under the READ Trust Fund Results Framework. At the global level, several key publications were finalized and training materials on the conceptual framework and tools began to be formalized. Background 5 Tracking Progress through 2013 The READ Trust Fund is currently organized around 12 main areas of program activity (see Figure 2). The remainder of this report provides updates for each program area as follows: overall highlights for 2013; cumulative global-level work (2009–2013); and cumulative country-level work (2009–2013). Together, these sections provide a complete picture of program achievements to date, including what each country has done to strengthen their student assessment system. Figure 2. Main Areas of Program Activity under the READ Trust Fund READ Trust Fund Global-level Work Country-level Work SABER– SABER– READ READ READ READ Student Learning Kyrgyz Angola Armenia Ethiopia Assessment Standards Republic Program Knowledge READ READ READ READ Coordination Sharing Mozambique Tajikistan Vietnam Zambia 6 READ Trust Fund 2013 Annual Report 2013 Highlights By the end of 2013, the activities supported by the READ Trust Fund program had made an undeniable impact on the work being carried out worldwide to strengthen systems that measure learning outcomes. The activities and outcomes achieved under the READ Trust Fund have been referenced in many high-level conversations on defining the post-2015 development agenda in education and on identifying effective ways to strengthen countries’ capacity to measure learning outcomes. At the global level àà More than 50 countries have used the SABER–Student Assessment tools developed under READ to analyze the status of their assessment systems. àà More than 15 different knowledge products on assessment topics have been published. àà Over 2,000 individuals have benefited from trainings and learning events on the SABER–Student Assessment tools. At the country level àà All eight READ-beneficiary countries have significantly increased their capacity to develop, carry out, and use data from assessment activities. àà Five new policies governing assessment activities have been adopted, and eight new testing instruments have been developed. àà More than 5,000 individuals have received some level of training on assessment. àà To sustain these capacity-building activities, the development of 13 formal training courses on assessment for teachers, school leaders, and education officials is underway. 2013 Highlights 7 Highlights of Global-level Achievements in 2013 SABER–Student Assessment àà Nearly 250 World Bank staff, representatives from other develop- ment agencies, and students and education officials from the Russian Federation were trained on key topics in student assessment and on the use of the SABER–Student Assessment tools. àà The SABER–Student Assessment tools were applied to 20 additional countries and full country reports are underway. àà Four new case studies on countries’ best practices and lessons learned in developing student assessment systems were —  published  one on the Gambia, two on the Russian Federation, and one on Uganda. Global Knowledge Sharing àà Through a partnership with the Association for Educational Assessment in Africa (AEAA) and the International Association for Educational Assessment (IAEA), 30 education officials and World Bank staff from the eight READ-supported countries participated in these associ- ation’s annual conferences and shared their results under READ with more than 700 assessment experts from over 60 different countries. àà Over 100 education leaders, technical experts, and policy researchers participated in a high-level symposium on “Assessment for Global Learning,” which drew greater attention to the role of assessment in the “Learning for All” and post-2015 development agendas. àà More than 2,000 additional copies of the READ Working Papers and 300 copies of books in the National Assessments of Educational Achievement series were disseminated. Program Coordination àà Three READ Council meetings were organized to monitor the program and set the agenda for a possible second READ program. àà Through collaborative efforts with READ Fee-Based Services and CICED, training sessions on the SABER–Student Assessment tools were conducted for more than 130 Russian students, faculty, and education officials. àà A presentation on CICED’s newly-developed School Achievements Monitoring tool was delivered during the World Bank’s high-level symposium on “Assessment for Global Learning”, in part as a way to introduce the tool to the wider development community. 8 READ Trust Fund 2013 Annual Report A classroom of primary school students in the Kyrgyz Republic. Highlights of Country-level Achievements in 2013 Angola àà Fifty Assessment Unit staff and representatives from all 18 teacher training institutions in Angola received training in Classroom Assessment and preparations were made to create training modules on Classroom Assessment for all pre- and in-service teachers. àà As part of the work to develop a national strategy for assessment, 34 Assessment Unit and provincial staff responsible for overseeing examinations received training in how to standardize and conduct National Examinations. Armenia àà Twenty-five assessment practitioners and academics participated in an intensive training program on topics related to National Large- Scale Assessments, including statistics, item analysis, and test questionnaire development. àà Training courses on Classroom Assessment and Examinations for pre- and in-service teachers were developed and piloted during workshops with 45 university lecturers and 25 Armenian teachers. Ethiopia àà More than 60 Ministry of Education staff at the central and regional levels were trained in how to use the new school inspection framework and guidelines, and nearly 28,000 schools participated in school inspection activities. àà After Ethiopian education officials participated in their first AEAA Conference and held a workshop to share lessons with all key educa- tion leaders, a decision was made for Ethiopia to become an official AEAA member. 2013 Highlights 9 Kyrgyz Republic àà A training program on classroom-based assessments for in-service teachers was developed and delivered to more than 50 lead teachers from all seven provinces. àà Over 40 National Testing Center staff received training in item devel- opment and an improved school leaving Examination was developed. Mozambique àà Eight Ministry of Education staff in Mozambique completed their courses and earned their Masters degrees in Education Management and Evaluation. àà The pilot of a newly-developed classroom assessment tool called Provinha was scaled-up from 68 schools in two districts to 188 schools in four districts, including two districts in Northern Mozambique. Tajikistan àà Upon the successful completion of the full-scale pilot of the new Unified University Entrance Examination, an event was held during which the President of Tajikistan officially inaugurated the new system. àà The construction of the new National Testing Center building was completed and 60 full-time staff members were hired to implement the new system. Vietnam àà Nine workshops with a total of nearly 300 representatives from the Ministry of Education, universities, and school leaders were held to finalize the overall framework and plans for training in assessment for pre-service teachers, in-service teachers, Masters students, educa- tion managers, and Ministry of Education assessment specialists. àà A two-day workshop with 100 education leaders from various depart- ments within the Ministry of Education was held to explore the possi- bility of delivering one or more of the assessment training programs through online e-learning. Zambia àà To communicate the findings of the 2012 National Assessment in grade 5, more than 5 communication materials were created, includ- ing a 10-minute film entitled, “Are Zambia’s Children Learning?” and a DVD with a teacher discussion guide and two case studies on effective teaching strategies. àà More than 30 Ministry of Education staff and 320 Education Standards Officers received training on issues related to carrying out and using results from National Large-Scale Assessments and Examinations. 10 READ Trust Fund 2013 Annual Report Figure 3. READ Trust Fund Allocations Figure 4. Breakdown of Total Disbursements by Global vs. Country 12 10 8 US$ MILLIONS 60% 35% 6 4 2 5% 0 2009 2010 2011 2012 2013 Global-level WB Trust Fund Global-level Management Fee Country-level Country-level Financial Summary The READ Trust Fund was designed to support activities at both the global and country levels. Initially, the fund was set up to be executed only by World Bank staff. However, mid-way through the program, it was agreed to allow for some of the funds to be executed by recipient country govern- ments. (The Kyrgyz Republic and Tajikistan are the only two countries with recipient-executed programs.) Global-level vs. Country-level Activities The US$32 million under the READ Trust Fund has been distributed so that 35 percent of the funds have been set aside for global-level activities and 60 percent for country-specific work programs (see Figure 3). While disbursements were initially greatest at the global level, rates of disburse- ments at the country level significantly increased in 2013 (see Figure 4). Reallocation of Country-level Funds In early 2013, after a careful review of the program, the READ Council determined that the program was primarily on track and well positioned for the final two years of implementation. However, for Vietnam (the country that had the largest Bank-executed grant at US $3.5 million), it was deemed unlikely that they would be able to spend down their remaining funds before the end of the program. The READ Council agreed to reallocate US$1 million of these funds to two other countries that had received —  smaller initial allocations  Armenia and Mozambique. Armenia received US$400,000 while Mozambique received US$600,000. Table 1 includes a financial summary of all program activity through the end of 2013. 2013 Highlights 11 Table 1. READ Trust Fund Financial Summary as of 12/31/13 (in U.S. dollars) Total Allocated Disbursements Commitments Balance Management Fee WB TF Management Fee 1,600,000 1,600,000 0 0 SUBTOTAL 1,600,000 1,600,000 0 0 Global-level Work SABER–Student 4,200,000 3,118,115 198,288 883,598 Assessment SABER–Learning 805,724 805,724 0 0 Standards Knowledge Sharing/ 5,870,000 4,133,967 25,643 1,710,390 Program Coordination SUBTOTAL 10,875,724 8,057,806 223,931 2,593,988 Country-level Work READ Angola 2,180,054 1,466,568 460,568 252,918 READ Armenia 1,400,000 669,041 310,641 420,317 READ Ethiopia 1,089,705 552,899 285,013 251,793 READ Kyrgyz Republic 2,345,729 739,659 1,370,755 316,109 READ Mozambique 3,025,000 2,394,703 200,831 535,963 READ Tajikistan 4,095,089 2,390,367 1,476,485 228,237 READ Vietnam 2,811,894 1,331,810 565,565 914,520 READ Zambia 2,350,242 2,189,165 0 161,078 SUBTOTAL 19,297,714 11,734,213 4,669,857 3,080,935 TOTAL OVERALL $31,773,438 $21,392,019 $4,893,788 $5,674,923 12 READ Trust Fund 2013 Annual Report Global-level Work READ Trust Fund activities at the global level are divided into four main categories—SABER–Student Assessment, SABER– Learning Standards, Global Knowledge Sharing, and Program Coordination. Global-level activities are centered on: àà developing knowledge products to expand the current understanding of what makes for an effective student assessment system; àà partnering and knowledge sharing with other donors and education development practitioners; and àà providing opportunities to share lessons across READ Trust Fund–supported countries and best practices in the field of student assessment more generally. The work in the areas of SABER–Student Assessment and SABER–Learning Standards is part of a larger World Bank initiative called Systems Approach for Better Education Results (SABER) (see Box 1). Many countries have a strong commitment to improving their assessment activities, but they need sustained support from the international community to ensure that the changes take hold. Global-level Work 13 SABER–Student Assessment SABER–Student Assessment makes available information and tools based on the latest evidence for “what matters most” in creating effective assess- ment systems. It involves: àà classifying and analyzing student assessment systems around the world according to key criteria; àà collecting information on these key criteria by administering a set of questionnaires in a particular country; àà producing country and regional reports to inform policy dialogue; and àà developing other products to add to the current knowledge base on how to develop and strengthen student assessment systems. Box 1. Collecting and Analyzing Data on Education Policies around the Globe The World Bank has created an open data tool that provides in-depth, comparative, and easily accessible data on education policies around the world. This tool, SABER, helps countries collect and analyze infor- mation on their education policies, benchmark themselves against other countries, and prioritize areas for reform, with the goal of ensuring that all children and youth go to school and learn. SABER aims to improve education quality by supplying policymakers, civil society, school administrators, teachers, parents, and students with meaningful data about key education policy areas, including early childhood development, student assessment, teachers, school autonomy and accountability, and workforce development, among others. SABER helps countries improve their education systems in three ways: àà Providing new data. SABER collects comparable country data on education policies and institu- tions to allow governments, researchers, and other stakeholders to measure and monitor progress. àà Benchmarking education policies and institutions. Each policy area is rated on a four-point scale, from “Latent” to “Emerging” to “Established” and “Advanced.” These ratings highlight a country’s areas of strength and weakness while promoting cross-country learning. àà Highlighting key policy choices. SABER data collection and analysis produce an objective snapshot of how well a country’s education system is performing in relation to global good practice. This helps highlight the most important policy choices to spur learning. To date, the World Bank, through SABER, has analyzed data from more than 100 countries to guide more effective reforms and investments in education at all levels, from pre-primary to tertiary education and workforce development. (Source: World Bank, www.worldbank.org/education/saber.) 14 READ Trust Fund 2013 Annual Report Public Disclosure Authorized S A B E R – S Y S T E M S A P P R O A C H F O R B E T T E R E D U C AT I O N R E S U LT S STUDENT ASSESSMENT 1 S A B E R – S Y S T E M S A P P R O A C H F O R B E T T E R E D U C AT I O N R E S U LT S STUDENT ASSESSMENT 3 Rubric Structure for Evaluating Data Collected on a Student Assessment System   DEVELOPMENT LEVEL LATENT EMERGING ESTABLISHED ADVANCED DIMENSION JUSTIFICATION EC—ENABLING CONTEXT Public Disclosure Authorized EC1—Policies 2010 Pilot Exercise What Matters EC2—Leadership, public engagement Disseminating Most for Student EC3—Funding and Using Student Survey of Educational Assessment Systems Assessment Systems: EC4—Institutional arrangements Assessment Public Disclosure Authorized A Framework Paper EC5—Human resources Information in Chile Classroom Assessment SA—SYSTEM ALIGNMENT SA1—Learning/ Marguerite Clarke quality goals María-José Ramírez BENCHMARKING SA2—Curriculum EDUCATION SYSTEMS FOR RESULTS Public Disclosure Authorized SA3—Pre-, in- service teacher training AQ—ASSESSMENT QUALITY The World Bank AQ1—Ensuring quality (design, administration, Human Development Network analysis) AQ2—Ensuring effective uses   © 2010 The World Bank Group, All Rights Reserved. September 1, 2010 Conceptual Framework and Diagnostic Tools Work on the SABER–Student Assessment knowledge products and tools began in 2010. Initially, the focus was on creating a conceptual framework and identifying the policies, institutional arrangements, and practices that matter most in establishing effective assessment systems (see Box 2). Based on this framework, a series of questionnaires and rubrics for assess- ing progress in the key indicator areas was developed. The same indicator areas were used to develop the READ Trust Fund Results Framework, which is being used to track each READ Trust Fund–supported country’s progress in strengthening its overall assessment system (see page 29). Early versions of the conceptual framework and corresponding question- naires and rubrics were piloted in Chile, New Zealand, and Uganda. Based on lessons learned from this pilot and other experiences, the tools have continued to undergo minor revisions and fine tuning. In addition to the finalized conceptual framework and “What Matters Most for Student Assessment Systems” paper, questionnaires and rubrics have been devel- oped for each of the main assessment types—Classroom Assessment, Examinations, and National and International Large-Scale Assessments. Country Reports Based on the information gathered through applying the SABER–Student Assessment tools, a country report is produced that summaries how well the country’s student assessment system is performing and provides recommendations for future improvement. In 2011, the SABER–Student Assessment tools were applied retroactively to all eight READ Trust Fund–supported countries to establish baselines against which to measure progress in developing their student assessment systems. Global-level Work 15 Box 2. Framework for Building an Effective Assessment System The conceptual framework for SABER–Student Assessment underpins all of the work being carried out under the READ Trust Fund. The framework is structured around two main dimensions of assessment systems: the types/purposes of assessment activities, and the quality of those activities. Assessment types and purposes àà Classroom assessment provides real-time information to support ongoing teaching and learning in individual classrooms. Classroom assessments use a variety of formats—including observation, questioning, and paper-and-pencil tests—to evaluate and support student learning, generally on a daily basis. àà Examinations provide a basis for selecting or certifying students as they move from one level of the education system to the next. Examinations cover the main subject areas in the curriculum and usually involve essays and multiple-choice questions. àà Large-scale, system-level assessments provide feedback on the overall performance of the education system at particular grades or age levels. These assessments typically cover a few subjects on a regular basis (such as every three to five years), are often sample-based, and use multiple-choice and short-answer formats. They may be national or international in scope. Quality drivers àà Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the policy framework for assessment activities, institutional structures, the availability of stable sources of funding, and the presence of trained assessment staff. àà System alignment refers to the extent to which the assessment is aligned with the rest of the educa- tion system. This includes the degree of congruence between assessment activities and system learning goals, standards, and curriculum. àà Assessment quality refers to the psychometric quality of the instruments, processes, and proce- dures for the assessment activity. It covers such issues as design and implementation of assessment activities, analysis and interpretation of student responses, and the appropriateness of how assess- ment results are reported and used. Crossing the quality drivers with the different assessment types/purposes creates the framework for the key indicator areas, such as policies, human resources, curriculum, etc., as shown in Figure 5. Figure 5. Framework and Key Indicator Areas for Student Assessment ASSESSMENT TYPES/PURPOSES Classroom Examinations Large-scale, system- assessment level assessments Enabling context Policies Leadership and public engagement Funding QUALITY DRIVERS Institutional arrangements Human resources System alignment Learning/quality goals Curriculum Pre- and in-service teacher training opportunities Assessment Ensuring quality (design, administration, analysis) quality Ensuring effectiveness (Source: Clarke, M. 2012. ”What Matters Most for Student Assessment Systems: A Framework Paper.” World Bank, Washington, DC.) 16 READ Trust Fund 2013 Annual Report Figure 6. Countries Participating in SABER–Student Assessment Activities (as of 12/31/2013)   Country Reports   Regional Reports   Case Studies IBRD 40127 JULY 2013 This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any endorsement GSDPM or acceptance of such boundaries. Map Design Unit Country Reports (35) Regional Report (13) Case Studies (12) Angola Oman Cambodia Australia Armenia Pakistan China Brazil Bahrain Papua New Guinea Hong Kong SAR, China Chile (2) Bangladesh Qatar Indonesia The Gambia Brunei Samoa Japan Germany Democratic Republic Saudi Arabia Republic of Korea Jordan3 of Congo Solomon Islands Lao DPR New Zealand Egypt Sri Lanka Malaysia Uganda3 Ethiopia Sudan Mongolia Singapore2 Ghana Syria The Philippines Russian Federation (2) Iraq Tajikistan Singapore1 Jordan1 Tunisia Thailand Kyrgyz Republic Uganda 1 Vietnam3 Lebanon United Arab Emirates Libya Vietnam2 Mauritania West Bank & Gaza 1 Also a Case Study Mozambique Yemen 2 Also a Regional Report Nepal 3 Also a Country Report Zambia (Source: World Bank, www.worldbank.org/education/saber.) Global-level Work 17 By the end of 2013, in addition to the eight READ Trust Fund–supported countries, the team had collected data and worked on full country reports for 27 additional countries. Throughout 2012 and 2013, there continued to be considerable demand from countries worldwide to use the SABER–Student Assessment tools to benchmark their assessment systems. By the end of 2013, in addition to the eight READ Trust Fund–supported countries, the team had collected data and worked on full country reports for 27 additional countries (see Figure 6 for a list of all countries). The team also had performed bench- marking exercises, using a shorter, consolidated version of the question- naires, in a group of countries in the East Asia and Middle East & North Africa regions of the World Bank. Case Studies In order to build the evidence base on best practices in reforming and strengthening assessment systems, a series of case studies were commis- sioned on what various countries around the world have done to develop their assessment systems. Each of the 12 case studies was presented at one of the READ Global Conferences, and some of them have been further developed and published under the READ Working Paper series. By the end of 2013, nine of the case studies had been published, including two on Chile, two on Russia, and one each on Brazil, the Gambia, New Zealand, Uganda, and Queensland, Australia. Reviews and Research on Key Assessment Topics A portion of the global work program is set aside for (i) reviews of the latest, high-quality research on key topics and issues in the area of student assessment, and (ii) collection and analysis of data on student assessment topics. As part of this workstream, the Education Note, “Measuring Learning— How Effective Student Assessment Systems Can Help Achieve Learning for All,” and the working paper, “Review of World Bank Support for Student Assessment Activities in Client Countries, 1998–2009,” have been published. 18 READ Trust Fund 2013 Annual Report Training course on student Training Sessions and Learning Events assessment during the World Bank’s Learning Week. In 2013, nearly 250 people participated in four different training events on the topic of student assessment and the corresponding SABER tools. These trainings were delivered primarily as 2-3 day workshops and included World Bank education staff, the United Kingdom’s Department for International Development education staff, students and education officials from Russia, and representatives from assessment agencies in the Pacific and Australia’s Department of Foreign Affairs and Trade. Since 2010, the SABER–Student Assessment team has participated in more than 30 of these events. As a result, the SABER–Student Assessment team has been able to reach more than 2,000 individuals. If one were to also include meetings at which the overall SABER program and country-level activities were discussed, such as country report valida- tion workshops and training activities, the total number of people reached would be more than 5,000. In order to continue to expand the reach of the program, work is underway to incorporate the training materials for SABER–Student Assessment into an e-learning course for education policymakers and practitioners. Global-level Work 19 SABER–Learning Standards In 2011, background work for a potential SABER domain on learning standards was initiated with READ Trust Fund support. The background work for SABER–Learning Standards was outsourced to the American Institutes for Research (AIR). Similar to the work being carried out on SABER–Student Assessment, the focus was on developing an evidence- based methodology to benchmark learning standards policies and systems in countries around the world. In 2013, a decision was made to incorporate the work on learning standards into the SABER–Student Assessment domain as part of the broader suite of tools and products available in this area. Conceptual Framework and Diagnostic Tools In 2011, the AIR team completed a report on key characteristics of effec- tive learning standards based on their review of the existing global evidence base and best practices. Subsequently, a set of questionnaires and rubrics were developed that could be used to collect and evaluate data on these aspects of a country’s learning standards. The tools allow for the collection of data on content, process, and performance standards in the areas of Language Arts, Mathematics, and Science. In 2012, the diagnostic tools were piloted, reviewed, and revised. A user’s guide was created to guide the data collection process. These products will become part of the broader suite of tools available under SABER–Student Assessment. Country Reports The piloting of the tools involved the collection of data and production of country reports for six pilot countries: Armenia, Jordan, Mozambique, Nepal, Peru, and Vietnam. The country reports summarize key findings and recommendations derived from application of the diagnostic tools. These reports will be made available as background material on the SABER website. Case Studies To build the evidence base in this topic area, and to capture lessons learned, a series of case studies that describe what various countries have done to develop and implement effective learning standards were commis- sioned. The six case study countries include Chile, India, Jordan, Namibia, New Zealand, and Singapore. These case studies will be published in 2014 as part of the READ Working Paper series. 20 READ Trust Fund 2013 Annual Report Since 2009, the READ Trust Fund has organized nearly 10 international learning events on assessment. Global Knowledge Sharing At the global level, the program works to identify opportunities to share lessons and best practices in student assessment with READ Trust Fund– supported and other developing countries. These activities have included global conferences and regional workshops, partnerships and collabora- tion with others working in student assessment, and the development and dissemination of knowledge products on assessment. These activities are meant to enhance the program’s work at the country level and increase overall awareness of best practices in assessment. READ Global Conferences and Regional Workshops Since 2009, the READ Trust Fund has organized three READ Global Conferences—in 2009, 2010, and 2011. These conferences have provided a forum for education officials from READ-supported countries, Russian officials and education experts, World Bank staff, and international assess- ment experts to share experiences, learn best practices, and highlight Russia’s role as a key donor in the area of education quality. In 2012, in lieu of a READ Global Conference, arrangements were made for READ to be a co-sponsor of the IAEA Annual Conference. In addition, the program team organized two READ Trust Fund Regional Workshops—one in the Eastern Europe and Central Asia region and one in the Africa region. By partnering with the IAEA, and organizing several panel sessions on READ Trust Fund and READ Fee-Based Services activities, the READ program was able to reach a much wider audience and connect the program with the wider global assessment community. In relation to the workshops, by having a regional focus, the READ Trust Fund-supported countries were able to focus on challenges particular to their region and build stronger connections with their regional neighbors. Based on the success of the partnership with the IAEA Conference in 2012, the READ Trust Fund followed a similar model in 2013, partnering with both the AEAA and the IAEA. Panel sessions on READ Trust Fund-supported country achievements were held at both organizations’ annual confer- ences. READ Trust Fund expo booths were also set up to disseminate global knowledge products developed under the program (see Appendix 2 for further details). Global-level Work 21 Other READ Trust Fund-sponsored Events Participants at the “Assessment for Global As the development community comes closer to finalizing the post-2015 Learning” symposium in development agenda in education, there have been serious conversations Washington, DC. about the possibility of a universal learning goal. In order to contribute to these discussions, and with the support of the READ Trust Fund, the World Bank hosted a symposium in November 2013 on the topic of “Assessment for Global Learning.” This two-day symposium brought together assess- ment experts and policy researchers to review and discuss the existing menu of tools and approaches for monitoring and supporting student learning, and to explore how best they can be used or improved to meet the needs of the post-2015 development agenda (see Appendix 2 for further details). Other Collaboration—PISA for Development and the Learning Metrics Task Force Beginning in 2009, and with the ongoing support of the READ Trust Fund, the World Bank and the Governing Board of OECD’s Program for International Student Assessment (PISA) have been working together to find ways to adapt the PISA tests so that low-income countries can more readily benefit from them. As a result of this collaboration, a new reading component skills assessment was developed and made available as an international option for the PISA 2012 survey, of which both Poland and Peru were able to avail. 22 READ Trust Fund 2013 Annual Report By the end of 2013, more than 4,500 print copies of the READ Working Papers had been disseminated. Continuing with this work, in 2013, the OECD announced a new initia- tive—PISA for Development. Through this initiative, the OECD will be pilot- ing a modified version of the current PISA, one in which certain adaptations have been made for the developing country context. Of the six countries selected to participate in the pilot—Cambodia, Ecuador, Guatemala, Senegal, Sri Lanka, and Zambia—the READ Trust Fund has been directly supporting Zambia’s international participation costs. Additionally, READ Trust Fund support has been used to develop capacity building plans for all six pilot countries. It is expected that capacity building activities will be incorporated into each country’s overall project implementation plan. Another means of collaboration has been through participation in the Learning Metrics Task Force. The UNESCO Institute for Statistics (UIS) and the Center for Universal Education at Brookings have joined efforts to convene this task force, which is investigating the feasibility of identifying global learning competencies and measures for the post-2015 develop- ment agenda. Through READ’s participation in this task force, the knowledge, tools, and experiences accumulated under the READ Trust Fund program have been able to inform these discussions. National Assessments of Educational Achievement Series The READ Trust Fund has been supporting the publication of a five-volume series on key concepts in national assessments of student achievement levels. These concepts include policy issues to address when designing and carrying out assessments, test development, sampling, data analysis, report writing, and the use of results to improve educational quality. By the end of 2013, Volumes 1, 2, 3, and 5 had all been published. (Volume 4 is forthcoming.) Public Disclosure Authorized Public Disclosure Authorized National Assessments of Educational Achievement 41789 National Assessments of Educational Achievement National Assessments of Educational Achievement National Assessments of Educational Achievement VOLUME 1 VOLUME 2 VOLUME 3 VOLUME 5 Assessing National Developing Tests and Implementing a Using the Results of a Public Disclosure Authorized Achievement Levels Questionnaires for a National Assessment National Assessment of in Education National Assessment of Educational Educational Achievement of Educational Achievement Vincent Greaney Achievement Public Disclosure Authorized Vincent Greaney Thomas Kellaghan Thomas Kellaghan Thomas Kellaghan Vincent Greaney Prue Anderson T. Scott Murray George Morgan Global-level Work 23 Knowledge Dissemination Knowledge products on display at the IAEA The READ Trust Fund has engaged in a wide range of activities to dissemi- Conference. nate the key messages of the program, highlight the role of Russia as a donor, and share the knowledge products developed. This has been achieved through various channels, such as: àà producing a video, which highlights the importance of assessment and education quality; àà creating a website dedicated to READ Trust Fund program activities (www.worldbank.org/readtf); àà disseminating stories on READ Trust Fund activities through internal/ external World Bank websites; àà establishing a working paper series on topics related to student assessment; and àà disseminating READ Trust Fund–supported publications at interna- tional events (i.e., World Bank education events, AEAA and IAEA Conferences, and the Eurasian Association for Educational Assessment (EAOKO) Conferences). By the end of 2013, more than 4,500 print copies of the READ Working Papers had been disseminated, in addition to almost 1,300 copies of volumes in the National Assessments of Educational Achievement series. To facilitate the use of global products by countries receiving READ Trust Fund support, as well as by the Russian donor, many products have been translated into Russian and Portuguese. By the end of 2013, translated materials included the “What Matters Most for Student Assessment Systems” Framework Paper, eight published working papers, and four volumes of the National Assessments of Educational Achievement series. 24 READ Trust Fund 2013 Annual Report READ Council Meeting in Paris, France. Program Coordination The READ Council, comprised of Russian authorities and World Bank education leaders, meets regularly to provide guidance on the strategic direction of the overall READ Program, including the READ Trust Fund. Additionally, the READ Trust Fund team has organized a number of partnership meetings with Russian authorities, READ Fee-Based Services, and Russia’s Center for International Cooperation in Education Development (CICED) to identify program synergies and areas for cross collaboration. Global-level Work 25 READ Council Meetings Prior to 2013, the program primarily held READ Council meetings on an annual basis. As the program headed into the final phase of implementa- tion, there was a need for increased frequency of these meetings to further discussions on next steps and the future of READ. In 2013, three READ Council Meetings took place—one in Paris, France; one via a video confer- ence between Washington, DC and Moscow; and one in Washington, DC. READ Trust Fund-related decisions made during these meetings included reallocations of country-level funds, approval of activities for the final phase of implementation, preparations for a final READ Global Conference in St. Petersburg, and agreement on future areas of work for a possible second READ Trust Fund. Collaboration across Programs—READ Trust Fund, READ Fee-Based Services, and CICED The READ Trust Fund, READ Fee-Based Services, and CICED teams communicate regularly and are continually looking for value-added ways to work together. In 2013, efforts to share information across programs resulted in several collaborative efforts. Some examples include: àà training on the SABER–Student Assessment tools at the Moscow Higher School of Economics and Russian Training Center; àà joint participation in the 39th IAEA Conference; àà the inclusion of one of CICED’s newly-developed assessment tools— School Achievements Monitoring tool (SAM) (see Box 3)—at the World Bank’s high-level symposium on Assessment for Global Learning; and àà broad dissemination of READ Trust Fund knowledge products among Russian partner organizations such as EAOKO and the Russian Training Center. Box 3. Russia’s School Achievements Monitoring Tool (SAM) The READ Program consists of two main components: the READ Trust Fund and READ Fee-Based Services. The primary objective of READ Fee-Based Services is to develop the capacity of Russians, and CICED in particular, to engage in international development activities in the area of education. Since its inception in 2008, CICED has served as a network organization to bring together Russian expertise in assessment and quality management in education. One of the key education quality assessment tools developed through CICED support is the School Achievements Monitoring tool (SAM). SAM is a testing instrument with the purpose of measuring primary school students’ proficiency in the areas of mathematics, language, and science. The assessment frame- work for SAM is based on Vygotsky’s theory and is designed to evaluate students’ competencies at three —  different levels  formal, reflexive, and functional. The entire SAM toolkit includes tests in each subject area, questionnaires for collecting background information, and recommendations on how to interpret and use the test results. (Source: CICED, www.ciced.org.) 26 READ Trust Fund 2013 Annual Report Country-level Work READ Trust Fund-supported programs are underway in eight countries—Angola, Armenia, Ethiopia, the Kyrgyz Republic, Mozambique, Tajikistan, Vietnam, and Zambia. World Bank Task Team Leaders and Country Teams are working closely with their respective government counterparts to implement a set of agreed-upon activities to further develop and strengthen the existing assessment systems in these countries. Creating a Results Framework Establishing a sound assessment system is not easily accomplished within a few years. Even the best-performing countries have taken decades to firmly establish their systems and effectively use results in ways that improve overall education quality. However, the SABER–Student Assessment tools make it possible to break down various aspects of this task into manageable steps by identifying key indicators, defining levels of development on each indicator, and describing what it takes to reach the next level. In order for the READ Trust Fund to be able to demonstrate measurable results, a results framework and key indicators were established. This framework draws heavily on the benchmarking tools created under SABER–Student Assessment and summarizes key indicators common across all four assessment types (see Table 2). All eight READ countries have been able to develop parts of their assessment systems in significant ways and lay a strong foundation for future work. Country-level Work 27 Table 2. READ Trust Fund Results Framework and Key Indicators Enabling Context (EC) EC1—Setting clear policies There is a formal document that provides guidelines about assessment activities. The formal document is available to key stakeholders and the public. EC2—Having strong leadership Key stakeholders support the assessment activities. There is key stakeholder support for continuous improvement of assessment activities. EC3—Having regular budget/funds for assessment activities There is a line item in the government education budget for assessment activities. The budget provides adequate funding in major areas, including design, administration, data processing, and reporting. EC4—Having strong organizational structures There is an agency, institution, or unit with the mandate to carry out assessment activities. The assessment agency, institution, or unit is accountable to a clearly recognizable body. EC5—Having effective human resources There is a team of people with the requisite skills/capacity to carry out assessment activities. There are opportunities available to build assessment capacity—for example, courses/training on test development, sampling, and so forth. System Alignment (SA) SA1—Aligning the assessment with learning goals There is a clear, common understanding among key stakeholders of what the assessment activities measure. Assessment activities are aligned with an official curriculum/learning standards that outlines what students are expected to learn. SA2—Providing opportunities to learn about assessment activities There are training sessions/courses for teachers to learn about the assessment activities. Teachers are involved in some aspect(s) of assessment-related activities. Assessment Quality (AQ) AQ1—Ensuring quality There is formal documentation about the technical aspects of the assessment activity. Assessment results are deemed valid and reliable by key stakeholders. AQ2—Ensuring effective use of assessment results Assessment results are disseminated in meaningful ways to key stakeholders. Assessment results are used to promote and inform students’ learning. 28 READ Trust Fund 2013 Annual Report Monitoring and Tracking Progress The SABER–Student Assessment tools were applied in all eight READ Trust Fund-supported countries to establish baselines against which to measure progress. During the first phases of project implementation, the team conducted a forecasting exercise to create a clear idea of the results one could expect by the end of the READ Trust Fund. Now that the program is in the final stages of implementation, the results reported are based on actual progress achieved. Upon completion of each country program, the same tools used to create the baseline will be applied to determine the overall development and impact in each assessment area. Many of the countries began with very latent assessment systems or systems that were just emerging. With READ Trust Fund support, all of these countries will be able to develop parts of their assessment systems in significant ways and lay a strong foundation for future work. The following section provides summaries of the cumulative progress achieved by the end of 2013 for each country. The table for each country shows their baseline results and progress to date for each assessment type. Figure 7 provides a description of the key elements included in these country tables. Figure 7. Key to Interpreting Country Tables Latent Emerging Established Advanced Classroom Assessment Examinations National Large-Scale Assessment (NLSA) International Large-Scale Assessment (ILSA) The black vertical line for The yellow arrow for each each assessment type assessment type represents represents the baseline level the activities that have been of development established at implemented to date and the beginning of the program. the corresponding amount No arrow means that this of progress according to the assessment type is not a key indicators outlined in the current area of activity in the benchmarking tool. country’s action plan. Country-level Work 29 Angola Total Grant Amount Disbursements as of 12/31/13 US$2,180,054 US$1,466,568 COUNTRY OVERVIEW When Angola ended its 25-year civil war in 2002, the country needed to rebuild its education system from the ground up. Since then, Angola’s primary education system has improved dramatically, but much work remains to establish a strong overall system that ensures quality learning outcomes. Similar to many other countries, although many more children now have access to school in Angola, these students are not necessarily acquiring the expected basic literacy and numeracy skills. There is strong commitment from Angola’s highest levels of government to improve the capacity of the education system to monitor quality and measure student learning outcomes. At the beginning of the READ Trust Fund, Angola’s education system had a weak assessment culture, no full-time staff trained in assessment, and very few dedicated resources for assessment activities. However, as a result of strong support from the Ministry of Education and READ funds, the country was able to complete its first ever national-level assessment of early grade reading and establish a foundation for future work in this area. In fact, the success under READ led to a US$75 million project focused entirely on improving education quality in Angola. Box 4. Angola’s Campaign to Improve Based on the results of EGRA, Angola recognized Literacy Outcomes the need to highlight the importance of reading, and started a national campaign to improve liter- The READ Trust Fund has played a critical role in acy skills throughout the country. This campaign Angola. With READ support, an Early Grade has, among other things, supported the creation Reading Assessment (EGRA) was carried out in of teacher manuals that include strategies and 2011  —  the first nationwide assessment to take activities for teaching literacy skills, as well as place in the country. The data collected during flyers targeted at parents and caregivers that implementation of EGRA have allowed the country emphasize the importance of reading. to identify areas where improvements are needed, particularly in the area of literacy. 30 READ Trust Fund 2013 Annual Report Table 3. Angola’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Note: ILSA is not an area of funded activity under the current READ Trust Fund program in Angola. Progress by Key Indicators The objective of the READ Trust Fund program in Angola has been 1) to improve the capacity of Angola’s Ministry of Education to assess student learning outcomes and 2) to develop a culture of evidenced-based decision making. At the beginning of the program, Angola’s assessment system was at a very latent stage of development. With READ Trust Fund support, Angola chose to focus first on NLSA. Based on their success in this area, Angola expanded this work to also include activities in Classroom Assessment and Examinations. It is expected that, by the end of READ, Angola will have made significant progress in laying a strong foundation for future NLSA, Examinations, and Classroom Assessment activities. A key focus of READ Trust Fund support has been to build a new Assessment Unit within the Ministry of Education capable of carrying out NLSA activities. After successfully implementing EGRA, the team has worked to use the results in the design of various interventions to improve learning outcomes in reading. Table 3 indicates overall progress to date based on achievements in the key indicator areas being tracked under the READ Trust Fund. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. Country-level Work: Angola 31 Clear Policies A teacher administrating EGRA in rural Angola. Due to the success of EGRA, a decision has been made to implement regular EGRA and NLSA exercises. Additionally, greater attention is being given to a policy mandating National Examinations in grade 6, which was never implemented due to logistical issues. The development of a five-year strategy for work in each of these areas is underway. Strong Leadership High-level education officials, including Vice Ministers of Education and the National Director, have consistently provided support for the activities being implemented with READ Trust Fund support. The Vice Minister responsible for overseeing assessment activities regularly meets with the Assessment Group and has led the Angolan delegations for several inter- national events, including a study visit to Brazil and the READ Global Conferences. Regular Budget In 2010, a line item for assessment activities was created in the education budget. Since then, a regular annual budget of around US$1 million has been allocated to support the development of Angola’s assessment system. Strong Organizational Structures In 2010, the Ministry of Education created a new Assessment Unit with a dedicated team of 14 staff. The Assessment Unit has been allocated space within the Ministry, and government funds have been used to provide them with the necessary equipment. 32 READ Trust Fund 2013 Annual Report Effective Human Resources Initially, capacity-building activities focused on training Assessment Unit staff in the skills necessary to implement EGRA. More recently, these activities have been expanded to include training in the areas of National Examinations and Classroom Assessment. In total, more than 200 key education stakeholders now have increased capacity to understand and implement assessment activities. àà Study Visit: In 2011, 21 Ministry of Education staff traveled to Brazil to learn from the country’s assessment practices and quality improve- ment efforts. In order to share these lessons back home, a three-day conference, which included more than 150 participants from the Ministry of Education, provincial governments, and teacher training institutes, was held on the group’s return. àà Conferences: Representatives from Angola’s Ministry of Education have a better understanding of key issues and best practices in assessment as a result of participating in international learning events such as the annual READ Global Conferences and the 2012 READ Regional Workshop. Also, in an effort to connect with the broader assessment community in Africa, representatives from the Assessment Unit participated in their first AEAA Conference in 2013. àà Training for EGRA: In 2010 and 2011, the Assessment Unit staff partic- ipated in six different workshops to assist them with the implementa- tion of EGRA. These workshops covered sampling, data entry, data cleaning and analysis, the EGRA instruments, and data collection and software. 75 teachers and teacher trainees also were trained in how to administer the assessment. àà Training on National Examinations: In 2013, a nine-day training on National Examinations was conducted for 14 Assessment Unit staff and 20 provincial education staff responsible for overseeing provincial- level examinations. àà Training on Classroom Assessment: 14 Assessment Unit staff and 2 representatives from all 18 of Angola’s teacher training institutions were brought together for a five-day training on classroom assess- ment and how it can be used to improve learning outcomes. During the training, preparations were made for the development of pre- and in-service teacher training modules on classroom assessment. Due to the success of EGRA, a decision has been made to implement regular EGRA and NLSA exercises in Angola. Country-level Work: Angola 33 Assessment Quality Left: Training on EGRA for education officials in Angola. As part of their capacity-building activities, the Assessment Unit team was responsible for implementing EGRA on a national scale. In 2011, the team Right: Angolan delegation at the 2013 AEAA Conference in adapted the tools, created the sample, recruited and trained teacher train- Tanzania. ees to implement the test, and oversaw the administration of EGRA in 144 schools throughout Angola’s 18 provinces. In 2012, the team worked on data cleaning, analyzed the data, and prepared a report with the final results. This report establishes the first baseline data on learning outcomes in Angola. A workshop was held with 100 provincial education staff members to share and discuss the preliminary results and identify actions for moving forward. Planned Activities In 2014, READ Trust Fund-supported activities in Angola will include: àà publication and dissemination of materials aimed at supporting literacy teaching and learning activities, based on the results from EGRA; àà continued development of pre- and in-service teacher training modules on Classroom Assessment for grades 4 and 6; àà additional training for Ministry of Education staff on classroom assess- ment; and àà additional training for Ministry of Education staff on how to design and implement National Examinations. 34 READ Trust Fund 2013 Annual Report Armenia Total Grant Amount Disbursements as of 12/31/13 US$1,400,000 US$669,041 COUNTRY OVERVIEW Education has been a priority sector for public investment in Armenia since the late 1990s. The government has implemented a number of reforms to increase access and improve quality in the education system. However, despite significant improvements, the country’s education system still faces challenges in increasing overall learning levels and in the size of equity gaps between rich and poor students. Armenia is committed to ensuring an efficient and effective allocation of resources in education. As a part of this commitment, serious effort has been given to creating new policies that govern Armenia’s system of measuring and monitoring learning outcomes. However, to effectively implement these policies, greater capacity is needed within different stakeholder groups so that they can carry out assessment activities in an appropriate manner and link the results to education policies, practices, and decision making. Box 5. Improving Implementation of Through the support of the READ Trust Fund, an Assessment Policies and Practices in Armenia expansive training program is underway in Armenia to equip various stakeholders with Laying a strong foundation for assessment activi- greater capacity in measuring learning outcomes. ties with policy frameworks, clear institutional Through a training program on Classroom arrangements, and regular funding is absolutely Assessment and Examinations, pre- and in-service essential, but it is only a first step. An effective teachers are acquiring the skills necessary to assessment system also requires a strong cadre carry out formative and summative assessments of people across the entire education system with in their classrooms. For education officials tasked the requisite skills to implement assessment with implementing NLSAs, an in-depth training activities and use the results to inform their educa- program in specialized skill areas, such as statis- tion practices. tics and item analysis, has been developed. Country-level Work: Armenia 35 Table 4. Armenia’s Assessment System: Baseline Results (2011) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Progress by Key Indicators Armenia was invited to become a READ Trust Fund–recipient country in 2011. The 2011 baseline results showed that while many of the necessary policies were in place for the various assessment types, there was a need to focus on capacity building and improving implementation of assess- ment practices.. With READ Trust Fund support, Armenia has chosen to primarily concentrate on building capacity across all four assessment types—Classroom Assessment, Examinations, NLSA, and ILSA. It is expected that, by the end of the READ program, Armenia will have made significant progress in the area of NLSA and improvements in Classroom Assessment, Examinations, and ILSA as well. Armenia has been actively implementing a number of capacity-building programs, designing and piloting assessment instruments, and analyzing data from past ILSA activities. Table 4 indicates overall progress to date based on achievements in the key indicator areas being tracked by the READ Trust Fund program. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. 36 READ Trust Fund 2013 Annual Report Effective Human Resources In Armenia, the READ Trust Fund program is supporting a number of capacity-building activities. Through participation in international confer- ences on assessment topics and specialized training courses, more than 100 key education stakeholders have increased their capacity to carry out assessment activities. àà Conferences: Delegations from Armenia’s Ministry of Education and other key agencies have participated in a number of international events on assessment, including the 2011 READ Global Conference, the 2012 READ Regional Workshop, the 2012 and 2013 IAEA Conferences, and the 2012 and 2013 EAOKO Conferences. Taking part in these events has enhanced Armenia’s understanding of key issues in assessment and further developed the country’s capacity to carry out assessment activities. àà Training for Teachers: READ funds have been supporting the develop- ment of training courses on Classroom Assessment and Examinations for Armenia’s pre- and in-service teachers. In 2013, two workshops designed for pre-service teachers were piloted with 45 university lecturers and another workshop designed for in-service teachers was piloted with 25 Armenian teachers. Work is underway to finalize the training modules so that they can be institutionalized and used by universities. àà Training for Education Policymakers: In the area of NLSA, an intensive training program for Assessment Testing Center (ATC) staff, university academics, and graduate-level students has been developed. In 2013, 25 individuals participated in four workshops that covered the follow- ing topics — General Statistics, Item Analysis, Test Questionnaire and Development, and National Assessment for Educational Policymakers. This training program will continue in 2014. Additionally, READ supported one ATC staff member to complete a Masters program in Educational Evaluation and Assessment at the Moscow School of Social and Economic Sciences. àà Masters Course: To further institutionalize capacity building in assess- ment in Armenia, work is underway to develop a Masters program in education that incorporates courses on student assessment. In 2013, a report on international best practices, program specifications, and content for such a Masters program was prepared and discussed with universities in Armenia. Alignment with Learning Goals The READ Trust Fund is supporting the application of the CICED-sponsored Information and Communications Technology (ICT) Literacy Test to grade 9 students in Armenia as a way to assess their competencies in using modern information and computer technologies. In 2013, after a pilot of the test, the instrument was adapted and finalized for use during the main survey. Opportunities to Learn about Assessment Activities With READ Trust Fund support, Armenia’s participation in the Trends in International Mathematics and Science Study (TIMSS) 2011 was finalized Country-level Work: Armenia 37 and preparations are underway for Armenia’s participation in TIMSS 2015. Left: Armenian teachers To effectively disseminate the results of TIMSS 2011, discussions took during the training course on classroom assessment. place with the Ministry of Education and ATC staff and an article targeted at key stakeholders was published. Additionally, a National Research Right: Participants during one of the NLSA workshops. Coordinator attended two planning meetings for TIMSS 2015. Assessment Quality Armenia plans to implement a NLSA in grade 8 as a way to assess students’ progress in the areas of Armenian Language, Literature, and History. READ has supported the design of the questionnaire and testing instrument for the pre-pilot, which will take place in 2014. Quality Assurance A feasibility study and training have been conducted to assist Armenia in deciding whether or not to introduce computer-adaptive testing and computer-based testing for the school leaving and university entrance examinations. When the study has been finalized, a decision will be made on what actions to take to continue to improve the current system. Planned Activities In 2014, the main READ Trust Fund-supported activities in Armenia will include: àà finalization of the course and workshops on Classroom Assessment and Examinations for pre- and in-service teachers àà completion of the training program on NLSA for ATC staff, policy makers, and academics àà development of a Masters program in education that includes courses on student assessment àà full-scale implementation of the CICED-sponsored ICT Literacy Test àà pre-pilot implementation of the NLSA in grade 8 and àà training on the analysis of TIMSS 2011 results and preparations for TIMSS 2015. 38 READ Trust Fund 2013 Annual Report Ethiopia Total Grant Amount Disbursements as of 12/31/13 US$1,089,705 US$552,899 COUNTRY OVERVIEW Ethiopia is the second most populous country in Africa, with 80 percent of its population living in rural areas. Although the education system in Ethiopia has undergone impressive expansion, it still faces serious challenges, particularly in the area of quality. The Government of Ethiopia sees education as a means to eradicate poverty, but realizes that this will only occur if the education system delivers a quality educational experience that promotes learning. A key component of work in Ethiopia’s education sector has been the development of a strong system for continuously monitoring education quality and learning outcomes. Targeted efforts have been made to strengthen existing institutions responsible for measuring learning outcomes and to establish new ones where necessary. Some of the main challenges for Ethiopia to overcome in this area have been the lack of policies, clear lines of responsibility, and regular funding. Box 6. Inspecting Schools and Monitoring Ministry of Education. A policy framework, guide- Education Quality in Ethiopia lines, and school classification standards have been created to govern work in this area. To date, Five years ago, activities to monitor education more than 60 Ministry of Education officials at the quality in Ethiopia were mostly undertaken on an central and regional levels have received training ad hoc basis and were not institutionalized in the in how to implement and use the school inspec- education system. Demonstrated by poor learning tion framework and guidelines. A pilot of the new outcomes, education leaders recognized a serious school inspection activities has taken place and need to develop a system for inspecting schools nearly 28,000 schools have completed a self- and monitoring education quality. assessment exercise that measures school quality Through the support of the READ Trust Fund, a against the standards in the school inspection new directorate dedicated entirely to school framework. inspection has been established within Ethiopia’s Country-level Work: Ethiopia 39 Table 5. Ethiopia’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Note: Classroom Assessment and ILSA are not areas of funded activity under the current READ Trust Fund program in Ethiopia. Progress by Key Indicators The 2009 baseline results indicated that for Ethiopia, both Classroom Assessment and ILSA activities were at a latent stage of development, NLSA was emerging, and Examinations were nearly established. Coordinating with other donors working in education, it was determined that READ Trust Fund–supported activities would focus primarily on further strengthening the areas of NLSA, Examinations, and School Inspection. (School Inspection contributes directly to education quality, but is not one of the areas being tracked under the READ Results Framework.) It is expected that by the end of READ, Ethiopia will have made significant progress in laying a strong foundation for NLSA and School Inspection activities and will have improved its system for Examinations. In Ethiopia, the key focus of READ Trust Fund support has been to develop formal policy frameworks, establish clear institutional arrangements, and build greater capacity to carry out assessment activities. Table 5 indicates overall progress to date based on achievements in these and other key indicator areas being tracked by the READ Trust Fund. A detailed break- down of this progress, by indicator area, is provided in the following narrative. 40 READ Trust Fund 2013 Annual Report Left: Ethiopian delegation at Clear Policies the 2013 AEAA Conference in Tanzania. After in-depth reviews were completed in each of the three target areas— NLSA, Examinations, and School Inspection—work commenced to estab- Right: Ministry of Education officials at the workshop on lish clear policies for guiding the work in each area moving forward. By the role of assessment in 2013, a policy and procedural framework for NLSA, a policy framework for education in Ethiopia. National Examinations, and a framework and guidelines for national School Inspection had all been created. Strong Leadership A number of workshops have taken place in Ethiopia to discuss and endorse the draft policy and procedural frameworks for each target area. More than 150 education leaders in Ethiopia have participated in these workshops, including the Education Minster, key Ministry of Education officials at the central and regional levels, School Inspection staff, and donor representatives. Regular Budget In the past, funding for NLSA activities was somewhat irregular. In 2012, a dedicated line item was created for NLSA activities and this area now receives a regular annual budget. Strong Organizational Structures In order to establish clear institutional arrangements for assessment activi- ties, a new agency and directorate were created in 2012. These new insti- tutional arrangements, which have greatly improved the enabling context for carrying out assessment activities in Ethiopia, include the following: àà National Educational Assessment and Examinations Agency (NEAEA), which is autonomous, led by a board, and responsible for both NLSA and National Examinations. àà Directorate for School Inspection, within the Ministry of Education, which has 10 staff members fully dedicated to School Inspection activities. Country-level Work: Ethiopia 41 Effective Human Resources Through participating in international conferences and workshops, study visits, and specialized trainings, more than 170 education leaders and practitioners in Ethiopia are now more aware of key issues in assessment and able to implement assessment-related activities. àà Conferences: Through READ, more than 35 staff members working on NLSA, National Examinations, and School Inspection have partici- pated in international events on assessment topics, such as the annual READ Global Conferences, the 2012 READ Regional Workshop, and the 2013 AEAA Conference. To share lessons learned from the AEAA Conference and to discuss the possibility of Ethiopia becoming a AEAA member, a follow-up workshop, which included more than 50 of Ethiopia’s education leaders, was held in late 2013. àà Training for NEAEA staff: In 2012, two training-related activities were organized for NEAEA staff — a study visit to South Africa for 5 staff to learn about that country’s assessment practices and a month-long, in-depth training for 60 technical staff in areas such as item writing, sampling, test administration, data analysis, and reporting assessment results. NEAEA’s increased capacity in these areas has improved the overall quality of existing assessment activities. àà Training for School Inspection staff: Before the Directorate for School Inspection was established, three Ministry of Education staff went on a study visit to Ghana to learn how another country conducts its school inspections and to incorporate those lessons into the design of Ethiopia’s new directorate. In 2012 and 2013, after the Directorate for School Inspections was established, more than 60 Ministry of Education staff at the central and regional levels received training in how to implement and use the newly-created school inspection frame- work and guidelines. To date, nearly 28,000 primary and secondary schools have participated in school inspection activities. Assessment Quality In 2013, work began to determine what is needed to develop sustainable item banks for both the NLSA and National Examinations. Having item banks in place would greatly enhance future assessment activities in these areas through the provision of a secure source of pre-screened test items with known technical characteristics. This work will continue in 2014. Planned Activities In 2014, READ Trust Fund-supported activities in Ethiopia will focus on: àà finalizing the policy frameworks and guidelines that were developed for NLSA, Examinations, and School Inspection; àà providing additional training for NLSA staff on sampling, data analysis, and reporting results; and àà completing the study on developing item banks for NLSA and National Examinations. 42 READ Trust Fund 2013 Annual Report Kyrgyz Republic Total Grant Amount Disbursements as of 12/31/13 US$2,345,729 US$739,659 COUNTRY OVERVIEW The quality of education in the Kyrgyz Republic was dramatically impacted by the collapse of the Soviet Union. To reverse the decline in education quality, the government and international donors began working together to re-energize the Kyrgyz education system. A number of reforms were implemented by the Ministry of Education in an effort to modernize the system and improve quality. Given all of these targeted efforts to improve education quality, the country was shocked when it received the results from the 2006 and 2009 PISA exercises and discovered that, in each instance, the Kyrgyz Republic ranked last out of all participating countries. The government realized that, in addition to collecting system-level data on learning outcomes, there was a serious need to actively monitor the education process. Through the support of the READ Trust Fund, the Kyrgyz Republic has been placing an even greater emphasis on conducting regular, high-quality formative assessments in the classroom (see Box 7). Box 7. Using Formative Assessments to Through the READ Trust Fund program, the Improve Learning Outcomes in the Kyrgyz Kyrgyz Academy of Education has developed a Republic training program on classroom assessment for in-service teachers. The purpose of this training is Since receiving the PISA 2006 and 2009 results, to enhance teachers’ ability to incorporate forma- education leaders in the Kyrgyz Republic have tive assessment into their teaching practices. By been struggling to find answers to why student so doing, it is hoped that teachers will be able to performance is so low and what needs to be more easily provide students with timely feedback done to improve education quality. While there on their performance and to intervene if students are data available on system-level performance, fail to learn the material. By early-2014, it is education leaders recognize the need for more expected that more than 5,500 teachers through- regular and systematic monitoring of learning in out the country will have received this training. the classroom. Country-level Work: Kyrgyz Republic 43 Table 6. Kyrgyz Republic’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Progress by Key Indicators The READ Trust Fund program in the Kyrgyz Republic is strengthening the capacity of institutions responsible for measuring student learning outcomes and improving the use of information from those assessments to improve teaching and learning. The 2009 baseline diagnostic showed that Examinations, NLSA, and ILSA were all emerging, while the weakest area was Classroom Assessment. With READ Trust Fund support, the Kyrgyz Republic has chosen to focus primarily on Classroom Assessment while also making concerted efforts to continue to strengthen the other assessment types as well. It is expected that by the end of READ, the Kyrgyz Republic will have made significant progress in developing Classroom Assessment practices and minor improvements in Examinations, NLSA, and ILSA. Implementation of the READ Trust Fund program was seriously delayed due to the April 7 revolution in 2010. However, in April 2011, the program began to get back on track and steps were taken to complete the activities started earlier. Table 6 indicates overall progress to date according to the key indicator areas being tracked under the READ Trust Fund. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. 44 READ Trust Fund 2013 Annual Report Clear Policies In 2011, more than 40 representatives from all major education stakeholder groups participated in two workshops. Through these workshops, the overall vision and primary objectives for further strengthening the Kyrgyz Republic’s student assessment system were agreed upon. Strong Organizational Structures In 2009, a detailed analysis of the strengths and weaknesses of the National Testing Center and the Kyrgyz Academy of Education’s Assessment Unit was conducted. The final report included recommendations for further strengthening and alignment. Effective Human Resources For the Kyrgyz Republic, capacity-building activities have included atten- dance at international conferences on assessment topics, study visits, and specific training courses. As a result, more than 200 education officials, assessment practitioners, and teachers have increased capacity to carry out assessment related activities. àà Conferences and Study Visits: Through READ Trust Fund-support, representatives from the Ministry of Education and key assessment institutions have participated in a series of learning events aimed at deepening their understanding of key issues in assessment and the importance of monitoring education quality. These events have included the annual READ Global Conferences, the 2012 READ Regional Workshop, the 2012 and 2013 IAEA Conferences, the 2012 and 2013 EAOKO Conferences, and a study visit to the Netherlands. àà Training for Teachers: A key focus of READ Trust Fund-supported activities in the Kyrgyz Republic is to improve in-service teachers’ capacity to carry out classroom-based assessments. In 2013, the Kyrgyz Academy of Education developed a training program on class- room assessment for lead teachers and primary teachers. The first two rounds of the training for lead teachers have been delivered to 53 lead teachers from all seven provinces. In 2014, these lead teachers will conduct the training for primary teachers. àà Training for National Testing Center (NTC) Staff: In order to improve the school leaving Examination for grade 11, international and national experts were hired to train 46 of NTC’s test developers and curriculum specialists in item development and to do a quality review of the test items created. As a result, an improved school leaving Examination is ready to be piloted in 2014. àà Training for Education Officials: From 2009–2012, seven assessment specialists the Kyrgyz Academy of Education, regional teacher training institutions, and the Ministry of Education participated in a series of trainings offered by the Russian Academy of Education’s Training Center in Moscow. Through these trainings, participants gained a better understanding of how to design an effective assessment system at both the national and subnational levels. Country-level Work: Kyrgyz Republic 45 Assessment Quality Through the support of the READ Trust Fund, the Center for Education Assessment and Teaching Methods is preparing to implement another NLSA of Reading, Mathematics, and Science in grade 4. In 2013, updated versions of the testing instrument and questionnaires were prepared and validated through a field trial. Based on the results, the main survey instru- ments have been finalized. Effective Use of Results In 2011, an in-depth secondary analysis of the Kyrgyz Republic’s results from PISA 2006 and 2009 and the country’s 2007 and 2009 NLSA exercises was conducted. This study aimed to identify the main factors underpinning the scores in Reading, Mathematics, and Science and to provide recom- mendations for policy action. Planned Activities In 2014, READ Trust Fund-supported activities in the Kyrgyz Republic will focus on: àà building capacity and training more than 5,500 teachers in summative and formative assessment practices; àà administering the NLSA in grade 4; àà piloting the improved school leaving Examination at grade 11; and àà developing a new strategy document to guide the development of the Kyrgyz Republic’s student assessment system. Left: A teacher assessing student learning in the Kyrgyz Republic. Right: Teachers participating in the in-service training course on classroom assessment. 46 READ Trust Fund 2013 Annual Report Mozambique Total Grant Amount Disbursements as of 12/31/13 US$3,025,000 US$2,394,703 COUNTRY OVERVIEW Since 2000, Mozambique’s education system has undergone rapid expansion and the number of students participating in primary education has more than doubled. The education system has struggled to keep up with this demand, resulting in shortages of teachers and other resources that have negatively impacted education quality. The government’s vision for its education system includes both access and equity goals and efforts have been made to establish a system to monitor learning outcomes. Prior to 2009, although several assessment activities had been taking place in Mozambique, they failed to interconnect as a cohesive system. In addition, little evidence existed that assessment results were being used to improve education quality. Through the support of the READ Trust Fund, the technical capacity of those designing and implementing assessment activities has been greatly enhanced and a new classroom-based assessment has been developed. Box 8. Assessment for Learning in Since the first pilot in 2011, Provinha has been Mozambique — Provinha administered five times — once at the beginning of each school year and again at the end in order to During a study visit to Brazil, staff from track student progress. Teachers administer and Mozambique’s Ministry of Education learned score the assessment and are actively involved about an innovative classroom assessment throughout the process. In 2013, 188 schools program that was being used to monitor early participated in Provinha. An impact evaluation is grade reading skills. They decided to create underway to determine the utility of Provinha as a something similar for Mozambique. Applying the tool to help teachers become more aware of knowledge they acquired through READ Trust student learning levels, adjust their teaching Fund-supported courses on assessment topics, practices, and improve their students’ learning staff from the National Institute for Education outcomes. (INDE) designed and piloted an early grade reading assessment for grade 3 students called Provinha. Country-level Work: Mozambique 47 Table 7. Mozambique’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Note: ILSA is not an area of activity under the current READ Trust Fund program in Mozambique. Progress by Key Indicators The main goal of the READ Trust Fund program in Mozambique is to improve the Ministry of Education’s technical and institutional capacity to assess student learning outcomes. This includes improving the quality of assessment work, the alignment of assessment-related activities with national learning goals, and helping to create a culture where assessment results are valued by education stakeholders at all levels. The 2009 baseline results showed that Classroom Assessment, Examinations, and ILSA were all emerging, while NLSA was the least-developed area. It is expected that by the end of the program, Mozambique will have made the most progress in Classroom Assessment while significantly improving Examinations and NLSA as well. In Mozambique, READ Trust Fund–supported activities have primarily focused on building capacity within the Ministry of Education for work in all assessment areas and developing a new classroom-based reading assess- ment in grade 3. Table 7 indicates overall progress to date according to the key indicators being tracked under the READ Trust Fund. A detailed break- down of this progress, by indicator area, is provided in the following narrative. Strong Leadership/Strong Organizational Structures In 2009 and 2010, two national workshops, which included 50 key officials from the Ministry of Education as well as international donors, highlighted the need to establish a national assessment system with clear linkages to the overall work of the Ministry. As a result, a decision was made to include a specific line item for assessment in the annual education budget and to reorganize INDE to have a separate unit dedicated to assessment. 48 READ Trust Fund 2013 Annual Report Effective Human Resources To date, capacity-building activities for Mozambique have included short courses on assessment topics for education leaders, a Master’s degree in Educational Management and Evaluation for Ministry of Education staff, and a study visit and participation in international events on assessment. àà Short Courses: In 2011 and 2012, a team of 10–15 staff from INDE, the Examinations Council of Mozambique (CNECE), and key universities participated in three short courses on assessment. The first two courses took place in Brazil and covered the topics of test design, item analysis, statistical analysis, and proficiency scale development. The third course took place in Maputo and covered the topics of data analysis, report writing, and dissemination strategies. As result of their participation in these courses, team members were able to apply their newly acquired knowledge to the development of a new classroom- based assessment. àà Master’s Program in Educational Management and Evaluation: With READ Trust Fund support, eight staff from the Ministry of Education (six from INDE and two from CNECE) and one university staff person were able to complete a two-year Masters program in Educational Management and Evaluation through a university in Brazil. The program included both distance and face-to-face learning. These staff are now qualified in the design and management of national assessments. àà Conferences and Study Visit: Nearly 40 government officials have participated in international events on assessment topics with READ support. These events include the annual READ Global Conferences, the 2011 and 2013 AEAA Conferences, an Impact Evaluation Workshop, and the 2012 READ Regional Workshop. Additionally, in 2010, 12 specialists from INDE and CNECE participated in a study visit to Brazil to learn about the country’s assessment agencies. These events have enhanced understanding of key issues in assessment and the use of assessment results to improve education quality. Provinha being administered to class of grade 3 students in northern Mozambique. Country-level Work: Mozambique 49 Alignment with Learning Goals In 2011 and 2012, the government created and field tested 70 education quality indicators, which are in the process of being finalized and incorpo- rated into documents that summarize their uses for a range of users. Additionally, in 2011, a blueprint for writing test items for different topics in the national curriculum was created. A bank of more than 600 test items is now available to draw on for various assessments. Assessment Quality Building on the expertise developed through various training activities, INDE has led the design and implementation of a grade 3 reading assess- ment called Provinha. The materials created for Provinha include a test kit with administration and interpretation manuals; a test booklet; a scoring guide; communication materials for parents, teachers, and school leaders; and results posters. After an initial pilot phase in two Maputo districts during 2011 and 2012, the program was scaled up to include an additional two districts in northern Mozambique. In 2013, 188 schools in these four districts participated in Provinha. The assessment is administered and scored by teachers and the results are intended to be used at the school level to improve teaching practices and student learning outcomes in reading. Follow-up workshops are organized with pedagogical directors to discuss the results and options for adjusting teaching practices accordingly. An impact evaluation has been set up to ascertain whether Provinha has a significant impact on student achievement. The results of this study will be available in mid-2014 and a decision made at that time about whether or not to further scale up the assessment. Planned Activities In 2014, READ Trust Fund–supported activities in Mozambique will be concentrated on continuing to scale up Provinha by implementing it in 207 schools across five districts and finalizing the impact evaluation of Provinha and disseminating the findings. In 2013, 188 schools in four districts in Mozambique participated in an early grade reading assessment. 50 READ Trust Fund 2013 Annual Report Tajikistan Total Grant Amount Disbursements as of 12/31/13 US$4,095,089 US$2,390,367 COUNTRY OVERVIEW The collapse of the Soviet Union and the ensuing civil war had a serious impact on education quality in Tajikistan. Since the signing of a peace accord in the late 1990s, the Government of Tajikistan has aimed to rebuild its education system and ensure the effective delivery of, and equal access to, a high- quality education. Part of this mission has been to establish a National Testing Center (NTC) tasked with the responsibility of measuring student learning outcomes. The main objective of READ Trust Fund support in Tajikistan has been to increase the capacity of the new NTC to implement assessment activities and introduce a new Unified University Entrance Examination (UEE). Although UEEs existed previously, the system was deemed by most stakeholders to be flawed and the test results unreliable. The introduction of the new UEE is expected to standardize testing practices and procedures, reduce corruption, and give students more equitable access to higher education. Box 9. Ensuring More Equitable Access to skills and continuing to build their capacity; Higher Education in Tajikistan constructing a secure building, equipped with an adequate Information Technology (IT) system; and The Government of Tajikistan is firmly committed ensuring that UEE test items are well designed to reforming the UEE required for university and aligned with university requirements. admission. Over the past five years, a new model for the UEE has been carefully designed and After a successful pilot, the President of Tajikistan piloted. Learning from other countries’ experi- formally adopted the new system in late 2013. ences and international best practices, the NTC 2014 will be the first year in which the new UEE has followed a detailed, step-by-step process to will be implemented on a national scale. It is introduce the new system. Careful attention has hoped that, as a result, qualified students will have been given to such issues as creating the neces- a more equal chance at obtaining admission to sary legal frameworks; hiring staff with the right universities. Country-level Work: Tajikistan 51 Table 8. Tajikistan’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations (UEE) NLSA ILSA Note: Classroom Assessment, NLSA, and ILSA are not areas of activity under the current READ Trust Fund program in Tajikistan. Progress by Key Indicators The 2009 baseline results showed that Tajikistan’s assessment system was at a latent stage of development. Given the strong interest of the government in the area of UEE, a separate diagnostic was conducted to evaluate the current Examinations system in this area. The diagnostic found the existing system for UEE to be very weak. Hence, a decision was made to support the establishment of a NTC in Tajikistan that would be responsible for developing and introducing a new system for UEE and, eventually, for carrying out NLSA and ILSA activities as well. Tajikistan’s new NTC is being supported through various funding streams, including the government’s own budget, the Open Society Institute, a World Bank education project, and the READ Trust Fund. READ Trust Fund support has been specifically targeted to activities that involve devel- oping, piloting, and introducing the new UEE system; building the capacity of NTC staff; and providing the necessary equipment. Table 8 indicates overall progress to date based on achievements in the key indicator areas being tracked under the READ Trust Fund. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. 52 READ Trust Fund 2013 Annual Report Clear Policies At the beginning of the READ Trust Fund program, a NTC Master Plan of Action and UEE Operations Manual were drafted to ensure the appropriate regulatory and legal frameworks were in place. The NTC charter was revised and adopted by the government. In 2011, the NTC Master Plan was updated and information was collected from various universities to develop the UEE model. Strong Leadership The NTC and the new UEE system are fully supported by the government. Two large workshops with leaders from the government, universities, civil society, and development partners were held — one for the launch of the READ program in 2011 and then another after the completion of the UEE full-scale pilot in 2013. In December 2013, the President of Tajikistan officially inaugurated the new system. Regular Budget In addition to various donor funds, the government has committed to allocate a certain amount of state funds for the NTC on an annual basis. Strong Organizational Structures At the beginning of the READ program, the NTC had limited staff and no office space. By the end of 2013, a new NTC building and four regional offices had been secured, the NTC had recruited 60 full-time staff, and the NTC offices had been equipped with basic IT equipment. Effective Human Resources NTC staff and Ministry of Education officials have participated in a variety of activities aimed at building capacity to carry out assessment activities and develop the new UEE system. With the support of READ and other donor funds, staff members have participated in: àà study trips to National Testing Centers in Azerbaijan, Belarus, Russia, the Ukraine, and Turkey; àà a series of trainings on assessment topics by the Russian Testing Center; àà specific training in test development and data analysis; and àà international events on assessment topics including the annual READ Global Conferences, the 2012 READ Regional Workshop, the 2012 and 2013 IAEA Conferences, and the 2012 and 2013 EAOKO Conferences. Country-level Work: Tajikistan 53 Alignment with Learning Goals Grade 10 students participating in the full-scale To ensure that the new UEE aligns well with established university admis- pilot of the UEE in Tajikistan. sion requirements, the NTC developed a database with information from 30 universities (covering more than 400 specialties) about entrance rules, procedures, requirements, and admission plans. These data were analyzed and used to design the UEE. Opportunities to Learn about Assessment Activities A public relations and communications strategy to introduce the new UEE was developed with READ Trust Fund support. By the end of 2013, 55 articles had been published in local newspapers and posted on local and international websites, 5 films for different target audiences had been produced and broadcasted on national television, and a series of meetings took place with schools, parents, and students. Quality Assurance In 2012, the NTC designed a pilot version of the new UEE and conducted a series of mini-pilots to test various aspects of the system including the testing instrument, registration, and administration. Then, in 2013, 7,000 grade 10 students participated in a full-scale pilot, which included the revised tests, administrative, and security procedures. The pilot was deemed successful and a decision was made to implement the new UEE system on a national scale in 2014. Planned Activities In 2014, READ Trust Fund-supported activities in Tajikistan will focus on continuing to fully establish the new UEE system. The legal frameworks for the NTC will be updated to reflect the President of Tajikistan’s decision to adopt the new UEE system; the NTC will acquire the remaining IT equip- ment; the UEE will be implemented on a national scale; and an indepen- dent evaluation of the new system will be conducted. 54 READ Trust Fund 2013 Annual Report Vietnam Total Grant Amount Disbursements as of 12/31/13 US$2,811,894 US$1,331,810 COUNTRY OVERVIEW Similar to Vietnam’s rapid economic development, its education system has improved at a remarkable pace. Having nearly met the goal of universal primary education, government officials are paying even greater attention to ensuring education quality. Vietnam’s vision for its education system is one that enables the country to become a globally competitive, knowledge- based economy. To ensure that investments in education produce quality learning outcomes, the government has made measuring learning outcomes and monitoring education quality a high priority. Assessment activities are taken seriously and departments tasked with assessment responsibilities are well-staffed. The challenge Vietnam faces relates more to the massive size of its education system and the large number of key stakeholders that need additional training in order to be able to effectively design and implement assessment activities. Box 10. Vietnam’s Remarkable Performance country, its students are performing at a level on the International Stage — PISA 2012 similar to peers in Germany and Austria and above the OECD average. While these results are impres- When the READ Trust Fund first started and sive, Vietnamese graduates still lack skills that are diagnostic reviews were being carried out in each in critical need in the workforce. country, Vietnam began to strongly consider the possibility of participating in its first ILSA. Doing With support from the READ Trust Fund, Vietnam so would not only further develop this aspect of its is conducting further analysis of its PISA results to assessment system, but also allow Vietnam to see identify areas in which curricular reforms are where it stood relative to other countries. A necessary to ensure that students graduate from decision was made and Vietnam signed up to secondary school with 21st-century skills. Due to participate in PISA 2012. the READ Trust Fund program, Vietnam will have the ability to make evidence-based decisions in Vietnam’s performance in PISA 2012 was literally curriculum reform. astounding. Results from the assessment showed that although Vietnam is a lower-middle income Country-level Work: Vietnam 55 Table 9. Vietnam’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Progress by Key Indicators The 2009 baseline results showed that Vietnam’s most developed area was Classroom Assessment, even though this was only emerging. It was decided that, with READ Trust Fund support, Vietnam would focus its efforts on building capacity among a wide range of stakeholders across all assessment types. Additionally, Vietnam is working to build assessment practices into their new curriculum for 2015 and to strengthen their current system for Examinations. It is expected that by the end of READ, Vietnam will have made progress across all assessment types and will have built a strong foundation for future activities. Vietnam has been focusing its efforts on ensuring that it has a system in place in which each stakeholder has the requisite knowledge to perform their expected role in assessment activities. Table 9 indicates overall progress to date in developing Vietnam’s assessment system based on the key indicator areas being tracked by the READ Trust Fund program. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. Clear Policies The Department of Education Testing and Accreditation has been working to develop test matrices that are aligned with the curriculum, and to create first semester and end-of-year tests. Test matrices and tests have been developed in a number of subjects for grades 1-6, 10, and 12. Accompanying these matrices and tests are clear guidelines on their use and a policy and regulatory framework that supports these new procedures for evaluating student performance. 56 READ Trust Fund 2013 Annual Report Ministry of Education and Effective Human Resources university staff during the workshop on e-learning in With READ Trust Fund support, Vietnam has begun to develop an exten- Vietnam. sive capacity-building program in assessment. To date, these activities have included attendance at international conferences, a study visit, train- ing for PISA, training to improve Examinations, the development of a general competency framework in assessment, and training programs in assessment for various key stakeholder groups. àà Conferences: Delegations of five to six education officials have partici- pated in each of the annual READ Global Conferences, the 2012 READ Regional Workshop, and the 2012 and 2013 IAEA Conferences. Following these conferences, Vietnam has organized its own annual READ-Vietnam Conferences to share lessons with all key stakeholders and discuss any recent activities. àà Study Visit: In 2013, 16 Ministry of Education and university faculty leaders traveled to South Korea to meet with Curriculum Department staff and learn how assessment practices have been included in that country’s new curriculum. These lessons are being incorporated into Vietnam’s own curriculum reforms, which are underway. àà Training for PISA: Vietnam recently participated in its very first ILSA, PISA 2012. With READ Trust Fund support, a team of Vietnamese officials participated in two PISA training courses in 2009 and 2010. The PISA 2012 results were released in late 2013 and Vietnam performed very well, higher than the OECD average in all subjects. àà Training for Examinations: In 2012, through participation in workshops on the University Entrance Examinations, nearly 200 representatives from high schools, universities, and the Ministry of Education have increased capacity to develop high-quality tests. Participants in these workshops received training in how to analyze and write test items and use test matrices. As a result, item writers are better equipped to write questions that align with the intended purpose of the Examination and the process for developing University Entrance Examinations has been improved. Country-level Work: Vietnam 57 àà Framework for Building Capacity in Assessment: A general compe- tency framework, which outlines the expected skills and knowledge in assessment for five key stakeholder groups (pre-service teachers, in-service teachers, masters students, education managers, and Ministry of Education assessment specialists), has been developed. In 2013, nine workshops with a total of nearly 300 representatives from the Ministry of Education, universities, and school leaders were held to discuss and validate the overall framework and the proposed training modules. àà Training Courses for Key Education Stakeholders: Based on the skills and knowledge outlined in the general competency framework, five training modules have been developed for the different key stakeholder groups. Given the large number of individuals that need to be trained in certain groups, a two-day workshop was organized to explore the possibility of delivering one or more of these training modules online through e-learning. In 2014, these training programs will begin to be implemented across all five stakeholder groups. Assessment Quality In 2012, an Early Development Index survey was conducted to measure the school readiness of preschool children in Vietnam. Ten early childhood education experts and education officials were trained in how to adapt the tool for Vietnam and how to conduct trainings for the provincial and district officers, who then trained preschool teachers to complete the questionnaires. Planned Activities In 2014, READ Trust Fund-supported activities in Vietnam will focus on: àà developing training materials and delivering the training programs in educational assessment for the different key stakeholder groups; àà improving the quality of High School Graduation and University Entrance Examinations; àà establishing an International Advisory Board to advise on the curricu- lum reforms and incorporate a competency-based assessment approach into the new curriculum and textbooks; and àà creating an Education Quality Management System for primary education. 58 READ Trust Fund 2013 Annual Report Zambia Total Grant Amount Disbursements as of 12/31/13 US$2,350,242 US$2,189,165 COUNTRY OVERVIEW Over the past few decades, Zambia has undertaken numerous education reforms with an eye to attaining universal primary education and improving education quality. Enormous progress has been achieved in school access and Zambia has nearly met the goal of universal primary education. At the same time, serious concerns remain about education quality and learning outcomes. According to the 2012 NLSA, less than 40 percent of learners in grade 5 are acquiring even the most basic literacy and numeracy skills. Zambia’s Ministry of Education is committed to monitoring the quality of learning and to using assessment results to make evidence-based reforms. The activities that comprise the READ Trust Fund program in Zambia are focused on strengthening the capacity of the institutions responsible for measuring student learning outcomes and on ensuring that findings from assessment activities are used to improve teaching and learning. Box 11. Zambia’s Campaign to Improve the Each activity was designed to reach specific Use of National Assessment Results audiences. For instance, an easy to read brochure with a summary of NLSA findings and recommen- Although the Examinations Council of Zambia dations for action was created for teachers, school (ECZ) has been conducting NLSA activities since administrators, and policymakers. A 10-minute 1999, it has struggled to disseminate the results in film about the NLSA results, entitled “Are Zambia’s ways that optimize their use to inform policy and Children Learning?”, was created for parents and teaching practices. In 2013, through the support other concerned citizens and aired on broadcast of the READ Trust Fund, key education stakehold- television. A DVD, including a teacher discussion ers came together to devise a broad-reaching guide and case studies on effective teaching communications plan that would raise awareness strategies, also was developed for use in teacher about learning outcomes across the country and in-service trainings. motivate people to take action. Country-level Work: Zambia 59 Table 10. Zambia’s Assessment System: Baseline Results (2009) and Progress to Date (2013) Latent Emerging Established Advanced Classroom Assessment Examinations NLSA ILSA Progress by Key Indicators The 2009 baseline results indicated that Zambia’s assessment system was at an emerging level of development for all four assessment types. With READ Trust Fund support, Zambia opted to target specific activities within each assessment type that would allow its system to evolve from emerging to established. These activities included quality reviews of current assess- ment exercises, further capacity building, and better use of assessment results to improve overall education quality. It is expected that by the end of READ, Zambia will have made sufficient progress to become nearly established in three different areas—Classroom Assessment, Examinations, and NLSA. In addition, due to Zambia’s participation in the PISA for Development pilot, some progress also is expected in the area of ILSA (see page 24). For Zambia, READ Trust Fund activities have focused primarily on support- ing the Examinations Council of Zambia (ECZ) to strengthen its current assessment practices. Table 10 indicates overall progress to date based on achievements in the key indicator areas being tracked by the READ Trust Fund. A detailed breakdown of this progress, by indicator area, is provided in the following narrative. Strong Organizational Structures The lack of an adequate IT system has hampered ECZ’s ability to effec- tively store and retrieve data from previous assessments and conduct further analyses. In 2012, an audit of ECZ’s IT systems was conducted and actions identified that ECZ can take to improve its IT policies and practices. 60 READ Trust Fund 2013 Annual Report Left: Education officials at the Effective Human Resources launch event on the 2012 NLSA results. With READ Trust Fund support, nearly 1,600 teachers, ECZ staff, Education Standards Officers, and other representatives from the Ministry of Right: ECZ, the Teacher Resource Center, and the Education have participated in events aimed at enhancing local capacity to World Bank working together carry out assessment activities. These activities have included participa- to create the film, “Are tion in international conferences and targeted trainings for key education Zambia’s Children Learning?” stakeholders. àà Conferences: READ has supported the participation of more than 35 ECZ and Ministry of Education staff in various international confer- ences on assessment topics. These events have included the annual READ Global Conferences, the 2011 and 2013 AEAA Conferences, the 2011 and 2012 IAEA Conferences, and the 2012 READ Regional Workshop. A representative from Zambia also delivered a presentation at the World Bank’s 2013 high-level symposium, “Assessment for Global Learning.” As a result of their participation in these events, education officials in Zambia have a greater awareness of best practices in student assessment and how to address common assess- ment challenges. àà Training for Examination Setters and Markers: Due to the rapid expan- sion of Zambia’s education system, many additional question setters and markers were recruited to assist with the Examinations in grades 7, 9, and 12. By 2012, a total of 261 additional question setters had been trained in test design, construction, and item writing and more than 900 markers had been trained in how to mark Examinations and detect and report cheating. àà Training for Education Standards Officers: According to a study conducted in 2011, over 75 percent of Standards Officers in Zambia had never received training related to their job as school inspectors and had a very limited understanding of assessment practices. To build capacity in this area, ECZ organized several three-day workshops on the principles of assessment and school inspection for all 360 Standards Officers in the country. ECZ intends to offer similar workshops every two years. Country-level Work: Zambia 61 àà Training for ECZ Staff: In 2011, ECZ staff participated in two training ECZ staff during training activities  —  a statistics course on sampling, basic and advanced on assessment frameworks in Zambia. statistical analysis, and report writing at the University of Michigan; and a study visit to the Caribbean Examinations Board to gain a better understanding of how to create item banks, improve candidate regis- tration procedures, and use social media to distribute materials and mitigate Examinations malpractice. Then in 2013, based on the findings of an in-depth quality review of Zambia’s existing NLSA program, a series of workshops took place to train nearly 30 ECZ staff in areas such as data analysis and the use of SPSS and Item Response Theory software and report writing. ECZ’s increased capacity in these areas greatly enhanced their analyses of the 2012 NLSA grade 5 results and the design of the new NLSA for grade 9. àà Training for Teachers: A study on existing teacher capacity in assess- ment identified a need to provide more systematic training for teachers in this area. In 2013, through a series of consultations with representa- tives from the Teacher Education Unit and Teacher Training Institutions, a decision was made to develop and incorporate a course on assess- ment into the existing curriculum at all teacher training institutions. An outline for this course has been agreed and work is underway to further develop it. System Alignment Zambia’s inherited its Examinations system from the United Kingdom. To improve the alignment of their Examinations with Zambia’s current curricu- lum and national learning goals, 20 ECZ and Ministry of Education staff received training in how to develop an assessment framework and syllabi for key subject areas  English, Mathematics, and Science. This training —  enabled ECZ staff to more closely examine what their Examinations are intended to measure and to identify improvements to the current system. 62 READ Trust Fund 2013 Annual Report Assessment Quality To improve the quality of the NLSA in grade 5, an in-depth technical review was conducted. This review helped identify the strengths of the current program as well as areas in which ECZ would benefit from additional train- ing. It also highlighted various improvements for ECZ to incorporate into the design of the upcoming NLSA in grade 9. Use of Results Prior to 2013, ECZ had primarily relied on the publication of one large technical report to communicate NLSA results. With the support of the READ Trust Fund, ECZ developed a broad-reaching communications plan to disseminate the results of the 2012 NLSA in grade 5. In addition to the technical report, 15,000 brochures, 7,000 posters, a general video and two teaching case studies, and a discussion guide for teacher in-service train- ings were produced (see Box 11). An event was organized to launch these materials and engage in a national dialogue on the actions needed to address Zambia’s poor education results. This event was led by the Minister and included nearly 100 representatives from all education depart- ments, international donors, and several media outlets. Planned Activities Through the support of the READ Trust Fund, in 2013, Zambia was selected to be one of six pilot countries participating in the PISA for Development initiative. Given that this initiative is being designed to include a strong capacity building component and several peer-to-peer learning activities, Zambia will greatly benefit from its participation. The initiative will officially be launched in 2014 and will take place over a three-year period. Under the READ program, more than 1,600 Ministry of Education officials, school leaders, and teachers in Zambia have benefited from training on assessment. Country-level Work 63 Looking Ahead Education Quality and Assessment in the Post-2015 Development Agenda 2014 will be the final year of the READ program. As countries implement the last round of activities in their action plans, attention will need to be given to what comes next. When the current READ program ends, the SABER–Student Assessment tools will be used to benchmark progress in each of the READ Trust Fund-supported countries. By comparing the results against their baselines, each country will be able to determine the level of development achieved, and the steps required for continuing this important work. In the global discourse, the attention being given to the importance of learning and monitoring education quality continues to grow. In the post-2015 era, work to establish strong systems for assessing learning outcomes is likely to expand. In this regard, both the READ program and the eight countries supported under it are well positioned to play an important leadership role. Having already experienced a five- year journey of activities and outputs at global and country levels, READ offers an exciting and evidence-based model to those wishing to learn how exactly to go about improving and institutionalizing the collection and use of information on student learning outcomes. 64 READ Trust Fund 2013 Annual Report APPENDIX 1: GLOSSARY OF ASSESSMENT TERMS Classroom Assessment: A variety of tools and procedures employed by teachers and students in the course of daily classroom activity to collect and interpret written, oral, and other forms of evidence on student learning or achievement (for example, oral questioning and feedback, homework assignments, student presentations, diagnostic tests, and end-of-unit quizzes). Provides “real time” information to support the teaching-learning process. Educational Assessment A group of interconnected policies, structures, and practices for the System: generation and use of information on student learning or achievement. Formative Assessment: Ongoing assessments, reviews, and observations in a classroom as part of the instructional process. Used to modify and validate instruction. National/International Also referred to as “large-scale assessments” or “large-scale surveys.” Assessments: Designed to provide data on the achievements of students in a curriculum area, aggregated to provide an estimate of achievement levels in the education system as a whole at a particular age or grade level. Involves the administration of achievement tests either to a sample or a population of students. Teachers and others may be asked to provide background infor- mation which, when related to student achievement, can provide insights on how achievement is related to factors such as household characteris- tics and levels of teacher training. “Public” or “External” Tests that measure individual student achievement in relation to an official Examinations: curriculum or other formal body of knowledge and skills. Results are used, alone or in combination with other information, to make highly consequen- tial decisions about individual examinees, such as promotion to the next grade or admission to higher education. Also referred to as “high-stakes” tests. Standards: Content standards are broadly stated expectations of what students should know or be able to do in particular subjects and grade levels. Performance standards are explicit descriptions of what students must know or be able do in order to demonstrate a particular level of proficiency on the content standards. Student Assessment: The process of gathering and appraising information on what students know, understand, and can do in order to make an informed decision about what to do next in the educational process. Also referred to as “learning assessment,” “educational assessment,” or simply “assessment.” Summative Assessment: Typically used to evaluate the effectiveness of instructional programs and services at the end of an academic year or other predetermined time, or to make a judgment of student competency after an instructional phase is complete. Appendix 65 APPENDIX 2: 2013 AEAA AND IAEA CONFERENCES In lieu of holding a stand-alone READ Global Conference in 2013, the Left: READ-sponsored READ Trust Fund combined efforts with two large assessment associa- participants at the AEAA Conference in Tanzania. tions —  the AEAA and IAEA. Each of these associations organizes an annual conference to bring together assessment experts from across the Right: READ-sponsored participants at the IAEA globe to discuss key issues and the latest developments in the area of Conference in Israel. measuring student learning. The READ Trust Fund sponsored the partici- pation of the four READ countries from Africa (Angola, Ethiopia, Mozambique and Zambia) at the AEAA Conference in Tanzania and the other four countries (Armenia, the Kyrgyz Republic, Tajikistan, and Vietnam) at the IAEA Conference in Israel. Sharing READ Trust Fund Results At both of these events, panel sessions were organized to present the work being carried out under the READ Trust Fund program. These sessions began with some introductory remarks on the key role that assessment plays in the “Learning for All” agenda and how the SABER– Student Assessment tools can help countries further develop their systems to monitor learning outcomes. Then, a representative from each READ-sponsored country shared with the audience specific examples of activities that have led to measurable improvements in their assessment systems. Through these sessions, the program was able to communicate and link READ’s key messages and results to the bigger community of international assessment experts. These two events drew many of the world’s leading experts in assessment and provided a tremendous opportunity to disseminate and share the products developed under the READ Trust Fund. Expo stands with more than 20 different READ-supported products on student assessment were organized for each event, and more than 1,000 copies of various READ publications were distributed. Recognizing Russia’s Contributions in the Area of Education Quality At the AEAA Conference, there were over 400 participants from more than 22 African countries. At the IAEA Conference, there were nearly 370 participants from 45 countries across the globe. By partnering with assess- ment organizations like AEAA and IAEA, and sharing the innovative work being accomplished under the READ Trust Fund, Russia has become widely recognized as a key player in the area of student assessment. 66 READ Trust Fund 2013 Annual Report APPENDIX 3: WORLD BANK SYMPOSIUM— ASSESSMENT FOR GLOBAL LEARNING Left: Chair of the Association Background for the Development of Education in Africa, Dzingai From November 7–8, 2013, through the support of the READ Trust Fund, Matumbuka and World Bank the World Bank hosted a high-level symposium on “Assessment for Global Vice-President, Keith Hansen. Learning.” There is broad agreement that there is likely to be a global Right: Policy researchers, indicator centered on learning in the post-2015 agenda. However, decisions technical experts, and have yet to be made on how that learning will be defined and how it should country representatives at the symposium. be measured. This event brought together technical experts and policy researchers who are active in the development community to review and discuss the exist- ing menu of tools and approaches for monitoring and supporting student learning, and to explore how best they might be used to contribute to the needs of the post-2015 development agenda. Agenda The symposium was primarily organized as a series of panel discussions which addressed the following questions: àà Why is measurement of learning important in the post-2015 develop- ment agenda? àà What are the available tools and approaches for measuring and effec- tively using information on learning? How can these tools and approaches be adapted or improved in order to better meet the learn- ing needs outlined in the post-2015 agenda? àà How can governments best go about adapting or creating measure- ment tools and approaches that respond to the post-2015 agenda? Outcome While the international community is generally aware that there is a learn- ing crisis, there is little concrete information available on learning levels in many countries. Although measuring learning is not without its challenges, participants agreed that not to measure learning is simply not an option. Drawing on the conclusions from the panel discussions, the education community now needs to decide exactly how the global learning indicator should be defined, what it will measure, and who will be responsible. Education is one of the most important drivers “ for ending extreme poverty and boosting shared prosperity. But guaranteeing a basic education means little unless schooling leads to learning for all children and youth. Achieving better learning outcomes depends not only on having enough classrooms, teachers, and textbooks, but also on having the right policy environment to ensure they contribute to learning.” Elizabeth King, Director of Education of the World Bank Group Education World Forum, January 22, 2014 68 READ Trust Fund 2013 Annual Report Russia Education Aid for Development Trust Fund Education, Human Development Network World Bank 1818 H Street, NW Washington, DC 20433 USA Website: www.worldbank.org/readtf Email: readtf@worldbank.org