Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004267 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 16496, TF A2125, Q 8780, H9160, D1370 ON A IDA GRANT IN THE AMOUNT OF SDR 7.8 MILLION (US$11.9 MILLION EQUIVALENT) AND GLOBAL PARTNERSHIP FOR EDUCATION GRANT IN THE AMOUNT OF US$6.9 MILLION ADDITIONAL IDA GRANTS IN THE AMOUNT OF SDR 5.4 MILLION (US$7.5 MILLION EQUIVALENT) AND AND AN EARLY LEARNING PARTNERSHIP (ELP) GRANT IN THE AMOUNT OF US$1 MILLION ON THE READ: RESULTS FOR EDUCATION ACHIEVEMENT AND DEVELOPMENT PROJECT TO THE GOVERNMENT OF THE GAMBIA February 28, 2019 Education Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 4, 2019) Currency Unit = GMD 1US$ = 0.58252 SDR 1US$ = 50.28 GMD FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez M. H. Ghanem Country Director: Louise J. Cord Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Meskerem Mulatu Task Team Leader(s): Ryoko Tomita ICR Main Contributor: Alison Marie Grimsland ABBREVIATIONS AND ACRONYMS AM Aide Memoire AF Additional Financing AFDB African Development Bank CBA Cost-Benefit Analysis CCM Coordinating Committee Meeting CCT Conditional Cash Transfer CEN Country Engagement Note CF Catalytic Fund COT Classroom Observation Tool CSR Country Status Report DLI Disbursement Linked Indicator ECD Early Childhood Development EFA-FTI Education For All – Fast Track initiative EGRA Early Grade Reading Assessment ELINL Early Literacy in National Language ELP Early Learning Partnership EMIS Education Management Information System ESMF Environmental and Social Management Framework ESSC Environmental and Social Safeguards Checklist ESSP Education Sector Strategic Plan ESSP Education Sector Support Program EVD Ebola Virus Disease FM Financial Management GDP Gross Domestic Product GER Gross Enrollment Rate GMD Gambian Dalasi GPE Global Partnership for Education GPI Gender Party Index IHS Integrated Household Survey HR Human Resources HTC Higher Teachers Certificate ICR Implementation Completion and Results Report ICT Information and Communications Technology IDA International Development Association IE Impact Evaluation IFC International Finance Corporation IFR Interim Financial Report IHS Integrated Household Survey IP Implementation Progress IPF Investment Project Financing IRI Intermediate Result Indicator IRR Internal Rates of Return ISR Implementation Status Results Report JPS Joint Partnership Strategy JSDF Japanese Social Development Fund LAN Local Area Network LBS Lower Basic School LEARNET Learning, Education, Activities and Resources Network LEG Local Education Group LMS Learning Management System MDG Millennium Development Goal M&E Monitoring and Evaluation MoBSE Ministry of Basic and Secondary Education MoFEA Ministry of Finance and Economic Affairs MoHERST Ministry of Higher Education, Research, Science and Technology MS4SSA Math and Science for Sub Saharan Africa MTEF Medium Term Expenditure Framework MTP Medium-Term Plan NAT National Assessment Test NCB National Competitive Bidding NEA National Environmental Agency NER Net Enrollment Rate NPV Net Present Value PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PPA Project Preparation Advance PPP Public-Private Partnership PSI-PMI Progressive Science Initiative Progressive Math Initiative PTC Primary Teacher Certificate PTR Pupil Teacher Ratio RBF Result-Based Financing READ Results for Education Achievement and Development RED Regional Education Directorate RPF Resettlement Policy Framework SCD Systematic Country Diagnostic SDP School Development Plan SDR Special Drawing Right SEGRA Serholt Early Grade Reading Ability SIG School Improvement Grants SMS Short Message Service SQAD Standards and Quality Assurance Directorate SSS Senior Secondary School STED Science Technology Education Directorate TTL Task Team Leader TVET Technical and Vocational Education and Training UBS Upper Basic School USD United States Dollar WAEC West African Examination Council WASSCE West African Senior School Certificate Examination TABLE OF CONTENTS DATA SHEET ............................................................................................................................ 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 15 II. OUTCOME .................................................................................................................... 20 A. RELEVANCE OF PDOs ............................................................................................................ 20 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 21 C. EFFICIENCY ........................................................................................................................... 24 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 27 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 27 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 28 A. KEY FACTORS DURING PREPARATION ................................................................................... 28 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 29 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 30 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 30 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 32 C. BANK PERFORMANCE ........................................................................................................... 33 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 34 V. LESSONS AND RECOMMENDATIONS ............................................................................. 35 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 37 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 68 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 71 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 72 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 83 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 94 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name GAMBIA - READ: Results for Education Achievement and P133079 Development Project Country Financing Instrument Gambia, The Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Additional Financing P160282-Results for 14-Sep-2016 IBRD/IDA Education Achievement and Development Project (READ) Organizations Borrower Implementing Agency Minister of Finance and Economic Affairs Ministry of Basic and Secondary Education Project Development Objective (PDO) Original PDO To increase access to basic education, improve quality of teaching and learning in lower basic schools, and strengthen education systems. Page 1 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 11,900,000 11,900,000 11,485,170 IDA-H9160 6,900,000 6,900,000 6,900,000 TF-16496 7,500,000 7,500,000 7,481,384 IDA-D1370 1,000,000 1,000,000 1,000,000 TF-A2125 Total 27,300,000 27,300,000 26,866,554 Non-World Bank Financing 0 0 0 Borrower/Recipient 16,000,000 0 0 Total 16,000,000 0 0 Total Project Cost 43,300,000 27,300,000 26,866,554 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 06-Mar-2014 25-Apr-2014 05-Dec-2015 28-Feb-2018 31-Aug-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 14-Sep-2016 9.37 Additional Financing Change in Results Framework Change in Financing Plan Change in Disbursements Arrangements 09-Feb-2018 16.24 Change in Results Framework Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial Page 2 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 17-Jun-2014 Satisfactory Satisfactory 5.71 02 18-Dec-2014 Satisfactory Satisfactory 6.51 03 17-Jun-2015 Satisfactory Satisfactory 9.33 04 23-Dec-2015 Satisfactory Moderately Satisfactory 10.43 05 22-Jun-2016 Satisfactory Moderately Satisfactory 14.04 06 20-Dec-2016 Satisfactory Satisfactory 14.87 07 29-Jun-2017 Satisfactory Satisfactory 22.23 08 05-Jan-2018 Moderately Satisfactory Moderately Satisfactory 23.71 09 21-Jun-2018 Moderately Satisfactory Moderately Satisfactory 26.84 10 30-Aug-2018 Satisfactory Satisfactory 26.87 SECTORS AND THEMES Sectors Major Sector/Sector (%) Information and Communications Technologies 3 Public Administration - Information and 1 Communications Technologies ICT Services 1 Other Information and Communications Technologies 1 Education 95 Public Administration - Education 25 Primary Education 12 Other Education 58 Page 3 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Social Protection 2 Social Protection 2 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Social Development and Protection 2 Social Protection 2 Social Safety Nets 2 Human Development and Gender 100 Education 100 Access to Education 27 Education Financing 4 Science and Technology 23 Teachers 23 Standards, Curriculum and Textbooks 23 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Vera Songwe Louise J. Cord Senior Global Practice Director: Tawhid Nawaz Jaime Saavedra Chanduvi Practice Manager: Peter Nicolas Materu Meskerem Mulatu Task Team Leader(s): Nathalie Lahire Ryoko Tomita ICR Contributing Author: Alison Marie Grimsland Page 4 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country Context 1. At the time of project appraisal, The Gambia had an estimated population of approximately 1.8 million, of which more than 60 percent were below the age of 24. High annual population growth projections of 2.8 percent continued to put pressure on public services systems. Further, vulnerability to external shocks related to both the global economic crisis and adverse weather conditions had led to an economic slowdown, with a real GDP growth of 2.2 percent in 2011. Although there was some limited recovery due to agricultural growth and tourism sector development in 2012, the fiscal environment for social services such as health and education was severely constrained due to large debt burdens and the limited formal economy. In the education sector, for example, external financing comprised more than 35 percent of the total education budget in 2009 with the World Bank as the largest contributor. 2. Despite the constrained resources for investment, education was seen by the Government as critical to boosting future growth and supporting economic development. Although the poverty rate had decreased in the decade prior to appraisal, the overall poverty headcount index at the time of appraisal was estimated at 48.4 percent. Further, there was a high degree of income inequality. In 2010, the poorest quintile of the population consumed only 5.6 percent of overall expenditures, while the wealthiest quintile consumed 46.5 percent. Sector Context 3. The Government’s commitment to education had historically been strong in the years leading up to project preparation and at the time of appraisal. In 2012, government expenditure on education as a percentage of total government expenditure was 20.07 percent (Ministry of Basic and Secondary Education (MoBSE) 17.5 percent and Ministry of Higher Education, Research, Science and Technology (MoHERST) 2.57 percent). However, salaries comprised the bulk of education expenditure, leaving less than 5 percent for other expenditures, including those to improve education quality. In The Gambia, the education sector was, and still is, supported by only a small number of donors; and external financing remains critical in funding non-salary expenditures in the sector. 4. Despite these fiscal constraints, the Government had made significant efforts to address access and quality challenges through, for example, incentive schemes to: enhance teacher deployment and raise qualifications; integrate public school curricula into madrassas1; and offer technology-informed teaching models to senior secondary schools, among others. Historically, interventions were piloted, evaluated, and eventually scaled up as appropriate. 1Madrassas teach English, Math and Science. Madrassas that are recognized by the Government use the government curriculum. On the other hand, Majalis teach the Koran and Islamic values but do not teach literary and numeracy. In many Majalis, children are in the custody of religious leaders while Daaras are places where children go and learn the Koran and Islamic values after school or during school holidays. Page 5 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) This approach to the introduction of new intervention was feasible, in part, by the small size of the country and the strong leadership of the Government. 5. At the time of project preparation, despite strong government commitment and some marginal improvements in education indicators over the previous decade, the sector still faced numerous challenges related to ensuring that: (i) all children attend school; (ii) children learn effectively while they are in school; and (iii) the education system had the capacity to deliver high quality education services across all regions. The schooling system in The Gambia has some inherent complexity in that multiple models of schooling are available to children. This includes private schooling, public schooling, madrassas that are recognized by the state and use the official curriculum, madrassas that are not recognized by the state, and Majalis or Daaras/ Koranic schools that do not teach foundational literacy skills and are not recognized by the state. 6. Inequitable access to education. From the period of 2003-2013 there was a 35 percent increase in enrollment in lower and upper basic education; however, the growth in enrollment was matched by the growth in the school-aged population, leaving the gross enrollment rate (GER) almost static. In 2013, the early childhood development (ECD) enrollment rate at 36.5 percent stood above the sub-regional average; however, the majority of ECD provision was private, and concentrated in urban centers, which severely limited opportunities for poorer, rural families. Despite these inequities in access, the increase in ECD enrollment between 2006 and 2013 from 21 percent to 36.5 percent signaled a growing demand for ECD and was seen as a promising strategy for improving school readiness and on-time enrollment rates in subsequent years. 7. At the time of appraisal, the Government had been working to enact a policy in which each community would have a school within three kilometers to ensure adequate access, and a thorough mapping exercise was carried out to identify which communities remained without adequate access to a school. At project appraisal in 2013, 17 percent of communities (or 339 communities) were without a Lower Basic School (LBS) within three kilometers and these were located mostly in the remote Regions 4, 5 and 6.2 Even in cases where there was a primary school within three kilometers of the village, for very young students (early grades, and in some cases early childhood students) the walk was too cumbersome. As a result, the Government had been piloting a transportation program in which donkey carts were developed as a type of school “bussing” system to ensure children could easily access schools. Weak infrastructure remained a challenge at all levels of the education system, but particularly in upper basic schools (UBS) and senior secondary schools (SSS). For example, only 35 percent of UBS and SSS had grid electricity at the time, and 80 percent were without Internet. 8. In terms of gender disparities in access, The Gambia had made significant progress in ensuring gender parity at the primary level, with gender disparities virtually nonexistent within the early grades. A policy which had been enacted at the time of preparation ensured certification and registration of Islamic schools (also called Madrassas) (which were following the government curriculum and where the Government was subsidizing English teachers) had been successful in incentivizing those families who were initially reluctant to send their children to school. This was 2The Gambia is divided into six administrative regions, which are numbered from 1 to 6. Region 1 is the capital city area of Banjul, and region 2 is the area closes to the capital, including the main tourist and commercial activities. Regions 3,4, 5 and 6 are progressively more remote form the capital, with region six being the most remote. Page 6 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) the case particularly in Region 6. Despite these efforts, significant regional disparities in enrollment remained. In lower basic education, for example, the GER was 110 percent in Region 1, but only 58 percent in Region 5. 9. In terms of socio-economic disparities, Early Grade Reading Assessment (EGRA) results also revealed disparities across socio-economic status with wealthier students outperforming their poorer peers with consolidated scores of 59.1 versus 48.5. Analysis at the time revealed that the wealthiest students consumed, on average, eight times more public resources than the poorest students, and, despite a school fee abolition policy in lower basic education, informal costs of education (uniforms, stationery, sandals, transportation costs) created barriers to schooling, particularly for the poorest families. Indeed, a survey of out-of-school children and data from the Integrated Household Survey (IHS) revealed that the reasons cited for school-aged children having never attended or having left the school system were largely centered around financial barriers as well as religious preferences, with the most common reasons being: (i) cost of access (46 percent); (ii) distance to school (15 percent), (iii) work and family activities (7 percent) and attendance in Majalis – Koranic schools (7 percent). The key costs cited included informal school fees and other participation costs such as the costs of uniforms, shoes, school bags and lunches. At appraisal, the number of out-of-school children was estimated to be 76,000. About 7 percent of families chose to enroll their children in Majalis rather than public schools for the reasons stated above, though this figure might actually have been higher. This is particularly true of Regions 5 and 6, where in some cases the enrollment rates in formal school at the community level hovered at 20 percent, with a large majority, particularly of school-aged boys, attending Majalis. Within the Majalis or Daaras, the state curriculum is not integrated, there is limited, if any, formal regulation, and literacy and numeracy are typically not taught. 10. Lagging educational quality. Improving students’ learning outcomes was identified as a national priority, prompted by the initial EGRA undertaken in both 2007 and 2009 which revealed low scores in all sections— letter/word sound, letter/word recognition, reading fluency and reading comprehension. The Government responded to these findings by providing extensive training of teachers in the phonics method. EGRAs conducted as the project was being prepared (2011, 2013) pointed to the positive impact of training especially on correct words per minute. Despite these improvements, however, challenges specifically in reading fluency and comprehension remained. The National Assessment Test (NAT) conducted annually in grades 3, 5, and 8 revealed further challenges in the quality of education provided. For example, in 2012, the average NAT score in English was 42 percent in grade 3 and 45 percent in grade 5. In mathematics, the scores were 37 and 42, respectively. The challenges in quality were attributed, in particular, to limited teacher competency both in subject area and pedagogy. 11. The quality of entrants for lower basic teaching in initial teacher training programs in The Gambia College (teacher training college) had been improving in the years prior to the project, including through the introduction in 2012 of the Primary Teacher Certificate (PTC) extension program. However, the quality of the teacher training programs remained low, and required further improvement to reflect best practices and recent system reforms, including improvements with regards to: literacy teaching and mother tongue instruction; improved pedagogic skills and content knowledge; and integration of information and communications technology (ICT) into teaching methods. Further, challenges in attracting high caliber teaching candidates remained a systemic challenge, as did equitable deployment of teachers throughout the country. Page 7 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Higher Level Objectives 12. The Results for Education Achievement and Development (READ) Project was designed to support a number of activities which aimed to increase equitable access to quality education and to strengthen sector management which were outlined in the Government’s Education Sector Strategic Plan (ESSP) II 2014-2022 and associated Medium-Term Plan (MTP). This focus was strongly aligned with the Government’s higher-level objective of achieving universal access to quality basic education and improving access to relevant and quality secondary education and training. The ESSP proposed a programmatic shift from a focus largely on the cycles of education (basic education, secondary education, etc.) to more results-oriented programming and select priority areas. The project sought to support this strategic plan through activities focused on increasing access and equity, and improving quality, as well as on promoting a more efficient delivery of education services, especially through the use of a results-based financing (RBF) approach in the form of disbursement linked indicators (DLIs). 13. The proposed project was also consistent with the Second Joint Partnership Strategy (JPS) (FY2013–FY2016 – Report No. 72140) developed by the Government of The Gambia, international Development Association (IDA), International Finance Corporation (IFC), and African Development Bank (AfDB) in 2013. The JPS represented a core strategy of the development partners from 2013 to 2016 and provided the framework for managing and harmonizing development efforts supported by the Government and bilateral and multilateral donors in The Gambia. The project was specifically aligned with Pillar 2 – Strengthening the institutional capacity for economic governance and public service delivery. The aim was to build upon previous efforts aimed at improving transparency with a focus on performance and quality of service delivery, as well as progress towards achievement of the millennium development goals (MDGs). The JPS primarily highlighted the central objective of improving the quality of basic education and, in turn, learning outcomes. The JPS further highlighted that support to ECD would be critical for school readiness and improved learning outcomes in basic education. The JPS also suggested that the provision of hardship allowances to teachers in remote and poverty-stricken areas be continued as it contributed to a more equitable deployment 3 of qualified teachers throughout the country. 14. Lastly, the READ Project was aligned with the twin goals of the World Bank – reducing extreme poverty and boosting shared prosperity (2013). The focus on building human capital through quality education had the long-term goal of improving productivity and earnings of the population. Further, improved access to education especially among the poorest, and improved quality education for all, would increase the ability of the bottom 40 percent of the population (according to income) to improve their earning potential, engage more effectively in public policy decisions, and improve their quality of life. Theory of Change (Results Chain) 15. The overarching objectives of the project were to: (i) improve access to basic education; (ii) improve quality of teaching and learning in lower basic schools; and (iii) strengthen education systems. As such, the project focused on several areas of support to lead to improved access, learning outcomes, and service delivery structured around three traditional IPF components, and a fourth results-based financing (RBF) component. The project was co-financed by 3Nationally, on average, the percentage of qualified LBS teachers in hardship areas has increased from 90 percent at project appraisal to 96 percent at project closing. Page 8 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) IDA and the Global Partnership for Education (GPE). At appraisal, in an effort to showcase the Government’s commitment to the activities supported under the Project, an indicative amount of US$16 million was pledged by the Government although this did not constitute a formal co-financing arrangement. With the combination of a traditional investment instrument and results-based financing, the project was designed to provide additional resources to the heavily donor-dependent sector and initiate reforms critical to the success of the access, quality, and system-strengthening sub-objectives. Table 1: Theory of Change Activities Outputs PDOs/Outcomes Long term Outcomes Construction of classrooms, LBS and ECD Increased Reduced number of out- fabrication of donkey carts classrooms, donkey enrollment of-school children carts Higher NER Training of teachers Reading curriculum Improved Increased productivity Hardship allowance review, training of teaching and teachers, teacher learning outcomes deployment Support to EMIS EMIS strengthening, System Reduced number of out- School grants Abolishment of fees strengthening of-school children Comparable, reliable Better data mechanism assessment data established and informed 16. for policy making Project Development Objectives (PDOs) 16. As described in the Financing Agreement, dated April 9, 2014, the PDO was to increase access to basic education, improve quality of teaching and learning in lower basic schools, and strengthen education systems. The PDO was identical in the Project Appraisal Document (PAD) and remained the same over the life of the project. Key Expected Outcomes and Outcome Indicators 17. It was expected that the project-supported activities would result in the key outcomes of: (i) increased access to basic education; (ii) improved quality of teaching and learning; and (iii) system strengthening of the sector. The five PDO-level indicators, of which one was a disbursement-linked indicator (DLI) were directly linked to the overarching objectives and were supplemented by 25 intermediate results indicators (IRIs), including 8 DLIs. PDO-level indicator 2 corresponded to sub-objective 1 (increase access to basic education); PDO-level Indicators 3 and 4 corresponded to sub-objective 2 (improve quality of teaching and learning in lower basic schools); and PDO Indicator 5 corresponded to sub-objective 3 (system strengthen education systems). A more thorough description of each indicator is provided below.  PDO-level Indicator 1 measured the total number of direct beneficiaries as calculated by the number of students who directly benefited from the project. The end target was 290,000 beneficiaries. Page 9 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079)  PDO-level Indicator 2 measured the number of additional students in public ECD, LBS, and UBS. As such it directly measured the evolution of access to school by tracking additional enrollments with a target of an additional 35,000 students.  PDO-level Indicator 3 measured mean scores in grade 3 (English) on the National Assessment Test (NAT). This was an adequate measure of improvement in learning outcomes, specifically reading, with special attention paid to linked anchor items between tests in years 2012 and 2016 to ensure reliability of the measure. The end target was 46 percent from a baseline of 41.5 percent.  PDO-level Indicator 4 measured the percentage of teachers in grades 1-3 showing good teaching performance as demonstrated through the new classroom observation tool (COT). This was a quantitative measure of improvement in teaching performance, which supplemented the direct measure of learning outcomes of PDO Indicator 3. The end target was set at 60 percent from a baseline of 0 percent.  PDO Indicator 5 measured the percentage of head teachers in lower basic schools adjusting school development plans in response to NAT results and approved by Regional Directorates using an agreed upon checklist (DLI). This was an adequate measure of system functioning, given that for head teachers to adjust school development plans in response to NAT results, the service delivery chain would need to be functioning effectively including (i) on time roll-out and analysis of national assessment tests; (ii) distribution of test results via regional education directorates (REDs) to schools; (iii) incorporation of test results into school development plans (SDPs) by head teachers via consultations with school management committees (SMCs); and (iv) analysis and approval of SDPs by regional education directorates/Standards and Quality Assurance Directorate (SQAD). The end target was 80 percent. Components 18. The project had the following components and sub-components: 19. Component 1. Increase access to basic education (estimated cost: US$6.0 million, revised cost (restructuring August 2016): US$6.0 million, actual cost: US$6.0 million). Component 1 was comprised of a series of interventions which were meant to improve access to education. The interventions each targeted specific barriers to access which had been identified in an analysis of IHS data and a survey of out-of-school youth, including long distances to school, high private education costs, and limited access to ECD opportunities given its impact on improving school readiness and on time enrollment in early grades. The sub-components included: 20. Sub-component 1.1 School construction. This sub-component supported construction of 40 multigrade schools (80 classrooms) in communities without an LBS within three kilometers, in support of the government policy to provide LBS within that distance. This built on previous IDA/GPE projects which supported the construction of basic cycle schools and multigrade schools, as well as construction of classrooms, teacher housing units, and rehabilitation of additional classrooms. Despite the improvements in access, data at the time of project preparation showed that 17 percent (339 communities) of all communities in the country did not have an LBS within three kilometers, thus in this context, these communities were selected for multigrade school construction. Water points were also provided and the sub-component also supported the completion of an additional 36 classrooms which had not been fully completed under the previous project. Page 10 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 21. Sub-component 1.2 Donkey Carts. The provision of 100 donkey carts, building upon the 82 supplied under the previous project, were supported under sub-component 1.2 to ease long distances to school which had been highlighted as a barrier to access. This was targeted in particular for children in early grades, who despite living within three kilometers of an LBS would still find the walk cumbersome. This sub-component included a redesigned lighter and safer donkey cart which acted as a kind of bussing system in the communities which were identified as being the farthest from schools and having the largest populations. 22. Sub-component 1.3 Community Based Early Childhood Care and Development. This sub-component focused on sustaining and expanding ECD provision by supporting: (i) the construction of 40 ECD sheds using the previously tested Japanese Social Development Fund (JSDF) community-based approach; (ii) training of facilitator volunteers and development of a program for ECD certification; (iii) provision of teaching and learning materials; (iv) supervision and monitoring by the ECD unit within MoBSE; (v) follow up data collection as part of an impact evaluation comparing the annexed ECD approach and community based approach4; and (vi) development of an ECD monitoring tool. 23. Sub-component 1.4 Provision of school sub-grants to support sub-projects to all lower and upper basic public schools and stationery to all students in lower basic education in hardship areas . Sub-component 1.4 was designed to be closely coupled with the abolishment of school fees for parents (elaborated as part of the DLIs in Component 4) to remove the financial burden of schooling that was affecting many low-income families. It included the provision of sub-grants to substitute for informal school fees. The approach for sub-grants to schools was phased with plans for the Government to take over the payment of sub-grants by the final year of the project. In addition, distribution of stationery packets for all LBS students in the designated hardship areas (there were about 36,000 students) in 2014 and 2015 was planned to address the financial burden of purchasing notebooks, pens, pencils and erasers, which children were required to have for school. The intervention was designed to have the Ministry take over financing of these items in 2016/2017. 24. Component 2. Improve quality of teaching and learning (estimated US$6.2 million, revised cost (restructuring August 2016): US$18.0 million, actual cost US$17.3 million). Component 2 was designed to improve the quality of teaching and learning principally through 8 interlinked sub-components which focused on: (i) the reform of pre-service teacher training and incentives to attract new teachers; (ii) support to in-service teachers (both financial support via top-ups and in terms of professional development); and (iii) improved reading and foundational skills (with a focus on reading in early grades, and ICT use). Table 2 below outlines the teacher quality interventions. Table 2: Teacher quality interventions Incentives to attract, retain and Stipends for student teachers, Hardship allowance for teachers in deploy teachers especially mathematics teachers remote areas Enhanced Training to improve quality Enhanced initial training Primary Expanded in-service courses, Teachers Certificate (PTC) and Higher targeting specific needs Teachers Certificate (HTC) 4Annexed ECD is attached to or in the same compound as LBS and qualified PTC teachers teach there. On the other hand, in community based ECD, the community and facilitators who had ECD training provided by the ECD unit teach there. Page 11 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Pilot to develop and disseminate Development of pilot of early reading in national languages in grades 1-3 and effective teaching methods continued support for reading in English Observation to provide coaching and Teacher supervision and support through classroom observation feedback 25. A more detailed description of the interventions includes the following: Pre-service training and incentives to attract new teachers  Sub-component 2.1 Enhanced Teacher Training included the reform of pre-service teacher training programs to focus more strongly on relevant subject content knowledge and pedagogical skills, the teaching of reading, and relevant use of ICT, as well as the development of a professional examination to provide a rigorous assessment at the end of initial teacher training; and  Sub-component 2.2 Stipends for teacher students were designed to attract students with the required qualifications for more advanced teacher training (the HTC program) via modest stipends of GMD150 per month (about US$5). The project planned to provide the stipend to approximately 1,600 HTC students per year and 700 PTC students in their third year to support their participation in in-service teacher training activities. Those enrolling in mathematics (about 260 annually) received an additional top up given shortages of teachers in this subject area. Improvement of in-teacher support  Sub-component 2.3 Hardship allowances for public lower and upper basic teachers included financing of the continuation of hardship allowances to lower basic teachers, increasing the scale of allowance for those in the most remote schools, and providing an additional incentive for female teachers in hardship schools. It was designed to complement the Government’s program in parallel which was financing the extension of the hardship allowance scheme to cover eligible upper basic teachers;  Sub-component 2.4 Extended teacher continuing professional development was meant to deliver a demand driven range of in-service professional development courses, with formal assessments, for up to 4,000 lower basic teachers (about 1,000 a year) using a combination of face-to-face training during school vacation times and ICT- supported learning at the cluster centers; and  Sub-component 2.5 Teacher supervision and support through the use of a standardized classroom observation tool (COT) provided constructive feedback to teachers and an assessment of teaching through the COT, from which data could then be aggregated and used as a measure of quality for use in policy formulation and informing SDPs. Improving Reading and Foundational Skills  Sub-component 2.6 Reading in the early grades with a focus on building reading abilities was designed to support the Government’s Early Literacy in National Language (ELINL) program by supporting evidence-based reading teaching practices, review of curriculum, development of reading materials, and monitoring of student improvement in reading over time; Page 12 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079)  Sub-component 2.7 Developing the evidence base for improved reading performance through the continuous support of the Conditional Cash Transfer (CCT) program for Koranic schools (Majalis/Daaras) which aimed at reinforcing and evaluating the Majalis program whereby in exchange for modest CCTs, the Majalis integrated literacy and numeracy curriculum into the schools to improve foundational skills for those children who were not enrolled in formal schools; and  Sub-component 2.8 LEARNET, a Public-Private Partnership (PPP) initiative for ICT integration in learning and teaching was designed to utilize technology (devices/tools) to improve teaching efficacy and learning outcomes. The focus was premised on a teacher-centered, classroom-oriented approach (rather than a computer lab setting) with the teacher using devices and tools such as whiteboards, projectors, etc. to improve pedagogy. LEARNET’s envisaged structure consisted of five key elements: (i) broadband connectivity for public UBS and SSS schools; (ii) digital resources for curriculum linked content via local area networks (LAN) and common portal access with centrally hosted (cloud-based) data; (iii) teacher development using a Learning Management System (LMS) as a tool and use of e- learning centers; (iv) devices focused on teacher-centric solutions; and (v) renewable energy including off-grid energy solutions for schools. The sub-component’s business model was PPP based to maximize scale, reduce costs (especially in terms of high capital costs such as broadband and devices), and to foster innovative approaches. Component 3. Technical and Institutional Support (estimated US$3.0 million, revised cost (restructuring August 2016): US$1.7 million, actual cost US$2.4 million) 26. Component 3 was designed to support institutional and capacity strengthening interventions to facilitate the implementation of the other three components and build effective communication within the Ministry to internal and external stakeholders. This includes development and implementation of a communications strategy including dissemination of information, training, outreach campaigns and surveys (sub-component 3.1) as well as Institutional Support and Capacity Building (sub-component 3.2) via continuous implementation of assessments of learning outcomes; strengthening data collection and utilization by reinforcing EMIS; building statistical capacity and regular data collection on student and teacher absenteeism; and operating costs of the project coordination unit (PCU). Component 4. Strengthening the education system with a focus on governance and management results (estimated US$3.6 million, revised cost (restructuring August 2016): US$1.6 million, actual cost US$1.6 million) 27. Under this component, key governance and management reforms, meant to address weaknesses in the system and introduce best practices, were addressed by utilizing DLIs as part of a four-year program of expenditures derivative of the Government’s MTEF (FY2013-2015) plus one extra forward year (FY2016). This component was designed to be cross-cutting in that it would incentivize reforms to support the other three components’ objectives (and thus the project’s sub-objectives) of: (i) improved access; (ii) improved performance of teachers and students; and (iii) system strengthening. It was designed to change critical behaviors that were posing a challenge to the education system at the time of appraisal, changes without which the other investments would have had limited effect. For example, utilization of national assessment tests is critical to school improvement, however, at the time NAT data was not systematically used to inform school improvement plans, and, it was found that there was a practice of excluding low performing students from sitting for the exams. As a result, the project included DLIs around ensuring Page 13 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) that all Grades 3, 5, and 8 students were participating in the assessment, and that assessment results were used to inform school improvement plans – thus contributing to the sub-objective of improved quality of learning, but also to further strengthen the education system. The component consisted of 9 DLIs each weighted US$400,000. They were aimed at aiding the Government in achieving key reforms such as the abolishment of school fees and establishment of reliable assessment data. The DLIs are detailed in Table 3 below, including the related component/sub-objective that each DLI was designed to support. Table 3: DLI Mapping by Sub-Objective DLI Description Sub-objective supported through the DLI Increase access Improve Strengthen to basic quality of education education teaching and system learning Timely MOBSE shall publish the package of final provision of bidding documents for the construction of X basic school 40 multigrade schools and 40 water points facilities Timely MoBSE shall transfer the first tranche provision of (equivalent to 40 percent of the entire sub-school annual allocation) of annual school sub- X X grants (lower grants to all public lower basic schools basic) Timely MoBSE shall transfer the first tranche provision of (equivalent to 40 percent of the entire sub-school annual allocation) of annual school sub- X X grants (upper grants to all public upper basic school basic) No charge of Annual formal and informal school fees paid school fees by parents to all public LBS shall be equal to X (lower basic) zero No charge of Annual formal and informal school fees paid school fees by parents to all UBS shall be equal to zero X (upper basic) Increase school MoBSE shall complete the verification on an attendance in aggregate average of 93 percent of students X X NATs5 in grades 3, 5, and 8 of all public schools in NAT for two full days in 2014 5 This DLI was critical to ensuring accuracy of national assessment results and in response to suggestions that the previous practice in some schools was to have only higher achieving students participate in assessments, thus resulting in findings which were skewed and, therefore, not representative of the total student body. The introduction of the DLI aimed to ensure fuller participation of all students as a way to increase the representativeness of the collected data. Page 14 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Increase school MoBSE shall complete the verification on an attendance in aggregate average of 95 percent of students X X NATs6 in grades 3, 5, and 8 of all public schools in NAT for two full days in 2015 Enhancement Improved school development plans for 70 of social percent of lower basic school in response to service NAT results shall be approved by the regional X X delivery directorates under the supervision of MoBSE and using the agreed upon checklist Improvement MoBSE shall complete the annual school of data census, including Madrassas and with a collection minimum of 95 percent response rate and X results and analytical report released by May 30, 2014 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 28. An additional financing (AF) with a restructuring was approved on September 14, 2016. The AF included a grant in the amount of SDR 5.4 million (US$7.5 million equivalent) from IDA as well as a grant from the Early Learning Partnership (ELP) Trust Fund (TF) in the amount of US$1 million. The original GPE financing of U$S6.9 million remained the same. The AF supported: (a) emergency temporary support for teachers’ salaries, allowances, and school grants (IDA) in response to the fiscal crisis in The Gambia stemming from a combination of economic and political crises; and (b) further development of ECD activities made possible by a grant from the ELP TF. 29. The fiscal crisis: In the period up to 2016, The Gambia’s fiscal position had become increasingly precarious. The fiscal deficit averaged 11 percent of GDP from 2013 through 2015. Public sector debt rose from 83.3 percent of GDP in 2013 to 108 percent in 2015. As a result, interest payments increased from 25 percent of revenues in 2013 to 40 percent in 2015 and were projected to reach nearly 50 percent in 2016. There were multiple causes of this crisis. The imposition of currency controls from 2013 to 2016, and the overvaluation of the Dalasi by as much as 30 percent, inhibited competitiveness. From 2014 to 2016 the Ebola virus disease (EVD) in the region resulted in a dramatic drop in tourism. Tourism typically accounted for 40 percent of GDP, and tourism revenue had fallen by approximately 60 percent during the EVD crisis. 7 This loss of income had been compounded by drought conditions in both 2011 and 2014, which reduced total agricultural output. By 2016 it was clear that the fiscal position was unsustainable, and the Government introduced a range of cutbacks. Given the severity of this fiscal situation, the Bank was proactive in supporting the Government in stabilizing the situation by provided additional financing in the Health, Education and Governance sectors. This AF approved under this project was instrumental in ensuring the gains which had been made in the sector up until that point could be maintained. The AF financed critical recurrent costs, including teacher salaries (US$5.3 million) and school grants (US$2.3 million). 6See above footnote. 7Although there were no documented cases of EVD within The Gambia, there was a severe impact on tourism due to the EVD cases throughout the region. Tourism had typically accounted for 40 percent of The Gambia’ annual economic output. Page 15 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 30. The AF was accompanied by a restructuring that introduced the following modifications: (a) changing five DLIs to IRIs due to challenges with the mechanism beyond the implementing agency control; (b) adjustment of an activity that supported broadband connectivity and electricity to focus more heavily on technology-based teaching methods; (c) evidence-based changes to the provision of ECD, moving from a community-based to an annexed approach; and (d) provision of textbooks. To accompany these changes, small modifications were made to the results framework, a reallocation of financing by category was introduced as well as changes to the costs of some components. The closing date of February 28, 2018 remained the same for the AF but was later extended to August 31, 2018 through a second restructuring approved on February 9, 2018 (which included no other changes). 31. The RBF mechanism was also reevaluated in this context of fiscal difficulties. Although the MoBSE had been making great efforts to achieve each of the 9 DLIs, and had at the time of AF approval already achieved 7 DLIs, the fiscal constraints at the central level posed challenges to the proper functioning of the RBF mechanism. According to the project design, the DLI funds under Component 4 were designed to be transferred to the MoFEA’s consolidated account once the MoBSE met the annual DLIs; via an internal agreement, MoFEA was then to transfer the funds to MoBSE. In a resource-constrained context, the MoFEA was unable to fully execute the budget and could not release the funds to the MoBSE. For example, while the MoBSE met the first two DLIs as early as September 2013 and March 2014, by 2016 the funds had yet to be transferred. As a result, the mechanism was restructured, and the associated funds reallocated, with the remaining DLIs becoming IRIs, thus ensuring that the core project activities and sought outcomes remained the same. This adjustment was met with the full support of both the MoFEA and of MoBSE. Revised PDOs and Outcome Targets 32. The PDO remained the same throughout the life of the project as did PDO-level indicators; however, the targets of two PDO-level indicators were adjusted slightly. Due to the higher than expected increase in enrollment in LBS and UBS, the end-of-project outcome targets for the indicators related to increasing enrollment in LBS including ‘Direct project beneficiaries’ and ‘additional students in public ECD, LBS, and UBS’ were increased from 35,000 to 48,000. In addition, the ‘NAT mean score’ target was adjusted downward given the increased enrollment of lower basic students since these additional student enrollments were predominantly from disadvantaged families and previous experience in The Gambia as well as international experience showed that performance on language tests by students with this background is lower, in general. Thus, the target was revised downward from 46 percent to 43 percent to remain conservative in the face of larger than expected numbers of children entering schools. It is important to highlight that despite this slight adjustment to the end-of-project target, both the original and revised targets were exceeded by project closing. 33. Additionally, the wording of two PDO-level indicators was modified to be more precise: ‘Teachers in grades 1-3 showing good teaching performance as observed through the new classroom observation tool’ was modified to ‘Teachers in grades 1-3 from a randomly selected 15 percent of schools, showing good teaching performance as observed through the new classroom observation tool.’ This addition of the sampling method used added important additional details to the indicator. Finally, the indicator ‘Head teachers in lower basic schools adjusting School Development Plans in response to the NAT result and approved by regional directorates using an agreed upon checklist (DLI year 3)’ was modified slightly to omit ‘DLI year 3’ from the indicator given that the DLI mechanism was dropped as mentioned earlier, although the associated activities and outcomes remained identical. Page 16 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 34. To reflect the scale up of activities under the AF, two IRIs were added: (i) Number of additional classrooms built for ECD; and (ii) Pupil textbook ratio of core textbooks in grades 1-9 in public schools. Furthermore, to meet updated World Bank mandated citizen’s engagement guidelines a citizen engagement indicator ‘citizen feedback included in the development of school improvement grant manuals’ was added. Lastly, DLI associated indicators 5 and 8, ‘Average attendance by students in public schools in NAT for grades 3, 5, and 8 for two full days’ were reclassified as IRIs. Revised Components 35. Changes to Components under the AF included the following: Component 1: Increase Access to Basic Education Sub-component 1.3: Community-based early childhood care and development (ECD) 36. The name of this subcomponent was changed to ‘Early Childhood Development’ to reflect the change in the approach under the AF to not only include community-based ECD but also annexed ECD, based on evidence through an impact evaluation (IE) which showed that the annexed model had the greatest impact. The changes to this sub- component were supported by the US$1 million ELP TF grant. Instead of building 40 community-based ECD structures as had been originally planned, the project was modified to support the construction of two multigrade LBS with two classrooms for grades 1–6, an ECD classroom, as well as staff quarters and a toilet block. As the IE had found more positive learning outcomes associated with the annexed ECD approach (attached to LBS), the Government decided to expand the ECD program following this model. 37. In addition, 48 multi-grade schools built under previous projects did not have an ECD classroom attached to them. In support of the Government policy to have three years of formal ECD before LBS, combined with the evidence-based preference for annexed ECD classrooms, the decision was made to attach ECD classrooms to 27 of the 48 multigrade schools (the lower number is based on the total ECD funding provided through the ELP grant) as well as provide furniture, teaching and learning materials, and additional training to ECD teachers. Sub-component 1.4: Provision of school subgrants to support subprojects to all public LBS and UBS and stationery to all students in hardship areas 38. As part of the emergency response to the fiscal crisis in The Gambia, of the US$7.5 million IDA AF, US$2.3 million was used to financially support school subgrants. The READ Project had already been supporting school subgrants for public UBS in Regions 3, 4, 5, and 6. Through the AF, temporary support was provided for school subgrants for all public LBS in Regions 1 through 6, all UBS in Regions 1 and 2, and all public SSS in Regions 1 through 6. The school subgrants were supported for 7 months as part of the temporary emergency funds. Page 17 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Component 2: Improve Quality of Teaching and Learning Sub-component 2.3: Hardship allowances for public lower and upper basic teachers in remote areas 39. As mentioned in the project description, the original project had been financing hardship allowances for public LBS teachers whereas the Government had been financing the hardship allowances for public UBS teachers. During the time when the proposed AF covered the teachers’ salaries and allowances, hardship allowances for public UBS teachers were also to be covered by the AF, after which the Government would then retake over payment of the allowances. Sub-component 2.8: LEARNET (Learning, Education, Activities, and Resources Network), a Public-Private Partnership (PPP) initiative for ICT integration in learning, and teaching, will provide broadband and curriculum-linked dynamic e-content to public schools 40. The name of this sub-component was changed to 'LEARNET will provide curriculum-linked dynamic e-content to public schools and provide teacher training on Progressive Science Initiative and Progressive Math Initiative (PSI-PMI)'. The PSI-PMI pilot program, an innovative ICT program using an interactive board, had been implemented in 24 schools in The Gambia since 2013 and was already supported under the component, and was proving to have a positive impact. At the same time, it was decided that the provision of broadband services to all UBS and SSS would no longer be supported under the project because of the high cost. Originally, it was envisioned that there would be PPPs and the private sector would support this subcomponent, but this did not materialize, given low investment in the country, which was further exacerbated by the political and economic crisis that followed. Estimates obtained through field- based research indicate that the initial setup cost for the 24 PSI-PMI schools close enough to be connected to the pop sites of fiber optics would be approximately US$500,000, and the recurrent cost for broadband connectivity would be approximately US$50,000 for these schools every 10 months. Furthermore, a thorough analysis showed that solar panels in support of the PSI-PMI schools was most effective given a generally highly unstable power grid. 41. As such, keeping in line with the original aim of the sub-component to facilitate ICT integration into teaching and learning, a decision was made to no longer aim to provide broadband connectivity, but rather to scale up the PSI-PMI approach, specifically the development of curriculum-linked digital content of mathematics, science and English, which had not already been covered in the earlier stages of the project, as well as fund solar panels for the PSI-PMI schools. Notably, the digital content was designed to be accessible without broadband access to adapt to the infrastructure challenges and lack of private sector funding. Sub-component 2.9: Teacher salaries for public LBS, UBS and SSS teachers, and allowances for eligible public LBS, UBS, and SSS teachers 42. Sub-component 2.9 was added under the AF to provide emergency assistance. Of the US$7.5 million AF, US$5.2 million was used to financially support all public LBS, UBS and SSS teachers’ salaries and allowances for all public LBS, UBS, and SSS teachers. Allowances supported included the following types: Provincial Allowance, Travel Home to Office Allowance, Civil Servants Special Allowance, Special Skills Allowance, Retention Allowance, Teaching Allowance, Double Shift 50 Percent Allowance, Cost of Living Allowance, and other allowances according to well-defined government Page 18 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) policy. The temporary funding was provided to avoid teachers going unpaid during the fiscal crisis, after which, the Government resumed payment of teacher salaries and allowances in June 2017. Other Changes 43. DLIs and Intermediate Results Indicators. The mechanism for DLIs was dropped under the AF, although the associated outcomes remained the same. In addition, minor changes were made to IRIs. The rationale for adjusting the DLI mechanism is further explained in the following section, and more detail on the changes to the IRIs are documented in Annex 1.B. which shows any changes to targets, wording of indicators, and addition of indicators over the life of the project. 44. Second restructuring. A second restructuring was approved on February 9, 2018, which extended the project closing date to August 31, 2018 (an additional 6 months) to ensure that the final activities, mainly distribution of the textbooks financed under the AF as well as the full handover of the construction of the multigrade schools. The additional months granted through the extension of the project closing date ensured that each of the project activities and outcomes could be fully realized even with slight delays in implementation due to the change in President and changes in management within the PCU. Rationale for Changes and Their Implication on the Original Theory of Change 45. The AF and accompanying restructuring did not have an impact on the Theory of Change because the modifications/additions fit within the nature and scope of the original project design. Even in spite of the fiscal crisis, during the restructuring several outcome indicators were upwardly revised to reflect the higher than expected positive results of the project prior to the AF. 46. The rationale for providing temporary financing for teachers’ salaries, allowances, and school subgrants was to reduce the macroeconomic burden on the MoFEA for a short period and ensure achievements were maintained and subsequently scaled up in the education sector. At the time, The Gambia was facing a fiscal crisis brought on by external shocks including the Ebola Virus Disease and delayed rainy season, as well as significant buildup of public sector debt and overvaluation of the Gambian Dalasi. The AF was developed at the request of the Government to continue to provide adequate service delivery. At the same time, additional funding from the ELP made possible the opportunity to scale up and make evidence-based changes to the ECD approach under the project. Lastly, technology-enabled teaching was reevaluated using a more cost friendly, sustainable model, rather than the original model which had a focus on provision of broadband and a PPP model that was not functioning in the resource constrained context. With the focus still squarely on improving access, quality, and system strengthening, the AF was critical to ensuring adequate education service delivery, while simultaneously making improvements to and scaling up activities where possible based on current evidence, so as to ensure progress towards achievement of each of the sub-objectives. Page 19 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 47. The project objectives were highly relevant and directly corresponded to the key challenges in the education sector in The Gambia. They were aligned with the ESSP and MTP which aimed to achieve universal access and relevant, quality basic education, and improve access to relevant quality secondary education and training. The objectives also directly corresponded to the priorities raised in the JPS (2013-2016), specifically Pillar 2: Strengthening capacity for economic governance and public service delivery. The objectives were meant to directly contribute to strengthening public service delivery by supporting learning in basic education, delivering services to previously out- of-school children, and improving systems within the sector to strengthen service delivery. At the same time, the project was aligned with the GPE strategic plan, particularly the goals to increase the number of children with mastery of numeracy and literacy skills, improve teacher effectiveness, and ensure greater numbers of children, especially girls, would complete primary school and enter secondary school. 48. The objectives and associated activities were backed by extensive data and lessons learned over many years of engagement in The Gambia. With support from the World Bank, the Government’s general approach was to pilot education interventions informed by analytical studies, and to undertake an evaluation of these smaller-scale pilot interventions. This was able to ensure that the project design was based on evidence. Further, this approach facilitated the adoption and scaling up of interventions which had initially proven to be effective on a smaller scale. Some of the activities which followed this approach included: CCTs to Koranic centers; the ECD scheme; multiple early grade reading interventions and the approach to mother tongue reading instruction; and community engagement at the school level. In addition to regular evaluation of interventions, efforts were undertaken to collect critical data, including: extensive school mapping exercises, regular NAT assessments, and rounds of EGRA testing. These evaluations allowed for well-developed targeting approaches, both in terms of ensuring access interventions were appropriate (appropriate supply and demand levers dependent on the context) and targeted the communities in greatest need. 49. The objectives remain consistent with the current Country Engagement Note (CEN 2018-2020) prepared by the Bank in 2018, following the change in President in early 2017, as well as the Systematic Country Diagnostic (SCD) prepared shortly thereafter. Under the CEN, there was a review of achievements under the Second JPS and a focus on three areas of engagement: macroeconomic stabilization, restarting growth, and protecting the poorest from the impact of macroeconomic instability. The READ project’s objectives remained relevant especially with regards to the third focus area, given the Project’s focus on: equitable access to quality learning opportunities through the targeting of rural and poor areas; : improving the quality of education, to ultimately lead to greater earnings; and ensuring more engaged and informed citizens. Page 20 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 50. Relevance of the AF. Under the 2016 AF and accompanying restructuring, though the scope of the operation was expanded, the original design remained largely the same. In addition to supporting ongoing activities, resources from the AF were used to provide temporary emergency assistance in the face of the mounting fiscal crisis and to scale up ECD, the latter with the provision of an additional ECD-specific grant. The temporary support to teachers’ salaries and allowances was instrumental in ensuring the system continued to function in the face of fiscal challenges beyond the control of the MoBSE – specifically ensuring that teachers would continue to be paid and school subgrants would be provided. Ultimately, these measures would ensure that progress that had been realized in the areas of increased access and in terms of learning outcomes up until that point were not lost. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Sub-Objective/Outcome Overall efficacy rating: Substantial 51. The following assessment of the PDOs is organized around three key sub-objectives: (i) increase access to basic education (including ECD, lower basic and upper basic education); (ii) improve quality of teaching and learning in lower basic schools; and (iii) strengthen education systems. For a detailed overview of activities undertaken and progress observed on each of the indicators over the life of the project, please refer to the Key outputs by component section in Annex 1. 52. The first sub-objective of increased access to basic education was achieved through a strategic combination of interventions to address critical challenges related to long distances to school, high private education costs, and need to promote school readiness. These included school construction, regular transportation for early grade children via donkey carts for selected communities, improved ECD teaching and monitoring approaches and expanded ECD offerings to improve school readiness, as well as provision of sub-grants to all lower and upper basic public schools and stationery packages to all LBS in hardship areas which would reduce the financial burden on parents. These interventions were supported by DLIs focusing on ensuring that the reform to abolish school fees at this level was fully implemented and ensuring timely delivery of school grants. 53. Over the life of the project additional 79,541 enrolled in public ECD, LBS and UBS, far exceeding both the original target of 35,000 students and the upwardly revised target of 48,000 students (introduced under the first restructuring). In addition to achieving the target of the PDO-level indicator, all associated IRIs, which measured key outputs to achieve improved enrollment were met or exceeded. This included 84 classrooms built at the primary level as well as 29 ECD classrooms – thus providing 42 additional places; regular provision of school grants which have since been taken over by the Government as part of its sustainability initiative; provision of stationery packages to over 54,000 students; reduction of the number of communities more than 3 kilometers away from a public school (in accordance with government policy) from a baseline of 17 percent (339 communities) to less than 5 percent (97 communities) today; the almost complete removal of informal school fees; and the development of an ECD Page 21 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) monitoring tool. As these measures have been implemented successfully and have resulted in a greater increase in enrolment than expected, the efficacy rating for this sub-objective is High. 54. The second sub-objective of improved quality of teaching and learning in LBS was achieved through a combination of: (i) incentives to increase the number of qualified teachers particularly in difficult to fill subject areas such as math and specific posts (particularly in hardship areas); (ii) enhanced teacher training including reform of pre-service teacher training and improved and expanded in-service support to teachers (both financial incentives, and improved professional development); (iii) targeted approaches to reading, mathematics and foundational skills (with a focus, especially, on early grade reading); (iv) teacher observation/supervision and classroom support to improve quality; (v) innovative technology-based teaching programs such as the PSI-PMI; and (vi) support to Majalis to provide foundational skills to out-of-school children. The diversity of activities reflects the complex context of The Gambia which is characterized by disparities between regions, as well as a diverse education system which encompasses both religious institutions such as Majalis, as well as public schools. 55. Achievement of the sub-objective is evidenced by an improvement of learning outcomes over the life of the project as measured by the PDO-level indicator on mean score in grade 3 English in the national assessment test (NAT), which increased by over 6 percentage points from a 2012 baseline of 41.5 percent to 47.8 percent by project closing (exceeding not only the revised target of 43 but also the original target of 46). This is a significant achievement given the typically slow process of improving learning outcomes and the ambitious target which was set. Despite the known challenges of “moving the needle” on learning outcomes over a relatively short period of time, the Government and partners remained committed to the objective, and gains were anticipated given the strategic combination of improved intensive early literacy programs and an enabling environment (more qualified and better trained teachers, an adequate learning environment, and suitable teaching materials). Indeed, over the life of the project changes to in-service and pre-service teacher training, improved early grading reaching approaches and accompanying policies and training (which included a change in timetable to allow for more reading instruction), as well as technology-based teaching approaches and improved supervision and observation of teachers were introduced which directly supported improved learning outcomes. These are fully detailed in the Key Outputs by Components section of Annex 1. 56. Supporting quality learning metrics (through Component 3) was critical to ensuring the reliability of the learning outcome data used to assess improvements in learning outcomes over the life of the project, and beyond. As such, the NAT was supported under the project, and specifically, linked items were developed with the support of assessment experts so that the rounds of exams were comparable. Improved learning outcomes described above are based on the results of 2012 and 2017 linked tests. Furthermore, DLIs were tied to student attendance for NAT testing, so as to ensure that the assessment was of all students, rather than better performing students. A robust verification method ensured the presence of the majority of children during testing days for an accurate picture of learning outcomes. 57. The PDO-level indicator on teachers in grades 1-3 from randomly selected 15 percent of schools, showing good teaching performance as observed through the new COT was also achieved, reaching 82.7 percent against an end target of 60 percent. The tool was developed to: (i) provide developmental feedback to teachers; (ii) guide supervisors and schools in conducting supervision; and (iii) provide a mechanism for systematic recording and reporting of Page 22 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) teaching quality. After the design and development of the tool under the project, a testing phase was undertaken in 109 schools with 345 teachers’ observations undertaken in August 2014. The tool was then further refined, retested and widely discussed in Coordinating Committee Meetings (CCMs) and within The Gambia College and MoBSE, with a final version in 2015. A two-day training was provided for nearly 2,500 head teachers, deputies, senior teachers and cluster monitors (reaching over 90 percent of head teachers) and then distributed to schools. The five areas of analysis in the COT, rated along a 5-point scale, include: (i) time on task, (ii) preparation and planning; (iii) presentation/lesson delivery; (iv) classroom management; and (v) student engagement in learning. Inconsistencies in scoring practices were addressed through a subsequent training, and final analysis of randomly selected 15 percent of LBS in each region were completed by the SQAD. As there has been an improvement in learning outcomes, and progress observed on each of the quality enhancement measures, the efficacy rating for this sub-objective is Substantial. 58. The third sub-objective of strengthening education systems was achieved through several governance and management reforms supported under Component 4 (Strengthening the Education System with a focus on Governance and Management Results) as well as Component 3 (Technical and Institutional Support). The key aim of governance reform was to ensure evidence-based decision-making and accountability at all levels. As such, an emphasis was placed on utilizing learning outcome data to inform SDPs. This sub-objective was achieved as measured by the indicator ‘Head Teachers in lower basic schools adjusting SDPs in response to NAT results and approved Regional Directorates using an agreed upon checklist’. Achievement of this indicator’s target was culmination of many system-strengthening activities, including: (i) implementation of national assessment tests; (ii) collection and analysis of test results at the central level; (iii) distribution of test results to regional directorates and to schools; (iv) trained head teachers’ review of NAT results to inform SDPs; (v) a functional school development planning process; and (vi) evaluation of the degree to which SDPs were adjusted through a systematic review. These achievements highlight the project’s significant contribution in fostering improved sector management and governance, especially as it relates to collecting high quality data to guide actions aimed at improving the country’s education system. By the project close, the end target of 80 percent of head teachers in lower basic schools adjusting SDPs in response to NAT results and approved Regional Directorates using an agreed upon checklist was exceeded – reaching 95 percent. 59. In addition, despite dropping several DLIs due to challenges with the financing mechanism, all 9 of the DLIs (some of which later became IRIs, rather than linked with financing) were achieved. This included several key governance reforms and system improvements including: timely provision of school facilities; timely provision of school grants; abolished school fees; improvement of data collection; and improved school attendance in NATs. The full set of DLIs and reforms under the project are elaborated further in later sections. As there is clear evidence of capacity development in the management of the sector, and indications that the school development planning process is being widely used as a tool for evidence-based planning, the efficacy rating for this sub-objective is Substantial. Justification of Overall Efficacy Rating 60. The project is rated Substantial in terms of its efficacy, having met all five PDO-level indicators’ targets, and 23 out of 25 intermediate results indicators’ targets. With the PDO-level indicators targets being met, there are more children in LBS, learning outcomes of grade 3 students improved, teachers in grades 1-3 have better teaching Page 23 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) performance, and head teachers understand how to develop SDPs using the NAT results. At the intermediate level, 23 indicators achieved or exceeded their targets. Among others, some examples of significant achievements include: the successful provision of school grants as a way to cover informal school fees; ensuring almost all the students take the NAT where previously head teachers were discouraging low performing students to stay home on the test days; and publishing annual school censes data on time with a response rate among public and private schools of more than 95 percent. 61. Although the targets of two IRIs were not fully met, one almost achieved its target (176 teachers were trained in the newly developed e-curriculum with the target being 200). The reason this target was not fully met is that the DLI funds that were going to be used to develop the e-curriculum were not released by the MoFEA and there was a shortage of funds. However, 176 teachers were more than enough to teach e-curriculum in the 24 schools where the technology has already been equipped. The second IRI for which the target was not fully met was ‘externally reviewed and overseen end-of-year examination in place for students completing the revised programs in The Gambia College’. This is because the firm strongly recommended that the revised curriculum be reviewed after the first one or two years of implementation, after which the end-of-year examination be developed. Therefore, this activity/indicator was transferred to the new project (ESSP, P162890) as an IRI and the firm has recently been engaged in the reviewing the revised curriculum and developing the end-of-year examination. C. EFFICIENCY Assessment of Efficiency and Rating 62. By the end of the project, 100 percent of the project budget was disbursed and a number of efficiency gains had been realized. In addition to efficient implementation of key project activities and smooth disbursement over the project life, unit costs for most project-supported activities were lower than most cost comparators, pointing to high cost effectiveness. The section below summarizes the efficiency gains realized over the life of the project by providing an overall cost-benefit analysis of the project as well as specific unit cost analyses for some of the major project- supported activities. Annex 4 provides the full efficiency analysis carried out for this ICR at project completion. 63. Cost-Benefit Analysis (CBA). Table 4 below compares findings from the cost-benefit analysis carried out during project preparation and from the analysis carried out at project closing. The internal rates of return (IRR) and net present value (NPV) of costs and benefits for all components combined shows that the project was economically viable and sound both at the project preparation stage and project closing based on data from actual number of project beneficiaries. Notwithstanding the adjustments introduced over the life of the project (financing, new activities, etc.), the results from the CBA continue to show that the project remains economically viable and sound. The lower bound of present discounted value of benefits for the overall project is estimated to be US$65.85 million compared with US$58.28 at project preparation stage. The increase is a result of the fact that the AF allowed more teachers and students to benefit from the project specifically from the quality and training activities supported under the project. As such, the present discounted value of costs is estimated to be US$37.93 million (US$25.99 million project cost and US$11.94 million opportunity/running costs) including other unquantifiable portions compared to US$31.5 million (US$19.63 million project cost and US$11.87 million opportunity/running costs) at the project preparation stage. The corresponding NPV of program benefits is US$27.69 million compared with US$26.56 million Page 24 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) at the project preparation stage. The IRR associated with this NPV is 21 percent which is slightly higher that the estimated IRR (20 percent) at project preparation. Therefore, although some benefits cannot be fully quantified in the model, the NPV from the quantifiable benefits is larger than the NPV costs, providing strong evidence of the efficiency of investments undertaken under the project. Table 4: Net Present Value (NPV) in millions of USD and Internal Rate of Return (IRR) (a) At project preparation stage Access & Quality Training Average equity IRR 23% 16% 21% 20% Discounted cost (present value of costs) $10.20 $17.48 $3.82 $31.50 Present value of direct project costs $7.72 $8.09 $3.81 $19.63 Present value of opportunity/other costs $2.48 $9.39 $.01 $11.87 Present value of incremental benefits $32.80 $20.08 $5.41 $58.28 NPV $22.60 $2.60 $1.36 $26.56 Benefit/cost ratio 3.2 1.1 1.4 1.9 (b) Based on actual data at ICR stage Access Quality Training Average & equity IRR 23% 16% 24% 21% Discounted cost (present value of costs) $10.30 $23.81 $3.82 $37.93 Present value of direct project costs $7.77 $14.40 $3.81 $25.99 Present value of opportunity/other costs $2.53 $9.41 $.01 $11.94 Present value of incremental benefits $32.79 $26.94 $6.12 $65.85 NPV $22.49 $3.13 $2.07 $27.69 Benefit/cost ratio 3.2 1.1 1.6 1.7 64. Cost-effectiveness of provision of textbooks and teachers’ guides. Provision of textbooks and teachers’ guides under the Project was cost-effective when evaluated in comparison to figures taken from a number of other countries in Africa as well as those from the final procurement of textbooks and teachers’ guides under the previous project in The Gambia in 2013. The READ Project provided a total of 1,407,558 subject textbooks and 33,436 teachers’ guides for grades 1-10 at a unit cost ranging from US$0.24 (G1 Integrated Social and Environmental Studies) to US$0.57 (G9 Science) for textbooks and from US$0.19 (G1 Integrated Social and Environmental Studies) to US$0.81 (G9 Science) for teachers’ guides. The previous education project in The Gambia had provided a total of 565,058 textbooks and teachers’ guides for grades 5-6 and 180,146 for grades 7-9 at a unit cost of US$0.60 in 2013. In the Democratic Republic of Congo (DRC), the unit cost of textbooks was US$1.327, in Sierra Leone the unit costs ranged from US$0.97 to US$1.66, and in Niger, the unit costs ranged from US$1.10 to US$1.40. The low-cost of textbooks under the READ project was in part attributable to the selected firm’s experience in providing similar textbooks under the previous Bank-financed education project in The Gambia and the larger number of textbooks Page 25 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) and guides procured under the READ project which allowed economies of scale to be realized. As such, the firm had experience producing similar texts – being familiar with the book size, inks, types of paper to use, etc. 65.Cost-effectiveness of classroom construction. The cost of classroom construction per square meter under the READ Project ranged from US$229 to US$276. The READ project used National Competitive Bidding (NCB), as mandated by the World Bank. Even without accounting for inflation between the previous project and READ project, the costs under this Project were highly consistent with those observed under the previous Bank-supported education project. Under the previous project, the cost per square meter ranged from US$191 to US$268 (2012- 2013). It is also important to highlight that The READ project contributed significantly to improved quality standards for school construction. For example, more durable, child friendly and hygienic materials were and are used in construction. Such constructions provide a safer and more hygienic teaching and learning environment and these structures are expected to last for more than 30 years assuming regular maintenance is undertaken. Despite some logistical challenges (e.g., inaccessibility of sites due to the rainy seasons and the large expanse of project sites), 12 contracts were finalized during the project with an average of 368 days between the contract signing date and the provisional hand-over of the sites. 66.Efficient execution of school grants. The project financed school grants in UBS in Regions 3-6 while the government financed such grants in LBS and UBS in Regions 1-2 and all SSS. Under the AF, for one year – the project supported such grants to all LBS, UBS and SSS. Following this, the project supported school grants in UBS in regions 3-6 while the government supported all LBS and UBS in Regions 1-2 and all SSS. The project contributed US$3.3 million to this activity, of which US$1 million was provided by the original project and US$2.3 million from the AF. The Government’s contribution was also significant – the annual cost of providing school grants to LBS in all regions, UBS in Regions 1 and 2, and SSS in all regions together was approximately US$4.3 million, which means that over the course of the project, the Government contributed almost US$13 million to the school grants. Under the project, the MoBSE was able to transfer the first tranche for the school grants to every LBS and UBS by the time school started on September 15 (DLIs 2 and 3). This was a difficult task as it was the first time that school grants were provided to every public school in the country and given that the amount of the grants varied according to school depending on the number of students. The MoBSE also made significant efforts to inform all head teachers and teachers in the country of the school grants, making sure that the eligibility criteria for the grants was well understood (and incorporated in the school grant guidelines). Evidence from the FM audits revealed that school grants were well-executed and used for intended purposes Following project closing, the Government is continuing to provide school grants. 67.Strong Financial Management Performance and Judicious Use of Project Resources. With support and oversight of the FM specialist, fiduciary tasks under this operation were effectively carried out. The Bank also provided significant support to strengthen the Government’s capacity in this key area. Throughout the life of the operation, external annual audits were conducted, which had overall satisfactory ratings with a few suggestions, they revealed no irregularities, and were of unqualified opinion. Interim unaudited financial reports (IFRs) were also submitted in a timely manner and in accordance with Bank procedures. In instances in which minor issues were identified, corrective actions were taken by the Bank and the Government. Available information from the internal and Page 26 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) external audits provides evidence that project resources were used for their intended purposes and that no significant financial leakages or losses had occurred during the project life. 68. In light of the evidence provided above and detailed in Annex 4, efficiency under the project is rated Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 69. The project was highly relevant at appraisal and continues to remain so today. In a context of expanded access, poor retention and poor quality, the project provided necessary inputs to address these key sector challenges. In addition, the project was well-aligned with country priorities and the JPS FY13-16. At project closure, the PDO continued to remain highly relevant to key priorities embedded within Gambia’s ESSP 2018-2030 and the CEN (2018- 2020), particularly with regards to developing human capital through improving access and educational outcomes in an equitable manner. The vast majority of the project’s envisioned outputs and outcomes were achieved, and the entire project funding amount was disbursed. 70. In light of the information provided above and detailed in Annexes 1, 3 and 4, ratings for Relevance of Objectives (High), Efficacy (Substantial), and Efficiency (Substantial), the Overall Outcome Rating for the Project is Satisfactory. E. OTHER OUTCOMES AND IMPACTS Gender 71. Hardship allowances were introduced in 2005 in order to incentivize teachers to take teaching positions in locations that were difficult to reach and fill. Initially, the MoBSE had been trying to post male and female teachers proportionately even in very remote areas. However, in 2014, under the project the MoBSE decided to provide an additional incentive for female teachers in hardship schools. With the hardship allowance, the deployment of teachers is more equitable across non-hardship and hardship schools, and there is some evidence that a greater proportion of female teachers are in hardship areas. At the school level, prior to the project there were more boys than girls in schools, however, with the government efforts to increase girls’ enrollment8, this trend was reversed. Gender Parity Index (GPI) of Primary Completion Rate, UBS Completion Rate and SSS Completion Rate were 1.06, 1.07, and 1.05, respectively, in 2018 (EMIS). Institutional Strengthening 72.Institutional strengthening was critical to project design and implementation as reflected particularly in Components 3 and 4. The focus on improving technical and administrative capacity within the Government, particularly in the areas of assessment, training, and data management and utilization is particularly evident. The PCU, fully integrated within the MoBSE, effectively managed and coordinated the project and given that M&E systems were integrated within preexisting MoBSE systems, the technical capacity to collect, collate, and disseminate data was built within the Ministry itself. The same is true of several interventions, such as the COT, teacher deployment, and many of the teacher training methods which were developed with external support but are now 8 Prior to the fee free education policy, which became effective in 2014 at the UBS level and in 2015 at the SSS level, girls received scholarships to incentivize enrollment and a Re-Entry Policy Program was developed to try to reengage girls who had previously left the system. Page 27 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) wholly managed by the Government. In the case of the innovative PSI-PMI approach, for example, a cadre of trainers has been developed from within The Gambia, so that now they are able to not only scale up the training within the country without additional support, but they are also able to train teachers outside of the country, given the high demand for effective science and math training. Senegal, for example, has requested Gambian expertise in this area given the successes under this project, and the World Bank regional initiative Math and Sciences for Sub Saharan Africa (MS4SSA) has designated The Gambia as a math and science teaching Node. As a Node country, the Gambian trainers have trained trainers and teachers in Rwanda, Niger, and Nigeria. Mobilizing Private Sector Financing N/A Poverty Reduction and Shared Prosperity 73. The provision of additional years of schooling under the project is likely to result in higher wages among beneficiaries in the future. As mentioned in the Annex 4 (Efficiency Analysis), analysis of IHS data found that an additional year of schooling increases projected future earnings. At the national level, an additional year of schooling yields a 6 percent return. In addition, the average estimated earnings increase from GMD2,299 for the working-age population with no formal education to GMD4,581 for those with a post-secondary education level. A high share of wage employment appears to be in the services sector and the poverty rate among wage employees is generally low. In The Gambia, more educated individuals tend to work in the services sector because it offers better economic benefits and formal employment. Only 34 percent of the working-age population with no education works in the services sector whereas 91 percent of the labor force working in the services sector has a post-secondary education. On average, 12 percent of the working-age population has wage-paying jobs; of those, the proportion of the working-age population with a postsecondary education is high at 81 percent. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 74. Evidence-Based Design. The project’s design was technically sound and logically structured across four components, with a balance between traditional IPF financing and a results-based component. The design was developed based on substantial previous experience in The Gambia, as well as extensive data from earlier evaluations and preparatory studies. These included, for example, a study on the impact of hardship allowances, preliminary evaluation results from an ECD impact evaluation, assessment of teacher competency based on NAT exam questions and mastery of the West African Senior School Certificate Examination (WASSCE) materials, and student assessment for those participating in the pilot science and math initiatives. As a result, project sub- components were highly targeted and included a combination of new investments, as well as refined and scaled-up activities based on promising interventions from earlier projects. In particular, lessons learned from the Third Education Project Phase II Project (P077903, 2006-2013, originally US$8 million with an AF of US$5.5 million IDA), and the Education for All Fast Track Initiative (EFA-FTI) Catalytic Fund (CF) Project (P115427, 2009-2013, US$28 million GPE) informed the choice of lending instrument as well as the content design. Implementation for example, had previously been satisfactory, but with persistent delays. Results based financing, in which project disbursement could be tied to satisfactory achievement of DLIs and contribute to system strengthening, was an opportunity to Page 28 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) mitigate previous implementation challenges. Further, content design, specifically building from previously tested activities that tackled disparities related to wealth, a persistent challenge in The Gambia, as well as continuing to improve the data collection – especially the National Assessment Test to ensure tracking of progress over time - were essential elements of the project design. 75. Use of existing structures and highly consultative processes. Strong ownership and commitment on the part of the Government was highly beneficial to project preparation, and later implementation. A highly experienced project coordination unit (PCU), as well as relevant ministry officials took the lead role in preparing the project. In addition, an extremely high degree of consensus building was essential to ensuring stakeholder buy-in across the sector during project preparation. This took the form of the Government’s well-developed practice of holding bimonthly CCMs which rotate regions and discussion circles to allow for honest exchanges between central authorities and local players on education reforms. This was particularly important given the continued efforts to decentralize sector management, as well as the increased accountability at school level as a part of the DLIs. Further, in addition to regular dialogue with key stakeholders such as teachers’ unions, external partners, and others, Joint Donor Sector Reviews were led by the Government and held biannually with development partners. The high degree of commitment and ownership on the part of the Government was key to sustainability measures being built into the project, such as building in M&E processes into the existing Ministry structures, phasing in approaches to government budget management for interventions such as school grants, and long-term effectiveness of school community engagement mechanisms. In addition, there was extensive cooperation between MoBSE and the MoHERST given MoHERST’s role in pre-service teacher training, as well as MoHERST focused capacity-building activities under the project. 76. Assessment of risks and mitigation measures. At the same time, there was due diligence given to risk ratings and realistic assessment of limitations. A high-risk rating was assigned to project design given the partial shift to results-based financing, a first in The Gambia. Further, although the PCU was highly experienced and the MoBSE had been improving capacity, capacity risks also constituted a high-risk rating. This was largely in part to the fact that effectiveness of project activities and their verification hinged partly on high quality verifiable data and synced data sets. The project preparation advance (PPA) of US$1.4 million was strategically used to mitigate these risks through a range of preparatory work, and capacity developed especially in data collection, utilization, and reporting. B. KEY FACTORS DURING IMPLEMENTATION 77. Factors subject to government and/or implementing entities control. Success of the project is due in part to a highly experienced project coordination unit which was well-versed in Bank operations, and, despite the complexity of the project and implementation of the new RBF component, able to effectively manage each of the sub- components. At the same time, a staff transition of the permanent secretary, and later the long-time project coordinator, combined with the country’s economic woes (detailed in the sections below) led to minor implementation delays. An extension of the project by just six months ensured that all activities were completed and was a testament to the absorption capacity of the Ministry, given that the AF was also included in the project timeline. Further, the government and implementing agency were strategic about sustainability measures, exhibiting best practice approaches to project design and implementation by (i) scaling pilots that had been shown Page 29 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) to be effective previously; (ii) building into the budget in a phased in approach various interventions, such as for example the school grants; and (iii) utilizing evaluations to inform policy decisions (e.g., ECD impact evaluation, survey of out-of-school youth, reports on early grade reading approaches, etc.) 78. Factors subject to World Bank control. The project benefitted from consistent World Bank team membership, including the same TTL throughout project implementation. In addition, highly experienced consultants provided technical assistance in the key of assessment, teacher policy, and civil works throughout and were able to leverage deep understanding of the sectoral challenge to guide continued capacity building efforts in these specific domains. The team consistency was highly effective in ensuring continuity of the project operation, and the ability to be reactive in the face of the fiscal crisis to effectively mobilize additional financing, and make minor, but important changes during the restructuring. This, ultimately, helped support the Government in achieving all of the project outcomes and key activities by project closure. 79. Factors outside the control of government and/or implementing entities. The Gambia is particularly vulnerable to external shocks, as was illustrated by the 2014/15 EVD crisis and the delayed rainy season which led to a poor harvest in 2015. Although The Gambia did not have EVD directly within the country, the severe impact on the tourism sector impacted the macroeconomic situation of the country. Further, in 2016 and early 2017 the macroeconomic situation further deteriorated, due to an overvalued exchange rate and rapidly increasing government domestic debt burden, with total debt over 100 percent of GDP by end 2016, and foreign exchange resources exhausted. This led to extremely limited resources for public spending and was a key impetus for the AF in 2016, given that without additional resources, the MoBSE may have been unable to pay teachers’ salaries and school grants to keep the sector functioning. Furthermore, uncertainty in early 2017 around the change of government exacerbated these challenges, as the former president was initially willing to step down. Despite these challenges, save for some minor delays, and the challenges described above with the DLI mechanism, the MoBSE was able to continue its rapid pace of project implementation and disbursement. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 80. The set of PDO-level and intermediate results indicators selected were found to be well-designed and effective in tracking progress towards achievement of the PDO. Each objective had at least one indicator which measured the expected results, and IRIs which measured progress of activities during the implementation period. Under the AF and accompanying restructuring, the results framework was slightly adjusted, with additional indicators added to measure the progress of the ECD scale up and textbook distribution, and indicators were upwardly adjusted where needed to reflect the additional activities. In the case of the PDO-level indicator related to NAT reading scores, the end-of-project target was adjusted slightly downward to reflect that a larger and more heterogenous pool of students would be taking the exam. It is, however, important to note that both the original and revised targets for this indicator were still exceeded. An indicator was not added to measure payment to teachers, as this was done on a Page 30 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) reimbursement basis, thus the required paperwork from the Government to disburse funds via the FM system was sufficient to verify and track this additional funding. 81. Notably, the M&E system was not designed to be stand alone as it was closely aligned to the objectives and indicators of the ESSP and MTP. Thus M&E arrangements for the ESSP within the MoBSE M&E function, which was in a critical coordinating role supported by the EMIS team and the Directorate of Planning of MoBSE was responsible for tracking the indicators and progress. Independent verification for two of the DLIs undertaken was supported through technical assistance and incorporated into the project as part of system strengthening. 82. Improvements to national assessment testing were also extremely important in ensuring the reliability of data and were key to the M&E design. The project supported developing linked items for comparability of the national assessment tests; built into project activities and high stakes indicators around effectively collecting and utilizing learning outcomes data; and put in a place a mechanism to ensure full student participation in the NAT to improve the reliability of results. 83. In addition to the design of field-based surveys as verification of specific DLIS (abolition of school fees, and adjustment of Annual School Development Plans for example), additional surveys and evaluations were built into the project for technical development of the sector and for learning/accountability purposes. These were far ranging, and highly targeted to assess various elements of the interventions, including: (i) follow up data collection for an IE in ECD; (ii) evaluation of the design of the COT; (iii) assessment of reading performance in Majalis via EGRA; (iv) evaluation of the early reading program; and (v) evaluation of the effectiveness of the hardship allowance program. M&E Implementation 84. The data was collected and analyzed in a sound manner utilizing the EMIS, as well as supplementary methods of collection and analysis for certain indicators. By implementing the M&E system via the Government’s own systems of collection, the project continued to build capacity within the Ministry. Monitoring systems were well developed and effective. For example, in the case of tracking teacher attendance, an IRI in the project, and later, under the AF, a secondary means to finalize teacher payment logs, the project was able to tap into and reinforce a multi-layering monitoring system. This included: (a) head teachers mandatory reporting to the regional education directorate (RED) if a teacher does not come to school for three days maximum without authorization; (b) 60 cluster monitors in the country, each in charge of about 12 schools, visiting each school at least twice a month; and (c) head teachers reporting attendance of teachers to the RED everyday using mobile SMS. The tradeoff was that given that the M&E mechanism was not a dedicated standalone system for the project alone, reporting occasionally took more time as it required tapping into multiple systems, and multiple responsible persons to collate all of the data. Thus, advance planning was necessary for up to date and complete biannual reporting. 85. In addition, studies were developed to supplement data in areas where interventions were new, or in new areas, and utilization of additional data such as through IHS. These additional studies included further analysis on early childhood development approaches, early reading approaches, and foundational skills development in Majalis, to allow for midstream adjustments. Indeed, evidence from the results of the early childhood development impact evaluation led to a change in the approach to ECD approaches which focused more on annexed based ECD provision. Additionally, DLI verification protocols were rigorous and detailed, pulling in third party verification where necessary, Page 31 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) especially in the case of adjusting school development plans utilizing NAT results, as well as abolition of school fees. Verification reports were provided on time and found to be consistent with internal MoBSE reporting. M&E Utilization 86. The selected indicators were tracked regularly, and timely updates were provided for all supervision missions, as well as during stakeholder meetings to update the public on progress toward achievement. This led to timely restructuring and additional financing for the project as well as achievement of nearly 100 percent of all targets by project close. The timely, transparent reporting led to early identification of bottlenecks, for example, challenges with textbook delivery due to funds flow issues at the Central Bank, and, solutions to address these bottlenecks. In the case of ECD, as mentioned above, evaluation data resulted in a mid-stream adjustment to the provision of additional ECD to ensure that funds were directed toward interventions with the greatest impact. 87. The well-developed reporting system also allowed for mid-stream supplemental trainings in areas where it was found that additional capacity building was required, especially in the case of newly launched initiatives, to ensure accuracy of reporting. This was true in the case of the COT, which was key to PDO-level indicator “teachers in grades 1-3 showing good teaching performance as observed through the new classroom observation tool.” Designed to capture improvements to teacher performance, it was found mid-stream that additional training was required to ensure that those utilizing the COT were not inflating ratings of teachers due to lack of thorough understanding of how to use the tool. This mid-stream adjustment ultimately led to more reliable data over time. Justification of Overall Rating of Quality of M&E 88. The quality of the M&E system is rated Substantial. The well-developed design, implementation, and utilization of the system allowed for consistent reliable tracking of progress made under the project, including utilizing data analysis to inform government policy decisions (as was the case for the ECD activity) and adjust the project as necessary as part of the additional financing. The integration into the MoBSE M&E system ensured stakeholder buy-in as well as long-term sustainability of the monitoring system and was essential given the large scale of data collected – for example to verify teacher presence at schools via SMS, real-time reporting from regional education directorates on various indicators, and analysis of school improvement grant utilization. Further, capacity building focused on improved monitoring and evaluation and long-lasting improvements to data collection and collation mechanisms, including in large scale national assessments such as the NAT within the government were made explicit in the project design. Other critical examples include integration of HR personnel data under the EMIS which improved the ability to address deployment challenges; as well as carrying out regular census exercises for accurate data on school enrollment to inform policy decisions. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 89. Financial Management (FM). The FM rating was Satisfactory throughout project implementation. Audits were undertaken biannually and were of unqualified opinion as they revealed no significant irregularities. Reporting standards and protocols were adequate and unaudited interim financial reports (IFRs) were submitted in a timely manner and in accordance with well established guidelines and procedures. The rate of disbursement of project funds was high throughout project implementation, reaching nearly 50 percent by the close of year 1 of the project, with only very slight procurement delays related to textbook delivery in the final months of project implementation. In Page 32 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) instances where any minor issues were identified, the Bank worked closely with the PCU and the Government to address them. The FM staff had adequate experience to carry out their required tasks effectively. 90. Procurement. Procurement under the project was sound and is rated Satisfactory. In general, throughout the life of the project, procurement remained on track and moved quickly, while closely following procurement guidelines. There were minor delays in procurement of textbooks in the final phase of the project under the AF due to challenges beyond MoBSE’s control with funds flow through the Central Bank during the fiscal crisis which were rectified before project close. This minor delay in implementation did not impede completion of project activities and the procurement performance continued to be satisfactory throughout. 91. Safeguards. Safeguards compliance is rated Moderately Satisfactory throughout project implementation due to some compliance challenges particularly in the last two years of the project, but fairly good performance overall. The project, a Category B, triggered three safeguard policies: OP/BP 4.01 on Environmental Assessment; OP/BP 4.11 on Physical Cultural Resources; and OP/BP 4.12 Involuntary Resettlement. An Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF), including institutional arrangements and roles and responsibilities for the various stakeholder groups involved as well as implementation and monitoring of mitigation measures was prepared during project preparation. A safeguards focal point in the PCU/MOBSE worked in collaboration with the National Environmental Agency (NEA) to complete the initial safeguards screening under national regulations, and both parties were actively engaged throughout the implementation process. 92. During implementation it was found that Environment and Social Safeguards Checklists (ESSCs) were not always systematically used for site visits and screening before the construction of schools started, and in the final stages, there were generic issues such as (i) hazardous disposal of construction waste; and (ii) improperly designed water towers. The issues were rectified before project closure, and corrective measures had been taken to clean up school sites. Two compensation issues for land donations were also found in the audit, but further dialogue and documentation on the part of communities and the Government resolved these issues. Further capacity building measures, and clearer systems for site reporting were subsequently built into the following project design to holistically address any lapses in safeguards compliance. C. BANK PERFORMANCE Quality at Entry 93. The project was well aligned with the education sector strategy and the country’s plan for national development. By building upon the lessons learned over many years of World Bank engagement in the sector and integrating important studies and analytical work focused on ECD, early grade reading strategies, and assessment standards, among others, the project was developed upon a strong evidence base. This allowed the Project to prioritize those interventions which were likely to have the greatest impact in addressing the identified needs. The PDO was and remains relevant and the results framework was comprehensive and closely linked with the project outcomes and activities. In some cases, it may have been useful to further scale up targets under the additional financing given the overall performance of the project. Page 33 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 94. The Government played a critical role in development and design of the project, and thus, was able to ensure the close alignment between activities to be supported under the project and sector wide priorities. Furthermore, the process was highly consultative via the regular CCMs as well as joint donor sector reviews, and extensive consensus- building with key stakeholders such as teachers’ unions, regional directorates, school communities, and other key administrators, and constituents. Further, close collaboration between MoHERST and MoBSE ensured fluidity of implementation especially in teacher training which is partially managed by MoHERST. In addition, extensive sensitization, training, and capacity assessments were completed prior to project implementation to prepare the implementing agencies and decentralized institutions to effectively manage the results-based financing element of the project which was new for the sector. The Bank rating for Quality at Entry is Satisfactory. Quality of Supervision 95. Supervision missions were undertaken regularly (minimum twice annually, but typically more frequently) to review progress and identify any areas for improvement. Technical assistance (TA) was mobilized as necessary to reinforce the capacity of the Ministry and the PDU, and documentation of progress made was extremely thorough throughout the project, which was critical for addressing any bottlenecks and seeking opportunities for integrating new analysis or additional funding as necessary. The project also greatly benefitted from consistency in TTL-ship throughout project implementation who had deep institutional knowledge of the sector and the Gambian context. An FM specialist conducted regular missions as did various procurement specialists, mostly based in neighboring Senegal. Combined with the diligence and detailed follow up of the task team, there were little to no issues in these areas and DLI verification, as well as flow of funds, control procedures, procurement, planning and budgeting progressed normally. Safeguards compliance was also reviewed regularly, save for some minor turnover in safeguards specialist staff towards the end of the project. The Bank rating for Quality of Supervision is Satisfactory. Justification of Overall Rating of Bank Performance 96. Based on the assessments above the Bank Performance is rated Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 97. Many of the interventions, given they were system wide improvements, are expected to benefit future cohorts of students and teachers with little to no additional funding, while others will require additional investments in the future. The quality interventions for example, are expected to continue to be impactful, given that the focus was primarily on training teachers in best practices for mother tongue reading instruction, and integrating technology into classroom practices, while also benefitting from more regular feedback via the classroom observation tool. At the same time, some continued investment will be required to sustain some of the initiatives, including additional construction to further close the access gap, especially at the ECD level, as well as providing teacher incentives, and scaling up technologically equipped schools. Given that the interventions grew out of the education sector strategy, and that the Government remains committed to their vision, it is expected that even those activities which require additional funding to continue will do so. 98. Furthermore, phased in approaches to government takeover of particular budget items was critical to ensuring the sustainability of outcomes. The Government willingness to couple a fourth results-based financing component focused on governance with the first three traditional investment financing components is evidence of their Page 34 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) commitment to sustainability of interventions. As an example, with the abolishment of school fees, the government after preliminary support via the project is now wholly responsible for school grants which are fully integrated into the Ministry budget. The plan is the same for interventions such as hardship allowances to teachers in future years. Furthermore, capacity building has played a role in each of the components, thus, for example, there is improved capacity for data collection, and data utilization, improved technical capacity to deploy and manage human resources, and at the more micro level, in the case of a number of the teacher training interventions, skilled Gambian trainers who can now internally manage training and/or expand their offerings to neighboring countries such as Senegal. 99. The preference on the part of the government to test approaches, evaluate them, and then scale up as appropriate is also particularly conducive to sustainability. This has been the case for early childhood development, support to Majalis, and the rollout of the best practice reading intervention. It also makes for a smoother transition to nationwide coverage, where applicable, given that lessons learned are integrated into intervention design. DPs such as the Bank and GPE have greatly supported this approach by continuing to build on the tested interventions through subsequent operations. The ongoing World Bank and GPE financed Education Sector Support Program (ESSP) that directly followed the READ operation for example, was based on a thorough review of successes, challenges, and lessons learned from the READ project interventions. 100. Despite these best practice approaches – phased in takeover of budget items by the Government, and the commitment to using evidence to inform intervention design – given low public spending on non-salary expenditures, there are macroeconomic challenges which continue to pose a risk to development outcomes. As such financial and technical support from development partners remains critical to the sector and the sustainability of project outcomes. This was especially evident when temporary relief for salaries was needed during the fiscal crisis, and although steps have been taken at the highest government levels to recover from the macroeconomic slippage, vulnerability to external shocks remains a threat to provision of adequate public services. 101. Despite this challenge, based on the strong government commitment, and current level of financing and support from both the Government and development partners in the country, it is highly likely that develop outcomes will be sustained. Thus, based on the above, the risk to development outcomes is rated low. V. LESSONS AND RECOMMENDATIONS 102. The Bank’s proactive engagement in crisis periods can be instrumental in ensuring that governments have the financial and technical resources to continue providing services when its own capacity and resources are limited. The Bank’s strategic decision to provide additional financing to the Government of The Gambia when it faced a severe fiscal crisis ensured that the achievements observed up until that point could be maintained and subsequently solidified and scaled up in the latter years of the project life. 103. Stability in the composition of government, PCU and Bank teams can facilitate smooth implementation of project activities and the achievement of project development objectives . Throughout most of this project the Government and PCU team remained stable. Although the Project Manager resigned in November 2017, the Deputy Project Manager, who had been working in the PCU since 2007, was immediately able to assume the position as Acting Project Manager. The Bank team was similarly stable. Although the TTL at the time of project preparation had moved shortly Page 35 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) after effectiveness, the subsequent TTL, who remained until the end of the project, had been a member of the preparation team and was very familiar with the project. The stability of the teams assisted in ensuring mutual understanding of the project objectives and consistency of purpose throughout project implementation. 104. A highly participatory process of implementation, including the senior management of the MoBSE, provides important insight into school-level challenges – and can inform decision-making . The MoBSE including the senior management, Minister, Permanent Secretary, Deputy Permanent Secretary and Directors, visited one out of the six Regions every two month for one week (Coordinating Committee Meeting or CCM). This means that every year, they visited each of the six Regions in the country. Each time, approximately 60 people attended a CCM and a group of 4 people spent an entire day at each of the 15 randomly selected schools. They became familiar with both physical and learning challenges by talking to school heads, teachers and students, and each group presented their findings to the CCM the next day. It is expected that the will work to solve the identified issues by the subsequent CCM (in two months). As this procedure has been followed for a number of years, the senior management team is very familiar with the challenges each school is facing and the support that they require. This knowledge can be used to inform decision making related to the school’s most urgent needs, including the need for additional teachers. 105. Piloting activities are a useful approach towards realizing, in the longer term, improvements on a larger scale. As was clear in the ‘READ’ Project, the MoBSE was open to new ideas and willing to test them out in pilot programs. The pilot programs that the MoBSE started during the project included the CCT program in Majalis, ELINL Program, and “Learn by playing” program (tailor made Grade 3 mathematics game). Evaluations of these pilots were important in guiding improvements to their design and providing further justification for and guidance in their scale up. For example, the ELINL program was combined with two other reading programs in the country – to become the integrated “The Gambia READ” program, which has been implemented nationwide since 2016. 106. The establishment of a robust M&E system should always be prioritized as a way to ensure that the achievements and challenges under a project are well-documented and monitored in order to allow Governments and project teams to fine-tune activities in order to enhance their overall effectiveness and impact. The READ project successfully established a sound M&E system and the numerous studies and evaluations which were undertaken during the life of the project were essential in further fine-tuning project-supported activities and introducing new interventions which had demonstrated their effectiveness in improving the overall functioning of the education sector in The Gambia. 107. Donkey carts proved a viable, if unconventional, solution to access to schooling in some areas – highlighting the importance of low cost and innovative approaches in achieving core sector objectives . In order to address barriers to access, the project introduced the provision of donkey carts as a bussing method to bring young children to school who otherwise would not be able to walk the long distance – with the government providing the donkeys and the Bank providing the carts. Further, this approach also had positive externalities in strengthening community engagement and accountability – as each had to feed and care for the donkeys, and drive the children to school, and in return had the use of the donkey for other tasks for the remainder of the day. This was a sustainable solution providing low-cost transport system for some of the most remote villages. This approach has been quite successful and continues today – with 182 carts in operation (from this and the previous project) – bringing 2,000 children to school daily. . Page 36 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improve access to basic education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Additional students in public Number 0.00 35000.00 48000.00 79541.00 ECD, LBS and UBS (Total) 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Exceeded. Baseline (EMIS 2013): Public ECD: 19,590, Public LBS: 184,279, Public UBS: 69,372, Total: 273,241 students. Actual Achieved at Completion (EMIS 2018): Public ECD: 31,849, Public LBS: 245,106, Public UBS: 75,827, Total: 352,782. The difference between the total number of students in 2013 and the number in 2018 is 79,541. Objective/Outcome: Improve quality of teaching and learning in lower basic schools Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Mean scores in (English) in Percentage 41.50 46.00 43.00 47.80 grade 3 in National Assessment Test (NAT) 31-Jul-2012 09-Apr-2014 28-Feb-2018 31-Aug-2018 Page 37 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Comments (achievements against targets): Target revised downward to reflect the increased enrollment rates. However, original and revised targets were both exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers in grades 1-3, from Percentage 0.00 60.00 60.00 82.70 randomly selected 15% of schools, showing good 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 teaching performance as observed through the new classroom observation tool Comments (achievements against targets): SQAD collected and evaluated randomly selected 15% of schools and submitted a report to the Bank confirming that 82.7 percent of teachers had good teaching performance through the new classroom observation tool. Objective/Outcome: Strengthen education systems Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Head Teachers in lower basic Percentage 0.00 80.00 80.00 95.30 schools adjusting School Development Plans in 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 response to NAT results and approved by Regional Directorates using an agreed upon check list Comments (achievements against targets): Trainings on the School Development Plans (SDP) were conducted four times throughout the project Page 38 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) by Standards and Quality Assurance Directorate (SQAD), Assessment Unit and Planning Directorate of MoBSE. 1. In Sep 2015, 800 head teachers and cluster monitors attended the training, 2. In January 2016, 230 head teachers who were not in the first training attended the training with the planning focal points and SQAD representative, 3. In April 2016, the second training was conducted for all head teachers, RED staff and cluster trainers, and 4. In June 2016, 200 head teachers and Planning Directorate's focal points attended a make up training to evaluate the SDP properly. In July 2018, third party consultants evaluated the SDPs and confirmed that 95.3 percent of SDP achieved the results. Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 290000.00 305000.00 330442.00 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Female beneficiaries Percentage 0.00 50.00 50.00 52.00 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Public Lower Basic School (LBS) students: 245,106, Public Upper Basic School (UBS) students: 75,827, Teacher Trainees who receive stipends: 2,458, teachers who are receiving hardship allowance (LBS and UBS): 1,801, and ECD students: 2,250. A.2 Intermediate Results Indicators Component: Increase access to basic education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Stationery packages provided Number 0.00 36500.00 36500.00 54611.00 Page 39 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) to students in lower basic 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 schools in hardship areas Comments (achievements against targets): Achieved (exceeded). Stationery packages were provided to all of the students in public lower basic schools in the designated hardship areas. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of additional Number 0.00 29.00 29.00 29.00 classrooms built for ECD 01-Jun-2016 01-Sep-2016 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. Two out of 29 ECD classrooms were annexed to two multi-grade schools built under the project. Early Learning Partnership (ELP) Trust Fund financed the construction of the 27 ECD classrooms. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of additional Number 0.00 80.00 84.00 84.00 classrooms built or rehabilitated at the primary 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 level resulting from project interventions. Comments (achievements against targets): Achieved. Originally it was planned to build 40 multi-grade schools with two classrooms each. After the ECD impact evaluation, which revealed that children in the Annexed ECD (attached to LBS) learned more. Subsequently, the MoBSE decided to build more multi-grade schools with Annexed ECD, instead of 40 community-based ECD structures. The cost of 40 community- based ECD sheds equaled the two multi-grade schools with two LBS classrooms and one ECD classroom each. Page 40 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Communities with > 3 km to Percentage 17.00 14.00 14.00 4.90 nearest lower basic school in Region 3,4,5,6 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. Communities that do not have LBS within 3 km significantly reduced from 17% to 4.9%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual formal and informal Amount(USD) 12.00 0.00 0.00 0.00 school fees paid by parents to public lower basic schools 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. In 2016, a third party firm conducted the evaluation to see if parents are asked to pay annual formal and informal fees and confirmed that they are not paying fees anymore. A very small proportion (2.9%) of parents paid US$0.3 on average, which means that actual achieved at completion is US$0.009 against US$12 at the baseline. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual formal and informal Amount(USD) 17.00 0.00 0.00 0.00 school fees paid by parents to public upper basic schools 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. In 2016, a third party firm conducted the evaluation to see if parents are asked to pay Page 41 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) annual formal and informal fees and confirmed that they are not paying fees anymore. Only 6.67% of parents paid US$2.7 on average. This means that that actual achieved at completion US$0.18 against US$17 at the baseline. Component: Improve quality of teaching and learning Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pupil textbook ratio of core Number 4.00 1.00 1.00 1.00 textbooks in grades 1-9 in public schools 06-Jul-2016 14-Sep-2016 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. All the core subject textbooks for grades 1-9 were distributed to all the public schools. Textbook to pupil ratio was 1:1 as of project closing. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers recruited or trained Number 0.00 1750.00 1750.00 1906.00 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. This is the number of additional qualified primary teachers resulting from project interventions. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Average teachers attendance Percentage 90.00 95.00 95.00 95.00 Page 42 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) on time 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. Each head teacher has been given a mobile phone and sends a text message with codes that describe teacher attendance. The data were collected at each Regional Education Directorate and analyzed at the HQ MoBSE. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion ECD monitoring tool Yes/No N Y Y Y developed by ECD Unit in MOBSE 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. The ECD monitoring tool was developed and is being used by the ECD unit of MoBSE. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reading curriculum Yes/No N Y Y Y guidelines developed by Directorate of Curriculum at 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 MOBSE based on evaluation results of early reading strategies Comments (achievements against targets): Achieved. There used to be three different reading curricula implemented in different regions of the country. The MoBSE combined them in 2016 and developed "The Gambia Read" program. Subsequently, the new reading curriculum guidelines were developed. Page 43 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Head Teachers skilled to Percentage 0.00 95.00 95.00 100.00 interpret NAT results 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. As the details were described under the comments section of PDO indicator "Head Teachers in lower basic schools adjusting School Development Plans in response to NAT results and approved by Regional Directorates using an agreed upon check list", four trainings were carried out throughout the project cycle for head teachers to obtain skills to interpret NAT results. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion UBS and SSS English teachers Number 0.00 200.00 200.00 176.00 benefited from training programs using the newly 15-Aug-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 developed e-curriculum Comments (achievements against targets): More than 85% achieved (176 of the original 200 teachers were trained). It did not reach 200 due to the limited resources. Component: Technical and Institutional Support Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Externally reviewed and Yes/No N Y Y N Page 44 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) overseen end-of-year 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 examination in place for students completing second year of revised PTC and HTC programs at Gambia College Comments (achievements against targets): Partially achieved. The work started, however, the firm who supported to revise the Gambia College curriculum indicated that the review of the revised curriculum has to be conducted prior to completing the development of the end-of- year examination. Therefore, it was decided that this indicator will be in the new project (Education Sector Support Program, P162890). The same firm has been hired to complete this work now. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Basic Education Sector Yes/No N Y Y Y communication strategy developed by the MoBSE 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. Basic education sector communication strategy was developed by the MoBSE with support from a communication firm. It was reviewed by the WB communication team and revised accordingly. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion HR supplementary data Yes/No N Y Y Y (personnel details, qualifications, promotion and 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 postings history) updated in HR database at Directorate Page 45 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) of HR at MOBSE Comments (achievements against targets): Achieved. All the HR supplementary data have been updated in HR database so now the HR directorate can see each teacher's information in one data file. Component: Strengthening the education system with a focus on governance and management results Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Average NAT attendance of Percentage 83.70 95.00 90.00 91.50 grade 8 students in all public schools for two full days 01-Sep-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved against the revised target. Original target represents a combined average for grade 8, 3 and 5 students before the two indicators were split at restructuring. The revised target of 90 represents Grade 8 only and was achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Average NAT attendance of Percentage 89.10 95.00 95.00 94.60 grade 3 and 5 students in all public schools for two full 01-Sep-2013 09-Apr-2014 28-Feb-2018 31-Aug-2018 days Comments (achievements against targets): Achieved. This was calculated as the total number of students divided by the number of students who participated in the all the subjects of NAT. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 46 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Target Completion Student teachers enrolled in Number 0.00 900.00 900.00 1074.00 revised PTC program 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved. After The Gambia College curriculum was revised with support from the international firm, more than 1,000 students enrolled in the program. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion DLI#2: First tranche (40%) of Yes/No N Y Y Y annual school grants transferred from MOBSE by 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Sept 15 in year 0 to all public lower basic schools (DLI year 0) Comments (achievements against targets): Achieved. The MoBSE submitted the bank transfer record of all the public lower basic schools that showed that school grants were paid by September 15. This was confirmed by the Bank FM team. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion DLI #3: First tranche (40%) of Yes/No N Y Y Y annual school grants transferred from MOBSE by 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Sept 15 in year 1 to all public upper basic schools (DLI year Page 47 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 1) Comments (achievements against targets): Achieved. The MoBSE submitted the bank transfer record of all the public upper basic schools that showed that school grants were paid by September 15. This was confirmed by the Bank FM team. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual training, promotion Yes/No N Y Y Y and postings exercise based on annually updated 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 personnel records in interfaced EMIS and HR systems Comments (achievements against targets): Achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion DLI #6: Annual School Census Yes/No N Y Y Y data collection carried out including Madrassas and 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 min. 95% response rate and results and analytical report released by May 2014 Comments (achievements against targets): Achieved. Annual School Census data were collected in December 2013 with more than 95% response rate and results and analytical report were released in the MoBSE website in May 2014. Page 48 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Citizen feedback was Yes/No N Y Y Y included in the development of school improvement grant 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 manual, measured by minutes of meeting with citizens Comments (achievements against targets): Achieved. During the development of school improvement grant manual, the MoBSE had discussions with schools and communities. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Public upper basic and senior Number 0.00 147.00 24.00 24.00 secondary schools installed with solar power 15-Aug-2013 14-Apr-2014 28-Feb-2018 31-Aug-2018 Comments (achievements against targets): Achieved against revised target and adapted indicator. Originally "public upper basic and senior schools connected to broadband -- changed to "public upper basic and secondary schools installed with solar power" during additional financing/restructuring in 2016. Page 49 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Increase Access to Basic Education (encompasses ECD, lower basic and upper basic education) Outcome Indicators 1. Additional students in public ECD, LBS, and UBS (total) The first objective of increased access to basic education was achieved through a strategic combination of interventions to address critical challenges including distance to school, high private education costs, and school readiness. These included school construction, regular transportation for early grade children via donkey carts for selected communities, improved early childhood development teaching and monitoring approaches and expanded ECD offerings to improve school readiness, as well as provision of sub-grants to all lower and upper basic public schools and stationery packages to all lower basic schools in hardship areas, to remove the financial burden on parents. These interventions were supported by disbursement linked indicators focusing on ensuring that the reform to abolish school fees at this level was fully implemented and ensuring timely delivery of school grants. Over the life of the project an additional 79,541 enrolled in public ECD, LBS and UBS, far exceeding both the original target of 35,000 students and the upwardly revised target of 48,000 students. 1. Number of additional classrooms built or rehabilitated at the primary level resulting from project interventions 2. Communities with > 3 kilometers to nearest lower basic school in region 3, 4, 5, 6 3. Stationery packages provided to students in lower basic schools in hardship areas 4. Number of additional classrooms built for ECD Intermediate Results Indicators 5. Annual formal and informal school fees paid by parents to public lower basic schools 6. Annual formal and informal school fees paid by parents to public upper basic schools 7. First tranche (40%) of annual school grants transferred from MoBSE by Sept 15 in year 0 to all lower basic schools 8. First tranche (40%) of annual school grants transferred from MoBSE by Sept 15 in year 1 to all public upper basic schools Page 50 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Key Outputs by Component (linked to the achievement of the Objective/Outcome 1) Component 1 Under Sub-component 1.1 a total of 84 additional classrooms were built. This includes 42 multigrade schools with water points in remote areas which were identified via school mapping as not having a lower basic school within 3 kilometers. Each completed school is comprised of 2 classrooms, accommodation for two teachers, a water point, and furniture. In addition, 36 urban classrooms which were not completed under the previous project, were fully completed under the READ project. Due to this targeted multigrade classroom construction, in addition to support from other development partners and additional government efforts, the percentage of communities without a lower basic school in regions 3, 4, 5, and 6 within 3 kilometers decreased from 17 percent (339 communities) in 2013 to just 4.9 percent (97 communities) by project close – well exceeding the end target of 14 percent. Under Sub-component 1.2 one hundred customized donkey carts were designed and constructed to provide daily transport for young school age children to and from lower basic schools.9 This built upon the previously developed 82 donkey carts from the previous project, using an improved lighter design with additional safety features – a donkey cart “version 2.0” outfitted with seats for 10 children each. In The Gambian context the donkey carts were found to be a locally appropriate, affordable, school “bussing” type option for children who lived within 3 kilometers of their lower basic schools but still found the walk cumbersome. Under the project, 100 additional communities benefitted from, and continue to benefit from daily transport to and from schools via a volunteer community member who in exchange for caring for the donkey and bussing the children to and from school each day via the cart, can make use of the donkey and cart during the day for other revenue generating activities. The innovative arrangement includes financing for the donkeys by the government and financing via the project funds for development of the carts. Regular supervision of the donkey carts program shows that approximately 2,000 children benefit from daily transport to and from school each day via the originally financed 82 carts and the additional 100 carts as a means toward ensuring regular attendance in lower basic schools. The process entails significant community engagement as well, given it is the community that internally arranges who would be the most appropriate to care for the donkey and transport the children to and from school. Under sub-component 1.3 the development and implementation of an ECD monitoring tool was supported and was finalized by year two of the project. This tool was used for regular monitoring and evaluation of the ECD initiatives within the country and led by the ECD unit. In addition, final analysis of an impact evaluation comparing impact on child development outcomes in both annexed ECD and community-based settings 9A blog to fully describe this innovative, yet simple approach to improved transportation to and from school for early grade children in support of regular school attendance can be found here: http://blogs.worldbank.org/education/how-donkey-school-buses-benefit-early-grade-children-gambia Page 51 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) was supported. Based on this evidence, the project was updated through additional financing combined with a restructuring to reflect the greater impact of the annexed ECD approach and challenges with the community-based approach in previous years under the Japan Social Development Fund (JSDF), which was found to have some design shortcomings. Thus, under the additional financing 29 additional annexed (attached to multigrade schools) classrooms were built. The schools, all in remote areas, were selected based on the size of the population ages 0 to 6, current enrollment of students, and economic disadvantage. Furniture for the classrooms, training of ECD teachers including in-service training, and provision of teaching and learning materials was also supported. Under sub-component 1.4 school improvements grants (SIGs) were provided annually to schools as shown, occasionally with minor delays, but generally on time and in close coordination with communities. This includes LBS (which were fully funded by the government), as well as phased in school grants to UBS, and SSS. School grants were utilized for quality improvement items, and typical expenditure items included stationery, chalk, material for posters, and extra teacher training, as well as grants for learning excursions (i.e. historical sites, the zoo), as well as minor maintenance such as repair of floors or minor water prepares. Schools were precluded from using the grants for major civil works, or additional pay to teachers. The SIGs were found to be critical to the successful to the near abolishment of fees and levies charged previously, as verified utilizing third party verification and in the last two years of the project, under the additional financing, there was a notable shift toward utilizing school improvement grants toward improving NAT scores, as communities because more and more aware of NAT assessments. In addition, under the sub-component 46,731 stationery packages (well exceeding the target of 35,000) were delivered to hardship schools to ease the burden of these additional costs to families in low income rural areas. The school improvement grant aimed to cover recurrent costs in schools –in an effort to reduce informal fees parents had previously been requested to cover. This was accompanied by a communications campaign which conveyed the need for schools to abolish informal fees. An independent review was undertaken in 2016 and collected data from 79 percent of head teachers, 952 students and 263 households. These data provided evidence that some students in a very small percentage of schools (2.9 percent of LBS schools and 6.7 percent of UBS schools) were charged fees. Based on these findings it is clear that the financial burden to families for schooling as significantly been reduced. Component 4. Four disbursement linked indicators were linked to reforms which supported improved access to basic education. This included (i) abolition of school fees for public lower basic school and upper basic schools, and phased in transfer of school grants from MoBSE to lower basic schools and upper basic schools. This method was utilized to ensure reforms were fully carried out which would remove financial access barriers. School grants also supported outcome two (improved quality of teaching and learning which is elaborated more below). Each of the four DLIs were completed under the project. The verification survey by an independent firm completed in January 2017 indicated that 97.1 percent of households at the LBS level did not pay anything, and that 2.9 percent had reported small payments (16GMD was equal to approximately 30 cents) for extra services (classes, food, arts and crafts, etc rather than school fees). The same firm concluded that at UBS level Page 52 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 93.3 percent of parents did not make any payments. 6.67 percent reported making payments, but as was the case for LBS, after further analysis this was linked to extra services at the school rather than formal or informal fees. Thus, the fees were completely abolished and school improvement grants, taken over fully by the government, used as a way to supplement the previous fee system for discretionary spending at the school level. Objective/Outcome 2 Improve quality of teaching and learning in lower basic schools 1. Mean scores of English in grade 3 in National Assessment Test (NAT) Outcome Indicators 2. Teachers in grades 1-3 showing good performance as observed through the new classroom observation tool (COT) Achievement of the objective is reflected through an improvement of learning outcomes over the life of the project as measured by the PDO objective mean score in grade 3 English in the national assessment test (NAT), which increased by over 6 percentage points from a 2012 baseline of 41.5 percent to 47.8 percent (exceeding not only the revised target of 43 but also the original target of 46). The PDO level indicator target teachers in grades 1-3 from randomly selected 15 percent of schools, showing good teaching performance as observed through the new classroom observation tool was also achieved, reaching 82.7 percent against an end target of 60 percent. The tool was developed to (i) provide developmental feedback to teachers; (ii) guide supervisors and schools in conducting supervision; and (iii) provide a mechanism for systematic recording and reporting of teaching quality. Final analysis of randomly selected 15 percent of lower basic schools in each region were completed by the Standards and Quality Assurance Directorate (SQAD). 1. Number of additional qualified primary teachers resulting from project interventions 2. Externally controlled end-of-year examination in place for students completing second year of revised PTC and HTC programs at the Gambia college Intermediate Results Indicators 3. Student teachers enrolled in revised PTC program 4. Reading curriculum guidelines developed by Directorate of Curriculum at MoBSE based on evaluation results of early reading strategies 5. Public upper basic and senior secondary schools installed with solar power Page 53 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 6. Textbook ratio of core textbooks in grades 1-9 in public schools 7. UBS and SSS English teachers who have benefitted from training programs using the newly developed e-curriculum 8. Average NAT attendance of grades 3 and 5 students in all public schools for two full days 9. Average NAT attendance of grade 8 students in all public schools for two full days Key Outputs by Component (linked to the achievement of the Objective/Outcome 2) Component 2 Sub-component 2.1 supported enhanced pre-service teacher training through a review of existing content, a redesigned pre-service teaching program (diploma in education), and the development of an accompanying professional examination. During the project the following resulted: (i) fully redesigned PTC (diploma in education) program; (ii) associated policy changes to support the change in curriculum – including carefully integrating the approach of rotational teaching given that teachers may need to teach a variety of subjects particularly in lower basic schools in rural areas; and the use of local language sin teaching literacy up to grade 3, both of which necessitated a change to the timetable structure. By project end 1,074 student teachers were enrolled in the revised PTC program. There was one related indicator however which was only partially achieved: Externally reviewed and overseen end-of-year examination in place for students completing second year of revised PTC and HTC programs (now called Diploma in Education and Advanced Diploma in Education respectively) at Gambia College. Although the programs were fully revised as described above, and the framework was developed, a full external review of curriculum prior to the putting the examination in place has not yet fully materialized. As a result, completion of this activity will take place under the subsequent Education Sector Support Program (ESSP)(P162890). Sub-component 2.2 supported stipends for student teachers to attract high quality candidates into the field, with additional top ups for students focused on mathematics teaching, given lower numbers of teachers in this subject area. 10,005 (cumulative) numbers of student teachers received stipends, during the life of the project, as part of the aim to develop a strong pipeline of qualified teachers for continued improvement in deployment of qualified teachers throughout the country. Exceeding the target of 1750 teachers, an additional 1,906 qualified primary teachers resulted directly from the project interventions. Page 54 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Sub-component 2.3 Hardship allowances (defined as schools that are more than 3 kilometers from the main road, and an additional “remote status” for schools more than 9 kilometers from the main road) were provided to a total of 8,450 (cumulative) number of teachers in years 2014, 2015, 2016, 2017, and 2018. Hardship allowances to teachers in regions 3-6 has helped to equalize the deployment of qualified teachers throughout the country. Traditionally more remote regions have had higher pupil teacher ratios (PTRs) and a lower proportion of qualified teachers, however, continuation of payment of hardship allowances over the past decade has changed the distribution significantly in every region. For example, over 95 percent of teachers are considered qualified as measured by the government policy on teacher certification. Sub-component 2.4 supported expanded in-service teacher training and was designed to be flexible based on identified gaps throughout the project. 6 number of cohorts of teachers (totaling 17,146 numbers of teachers. Some teachers had more than one training) were trained under the project. Rather than short-term stand -alone teacher trainings, the trainings were designed to take place over several weeks (sometimes up to 70 days) across holiday breaks. To assess improvement, linked pre and post-test were also administered. In this way, there was a clear evaluation mechanism to demonstrate whether content knowledge had improved as a result of training. The table below describes what types of training were planned and actually conducted, when and how long the trainings were for. Table 1: Planned and conducted trainings in the Project Type of Training Number of Number of Year of Training Days of Days of training (actual) Teachers Teachers (actual) training Trained Trained (actual) (planned) (planned) First cohort 900 2892 2013 10 20 days (a total of 130 days but 110 days were covered by the previous project and READ project covered 20 days) Second cohort 1100 974 2014 10 days 70 days Third cohort 1100 1391 2016 10 days 60 days Forth cohort 1100 1200 2018 10 days 20 days under READ MEZZO Maths 0 2900 2016 0 10 days Synchronized EGRA 925 1368 Easter 2015 10 days 10 days 925 732 Summer 2015 10 days 10 days 925 1535 2016 10 days 10 days 925 4154 2018 10 days 19 days Page 55 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Sub-component 2.5 The classroom observation tool was developed in year 1 of the project (validated by the government in March 2015) with support from technical assistance. It was followed by a two-day training on use of the tool to over 90 percent of all head teachers, deputies, and cluster monitors in March 2015, and shortly thereafter distributed to schools. It was used regularly to support teacher supervision and feedback throughout the project, and data during based on performance assessment using the tool was used to inform to the PDO indicator on teacher performance: “teachers in grades 1-3 showing good teaching performance as observed through the new classroom observation tool” which was fully achieved by project close. Sub-component 2.6 The new early literacy program of integrated reading strategies (Gambia READ) was successfully introduced to all public schools beginning in September 2015 in collaboration between expertise in Serholt Early Grade Reading Ability (SEGRA), Jolly Phonics, and Early Literacy in National Language (ELINL) programs. As a first key deliverable, the reading curriculum guidelines were developed and validated. As part of the roll out nearly all schools adopted the new timetable including 60 minutes of reading lessons in national languages and 60 minutes of reading lessons in English daily. In particular, the heavy emphasis on learning to read in mother tongue language as aligned with the Government’s own strategy as well as best practices in reading acquisition. Under the sub-component 3 cohorts of teachers, totaling 3,444 received two-week trainings on the new integrated approach. In addition, readers for every student in grades 1-3 for every national language (Wolof, Fula Mandinka, Serahule, Jola) were produced and distributed, and in the final year of the project supplementary readers were developed in each of the local languages as well. Early evaluations (November 2015) showed that the new reading timetable was not being followed in some schools and trained teachers were not being utilized effectively for reading teaching. As a result, additional technical assistance was sought, and subsequent training held in 2017. Sub-component 2.7 supported monthly CCTs to Majalis (from 2012-2018) on the condition that they release students to participate in foundational courses in mathematics and reading, given that reading and numeracy is not typically taught in this context, with facilitators provided by the MoBSE. The CCTs were provided to 17 Majalis (12 in the primary stage, with an additional five added in 2016) and reached 1500 number of talibehs (students). To assess whether children were learning to read and basic numeracy, the sub-component included a simple diagnostic using an adapted EGRA and EGMA which revealed the overall performance of the children was highly satisfactory as 82 percent of children read the passage and answered all the questions correctly and the mean score of EGMA was 77 percent. Sub-component 2.8 was designed to utilize technology (devices/tools) to improve teaching efficacy and learning outcomes. The initial structure focused on broadband connectivity for public schools, digital resources for curriculum linked content; teacher development using a learning management system and use of e-learning centers; devices focused on teacher-centric solutions; and renewable energy including off-grid solutions for schools. During the AF this sub-component was adjusted to move away from broadband connectivity provision given challenges in Page 56 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) securing the public private partnership to finance these significant investments, and instead, with the focus still on improving teaching through technology centered innovations, the emphasis was shifted to supporting the Progressive Science Initiative-Progress Math Initiative (PSI-PMI). The associated intermediate indicator was changed from public upper basic and senior secondary schools connected to broadband to public upper and senior secondary schools installed with solar power to reflect a more cost-friendly doable approach. Through this intervention by the end of the project (i) 24 schools were outfitted with solar panels (meeting the target of 24); (ii) 30,068 students were taught using the PSI-PMI approach toward better outcomes in science and mathematics; (iii) curriculum-linked digital content in math science and English for grades 7-12 was developed; and (iv) 176 (out of the 200 originally anticipated) teachers were trained in the new e-curriculum approach (intermediate results indicator UBS and SSS teachers benefitted from training programs using the newly developed e-curriculum). The intervention has shown promising results as demonstrated in preliminary results of an impact evaluation. Sub-component 2.9 Under the additional financing, as part of emergency relief, teachers’ salaries of lower basic schools, upper basic schools, and senior secondary schools were reimbursed to the government for the moths of November, and December 2016, as well as January, February, March and a portion of April 2018. The aim was to prevent deterioration of the sector and ensure teacher payment, and by extension teacher presence in classrooms, a key ingredient for classroom learning, even in the face of the pervasive fiscal crisis as earlier explained. Within the same timeframe a validation of civil servants was carried out within the government with support from the governance global practice. Any ghost or absentee teachers were removed from the reimbursement lists and a total of US$7.5 million, as originally intended, was used to reimburse teachers’ salaries. The intervention was successful in ensuring stability, and, because it was reimbursement based with a clear exit strategy, was able to provide temporary relief after which the government fully took over payments again - fully resuming in May without the need for reimbursement. Component 4 included the DLIs (i) Average NAT attendance of grades 3 and 5 students in all public schools for two full days, and (ii) Average NAT attendance of grade 8 students in all public schools for two full days. These two DLIs were critical to ensuring accuracy of national assessment results in response to suggestions that the previous practice in some schools was to have only higher achieving students participate in assessments, thus resulting in skewed results not representative of the total student body. The DLI indicator removed the incentive to skew results, and by close tracking of attendance via a multi-layered verification system, ensured data reliability, thus it supported both the quality improvement objective, and system strengthening objective. Both DLIs were met, reaching the 95 percent participation in NATs. Objective/Outcome 3 Strengthen Education Systems 1. Head teachers in lower basic schools adjusting school development Outcome Indicators plans in response to NAT results and approved by regional directorates using an agreed upon checklist Page 57 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) The key feature of underlying governance reform was ensuring evidence-based decision making and accountability at all levels. As such there was an emphasis placed on utilizing learning outcome data to inform school development plans. The objective was achieved as measured by the indicator Head Teachers in lower basic schools adjusting School Development Plans in response to NAT results and approved Regional Directorates using an agreed upon checklist. Achievement of this indicator was a culmination of many system strengthening activities, as it required (i) implementation of national assessment tests; (ii) collection and analysis of test results at the central level; (iii) distribution of test results to regional directorates and to schools; (iv) trained head teachers review of NAT results to inform school development plans (additional details provided below); (v) a functional school development planning process; and (vi) evaluation of the degree to which school development plans were adjusted through a systematic review. As a result, it points to system strengthening and functioning at multiple levels with the focus squarely on system improvements to support improved learning outcomes. By the project close the end target of 80 percent of head teachers was exceeded and reached 95 percent. Head teachers incorporating National Assessment results in School Development Plans (SDPs). By design, the National Assessment result in The Gambia is used for monitoring the progress of learning outcomes over time at national level, as well as providing information of individual student performance at school level. In the latter, NAT result was often only used to place students on a merit list and was rarely discussed in the SDPs. The training of headteachers on incorporating NAT results in SDPs supported headteachers to interpret the assessment scores in a more meaningful and diagnostic manner. This includes reporting learning outcomes by subject thematic areas (e.g. vocabulary, grammar and comprehension for English), comparing results with regional and national standards, and presenting the analysis in user-friendly graphs to better inform teaching and learning. At the end of the project, more than 95 percent of the schools have included NAT results in the development of their SDPs, and headteachers had improved skills in identifying strength and weakness in teaching and learning and can better target areas in need of improvements. 1. Basic education communication strategy developed by STED 2. Average teachers’ attendance on time 3. Head teachers skilled to interpret NAT results 4. Citizen feedback was included in the development of school Intermediate Results Indicators improvement grant manual, measured by the minutes of meeting with citizens 5. HR supplementary data (personnel details, qualifications, promotion and postings history) updated in HR database at Directorate of HR at MoBSE Page 58 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 6. Annual training, promotion and postings exercise based on annually updated personnel records in interfaced EMIS and HR system 7. Annual school census data collection carried out including madrassas and minimum 95% response rate and results and analytical report released by May 2014 Key Outputs by Component Component 3 Component 3.1 Under sub-component 3.1, with the support of technical assistance, a communications strategy was developed. It was focused on public relations and sharing information and progress with both audiences internal to the government as well as external to the government. Methods for dissemination and engagement included TV, radio, newsletters, and various documentation of Ministry initiatives. Component 3.2 which was designed to support institutional and capacity strengthening resulted in the following: (i) the method for collection of teachers on time attendance was refined and data collection methods improved. Tracking showed that average teachers on time attendance was 95 percent by project close (exceeding the target of 95 percent); (ii) a national qualifications framework for higher education was developed as part of the capacity building support within MoHERST; (iii) WAEC (the regional testing body) was supported in test distribution, and test analysis; (iv) linked items were created in the NAT so that tests would be comparable and valid for drawing conclusions about student performance over time. This was supported through external technical assistance and directly supported analysis of achievement of the core PDO indicator on quality; and also helped identify specific content areas where students were performing poorly. The sub-component was critical in supporting the full chain of evidence-based school improvement approaches by supporting: (i) improved reliability of the NAT assessment and linked NAT testing; (ii) methods for analyzing and distributing NAT testing results; and (iii) training of head teachers to interpret NAT results to inform school improvement plans. By project close 100 percent of head teachers had received training in how to interpret NAT test results, against a target of 95 percent. Improvement of the EMIS, regular payment of PCU staff salaries, and operational costs were also covered as well as various capacity building efforts to support the other two components, as well as citizen engagement. Added under the additional financing, the indicator to incorporate citizen engagement feedback into school improvement grant manuals was added, although this practice had already been taking place through school management committees. As was the case for the other related indicators under component 3, it was fully achieved with participation as verified through school-based committee meeting minutes. Page 59 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Component 4. Under component 4 the following DLIs were achieved which supported the outcome of strengthened education systems. Each of these indicators/DLIs were fully achieved:  Annual school census data collection (including madrassas in accordance with government policy) was carried out with a minimum 95 percent response rate, and results of the analytical report shared online;  HR supplementary data (personnel details, qualifications, promotion, and posting history) was updated in the HR database at the Directorate of MoBSE. This allowed for a clear picture of qualified teachers to address quality and access challenges related to even teacher deployment.  Annual training, promotion and postings exercise based on annually updated personnel records were updated in an interfaced EMIS/HR system so as to improve HR and management of teachers. B. DETAILED RESULTS FRAMEWORK AND MAIN CHANGES Indicator Original (2014) AF/Restructuring (Approved Sept 2016) Restructuring Project NOTES - extension Closing only (Feb (original and 2018) AF) (August 2018) Unit of Original End Results Indicator Change Revised Achievement Overall Measure baseline target as of targets for as of Feb Achievement AF (Sep original 2018 2016) closing date Restructuring of February 2018 Direct Project Beneficiaries Number 0 290,000 300,463 No change 305,000 316,732 330,442 Achieved, Linked to all PDO objectives Female beneficiaries % 0 50 52 No change 50 52 52 Achieved, Linked to all PDO objective Additional students in public ECD, Number 0 35,000 45,702 No change 48,000 64,519 79,541 Achieved, LBS, and UBS (Total) Linked to PDO objective of increasing access Page 60 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Mean scores in English in grade 3 in % 41.5 46 n/a No change 43 47.8 47.8 Achieved, National Assessment Test (NAT) Linked to PDO objective of improving quality Teachers in grades 1-3 showing % 0 60 0 Reworded: 60 82 82.7 Achieved, good performance as observed Teachers in Linked to through the new classroom grades 1-3, from PDO objective observation tool randomly selected of improving 15% of schools, quality of showing good teaching and teaching learning performance as observed through the new classroom observation tool Head teachers in lower basic % 0 80 0 Reworded: (DLI 80 70 95.3 Achieved, schools adjusting School #3) removed Linked to Development Plans in response to PDO objective NAT results and approved by of regional directorates using an strengthening agreed upon checklist (DLI year 3) education system Indicator Original (2014) AF/Restructuring (Approved Sept 2016) Restructuring Project NOTES - extension Closing only (Feb (original and 2018) AF) (August 2018) Unit of Original End Results Indicator Change Revised Achievement Overall Measure baseline target as of targets for as of Feb Achievement AF (Sep original 2018 2016) closing date Restructuring of February 2018 COMPONENT 1 Number of additional classrooms built Number 0 80 80 No change 84 80 84 Achieved, or rehabilitated at the primary level Linked to PDO resulting from project interventions objective of increasing Page 61 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) access Communities with > 3 km to nearest % 17 14 12.5 No change 14 12.5 4.9 Achieved, lower basic school in Region 3, 4, 5, 6 Linked to PDO objective of increasing access Stationery packages provided to Number 0 36,500 46,731 No change 47,000 46,731 54,611 Achieved, students in lower basic schools in Linked to PDO hardship areas objective of increasing access ECD monitoring tool developed by ECD Yes/No no Yes Yes No change Yes Yes Yes Achieved, Unit in MoBSE Linked to PDO objective of improving quality 0 NEW: Number of 29 0 29 Achieved, additional Linked to PDO classrooms built for objective of ECD increasing access COMPONENT 2 Number of additional qualified primary Number 0 1,750 994 No change 1,750 1,906 1,906 Achieved, teachers resulting from project Linked to PDO interventions objective of improving quality Student teachers enrolled in revised Number 0 900 0 No change 900 550 1,074 Achieved, PTC program Linked to PDO objective of strengthening system Page 62 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Indicator Original (2014) AF/Restructuring (Approved Sept 2016) Restructuring Project NOTES - extension Closing only (Feb (original and 2018) AF) (August 2018) Unit of Original End Results Indicator Change Revised Achievement Overall Measure baseline target as of targets for as of Feb Achievement AF (Sep original 2018 2016) closing date Restructuring of February 2018 Externally controlled end-of-year Yes/No No Yes No Slight rewording: Yes No No Partially examination in place for students Externally reviewed achieved, completing second year of revised PTC and overseen end- Linked to PDO and HTC programs at Gambia College of-year examination objective of in place for students strengthening completing second education year of PTC and HTC system programs at Gambia College Head teachers skilled to interpret NAT % 0 95 95 No change 95 95 100 Achieved, results Linked to PDO objective of improving quality of teaching and learning UBS and SSS English teachers benefited Number 0 200 0 No change 200 100 176 Partially from training programs using the newly achieved, developed e-curriculum Linked to PDO objective of improving quality of teaching and learning Page 63 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Reading curriculum guidelines Yes/No No Yes Yes No change Yes Yes Yes Achieved, developed by Directorate of Curriculum Linked to PDO at MoBSE based on evaluation results objective of of early reading strategies improving quality of teaching and learning Indicator Original (2014) AF/Restructuring (Approved Sept 2016) Restructuring Project NOTES - extension Closing only (Feb (original and 2018) AF) (August 2018) Unit of Original End Results Indicator Change Revised Achievement Overall Measure baseline target as of targets for as of Feb Achievement AF (Sep original 2018 2016) closing date Restructuring of February 2018 Public upper basic and senior schools Number 28 147 0 Changed to public 24 24 24 Achieved (after connected to broadband upper basic and restructuring) senior secondary Linked to PDO schools installed objective of with solar power strengthening education system 4 NEW: public 1.00 4 1 Achieved, textbook ratio of Linked to core textbooks in improving grades 1-9 in public quality schools COMPONENT 3 Basic Education communication Achieved, strategy developed by STED Linked to PDO objective of strengthening Yes/No No Yes No education Yes Yes system No change Page 64 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Average teachers’ attendance on time Achieved, Linked to PDO objective of improving quality % 90 95 95 No change 95 95 95 Supplementary data (personnel Achieved, details, qualifications, promotion and Linked to PDO postings history) updated in HR objective of database at Directorate of HR at strengthening MoBSE education No Yes Yes Yes Yes Yes system Yes/No No change Indicator Original (2014) AF/Restructuring (Approved Sept 2016) Restructuring Project NOTES - extension Closing only (Feb (original and 2018) AF) (August 2018) Unit of Original End Results Indicator Change Revised Achievement Overall Measure baseline target as of targets for as of Feb Achievement AF (Sep original 2018 2016) closing date Restructuring of February 2018 Annual training, promotion and Achieved, postings exercise based on annually Linked to PDO updated personnel records in objective of interfaced EMIS and HR system strengthening education No Yes Yes Yes Yes Yes system Yes/No No change NEW: Citizen Achieved, feedback was Linked to PDO included in the objective of development of strengthening school improvement education No grant manual, Yes Yes Yes system measured by Page 65 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) minutes of meeting with citizens COMPONENT 4 DLI#4: Annual formal and informal Amount 12 0 0 The word DLI 0 0 0 Achieved, Linked to school fees paid by parents to public (USD) removed. PDO objective of lower basic schools (DLI year 1) increasing access DLI#7: Annual formal and informal Amount 17 0 0 The word DLI 0 0 0 Achieved, Linked to school fees paid by parents to public (USD) removed. PDO objective of upper basic schools (DLI year 2) increasing access DLI#2: First tranche (40%) of annual Yes/No No Yes Yes No change Yes Yes Yes Achieved, Linked to school grants transferred from MoBSE PDO objective of by Sept 15 in year 0 to all public lower increasing access basic schools (DLI year 0) DLI#3: First tranche (40%) of annual Yes/No No Yes Yes No change Yes Yes Yes Achieved, Linked to school grants transferred from MoBSE PDO objective of by Sept 15 in year 1 to all public upper increasing access basic schools (DLI year 1) DLI #5 and #8: Average attendance by % 88 95 Indicator split into students in public schools in NAT for two sub-indicators grade 3, 5, and 8 for full two days (DLI below for clarity. year 1 and 2) 89.1 92.3 NEW: Average NAT 95 94.6 94.6 (if rounded, (2013 attendance of grade achieved) baseline) 3 and 5 students in Linked to PDO all public schools for objective of two full days increasing access Page 66 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 83.7 87.1 NEW: Average NAT 90 90.57 90.57 Achieved, (2013 attendance of grade Linked to PDO baseline) 8 students in all objective of public schools for increasing access two full days DLI#6: Annual school census data Yes/No No Yes Yes No change Yes Yes Yes Achieved, collection carried out including Linked to POD Madrassas and min. 95% response rate objective of and results and analytical report strengthening released by May 2014 education system Page 67 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Nathalie Lahire Task Team Leader, Senior Education Economist Ryoko Tomita Education Economist Wolfgang M. T. Chadab Senior Finance Officer Rose Mungai Senior Economist/Statistician Tshela Rose-Claire Pakabomba Program Assistant Aisha Rahman Khan Operations Officer Ismaila B. Ceesay Lead Financial Management Specialist Anubha Verma Consultant Daniela Anna B. D. Junqueira Senior Counsel Kebede Feda Consultant (Economic Analysis) Sidy Diop Senior Procurement Specialist Anders Jensen Senior Monitoring & Evaluation Specialist Lalaina Noelinirina Rasoloharison Program Assistant Upulee Iresha Dasanayake Consultant (Environmental and Safeguards Specialist) Ngor Sene Financial Management Specialist Mugambi Mugisha Mwendia Finance Analyst Faly Diallo Finance Officer Supervision/ICR Ryoko Tomita Task Team Leader, Senior Economist Mamata Tiendrebeogo Senior Procurement Specialist Mouhamadou Kabir Ndoye Procurement Specialist Tahirou Kalam Financial Management Specialist Page 68 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Gernot Brodnig Senior Social Specialist Ruma Tavorath Senior Environmental Specialist Josephine Lutta Kiyenje Senior Education Specialist Maiada Mahmoud Abdel Fattah Kassem Financial Officer Frank Anthony Fariello Lead Counsel Christiaan Johannes Nieuwoudt Financial Officer Dilip Kumar Prusty Chinari Financial Analyst Lalaina Noelinirina Rasoloharison Program Assistant Bernardo da Cruz Vasconcellos Education Specialist Laura S. McDonald Operations Officer Alison Marie Grimsland Education Specialist B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 39.442 366,893.18 FY14 31.342 187,472.72 FY15 0 2,362.25 FY17 0 0.00 Total 70.78 556,728.15 Supervision/ICR FY14 0 29,284.45 FY15 42.150 219,105.67 FY16 31.960 361,132.36 FY17 35.473 509,491.64 FY18 35.327 378,848.61 FY19 5.870 41,826.54 Page 69 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Total 150.78 1,539,689.27 Notes: Staff time and cost includes the preparation and supervision of the AF. The Gambia Education Sector Plan Development Grant from GPE in the amount of US$440,250 (activation date: December 5, 2016 and closing date: April 30, 2018) for the new Gambia Education Sector Support Program (ESSP, P162890. AIS sign off: January 9, 2017) was linked to the READ project, as the preparation of the Education Sector Plan takes time and it had to start as soon as possible. This is the major reason that the staff costs (including the consultant and travel time) were high. Page 70 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 3. PROJECT COST BY COMPONENT Table 1: Costs and Expenditure by Component (US$ million) Component Initial Revised Revised Costs Total Cost Actual Actual Expenditure Costs Costs Including Expenditure as a Percentage of Including AF restructuring Total Revised Cost August 2016 February 2018 (no change made) 1. Increase access to basic 6.0 6.0 6.0 6.0 6.0 100 education 2.Improve quality of 6.2 18.0 18.0 17.3 96 teaching and 18.0 learning 3. Technical and Institutional 3.0 1.7 1.7 1.7 2.4 141 Support 4. Strengthening the education system with a focus on 3.6 1.6 1.6 1.6 1.6 100 governance and management results Total Cost 18.8 27.3 27.3 27.3 27.3 100 Page 71 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 4. EFFICIENCY ANALYSIS 1. This section provides the efficiency of investment in education in The Gambia based on key sector indicators and estimates of economic internal rates of returns (IRR) of actual indicators observed at the end of the project implementations. First, an analysis of key sector issues, and the returns to education are provided. This analysis: (i) highlights key issues in the education, (ii) reestablishes the linkages between educational outcomes and the labor market, (iii) analyzes The Gambian’s labor market in terms of signaling education as important tools for better earning based on returns to both individuals and households, and (iv) present key rationale for investing in education in terms of key outcomes on access, equity and private benefits of education. Second, a cost-benefit analysis of the project is presented providing evidence of the efficiency of investments undertaken under the project. Third, a detailed unit cost analysis for some of the major project activities is carried out. Challenges in the education sector 2. The large number of out-of-school children is a major challenge facing the education sector in Gambia with about one-third of primary school-age children (ages 7-12) being out of school while the incidence varies largely across regions and districts. In 2015, out of a total of 330,749 primary school-age children, around 100,000 were out-of-school 95 percent of whom had never been in school. Figure 1 shows the breakdown of the out-of-school incidence by district in LBS, illustrating variations within regions, and districts with out-of-school rates reaching 80 percent. Districts in Regions 1 and 2 have a lower out-of-school rate, although it varies across districts. The disparities between particular districts are significant. For example, the out-of-school rate is 5.8 percent in Banjul City council district in Region 1 while it reaches 80 percent in the Upper Saloum district in Region 5. Overall, there is at least one district in each region with an out-of-school rate below the national average (30.3 percent). Except in Region 1, all regions also have a district with an out-of-school rate above the national average. Page 72 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Figure 1: Out-of-school rate by district at LBS level 80 71 74 68 61 64 55 56 56 57 51 50 51 48 44 43 43 36 37 35 31 33 32 32 29 28 27 26 23 21 23 24 21 18 19 21 18 17 18 10 12 10 10 6 8 8 6 9 Banjul city council Banjul city council Banjul city council Kiang Central Foni Bintang Karanai Sere Kunda Central Illiasa Kantora Bakau Sere Kunda West Foni Kansala Jimara Kombo North Kombo East Lower Niumi Central Badibu Jarra East Nianija Foni Jarrol Foni Brefet Kombo South Foni Bondali Lower Badibu Wuli West Kiang West Kiang East Janjanbureh Niani Jarra Central Niamina East Upper Saloum Sandu New Jeshwang Kombo Central Upper Niumi Sabach Sanjar Jarra West Niamina West Niamina Dankunku Sami Sere Kunda East Lower Fuladu West Lower Saloum Wuli East Tumana Basse Jokadu Upper Fuladu West Region1 Region2 Region3 Region4 Region5 Region6 Source: Authors’ estimations based on IHS 2015 3. While many factors contribute to the out-of-school incidence rates in The Gambia, religion is cited as the main reason for being out-of-school across wealth quintiles, regions, and ethnic groups and varies depending on the level of education of the household head. It is most often cited by the wealthiest households because other reasons are not much of a concern for this and out-of-school rate is low, those located in region 5, the Wolof ethnic group and children coming from households in which the head has no formal education (figure 2). Figure 2: Reasons for out-of-school by wealth quintile, region, ethnicity and head education breakdown by gender and area of residence, LBS (age 7-12) Religion Too Young Costs Quality Work Others 100 80 60 40 20 0 Fula/tukulur/lorobo Some primary Region1 Region2 Region3 Region4 Region5 Region6 Some lower secondary Some upper secondary Mandinka/jahanka Jola/karoninka Others No education Wollof Serahulleh Q1 Q2 Q3 Q4 Q5 Wealth Quintile Region Ethnicity Head educ Source: Authors’ estimations based on IHS 2015 Page 73 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 4. On average, less than half of The Gambia’s working-age population has completed primary education and more than half of the population has no formal education – a concern for future economic growth and global competitiveness. The Gambian working-age population tends to have a lower level of educational attainment. In 2015, the average years of schooling equaled 3.7 at the national level, 5.3 years among the youth population and 1.4 years among the older cohort. Similarly, the distribution of the working population in terms of age group shows that although educational attainment of the labor force in The Gambia is growing, about 58 percent of the working- age population have no formal education. For instance, 34 percent of the youth cohort has not received any formal education - a significantly lower proportion than the adult cohort with no formal education which stands at 87 percent. When the distribution of the working-age population is disaggregated by gender and geographic location, the following are evident: (i) there are more women (63 percent) than men (53 percent) who have not received a formal education. This gender gap is likely going to be a constraint to productivity, trapping a segment of the population in low productivity sectors that do not require a high level of skills, which are usually developed through schooling. To increase economic growth, barriers that prevent adolescent girls and women from going to school need to be addressed; (ii) Rural areas, especially in Region 5, have the highest percentage of working-age people with no education. For example, 81 percent of the population in Region 5 has not received any formal education and 72 percent of the rural population has no education. This suggests that the labor market is likely composed of workers with low skills who have not completed primary education and this increases the likelihood of working in the informal sector which hinders the productivity of the country. Figure 3: Distribution of the Working Population by level of education in terms of age group, gender, area and region (%) No education Some primary Some lower secondary Some upper secondary TVET Some post secondary 100% 80% 60% 40% 20% 0% Female Urban Age 15-24 Age 25-34 Age 35-44 Age 45-54 Age 55-64 Age 15-64 Region1 Region2 Region3 Region4 Region5 Region6 Male Rural Age group Gender Area Region Source: Estimations based on IHS 2015 Returns to education 5. In terms of investments in education, The Gambia’s labor market signals higher returns and better employment opportunities for individuals with a higher level of skills. While education has direct and indirect economic and social impacts, this section only focuses on the economic effects. As such the efficiency of the education sector is measured in terms of the value that education offers though earnings and employment opportunities. For example, the average estimated earnings increase from GMD2299 for the working-age population with no formal education to GMD4581 for those with a post-secondary education (Figure 4). A high share of wage employment appears to be in the services sector and the poverty rate among wage employees is generally low. In The Gambia, more educated individuals tend Page 74 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) to work in the services sector since it offers better economic benefits and formal employment. Only 34 percent of the working-age population with no education work in the services sector whereas 91 percent of the labor force working in the services sector has a post-secondary education. On average, 12 percent of the working-age population have wage-paying jobs, of those, the proportion with a post-secondary education is high at 81 percent. Figure 4: Average monthly earning in GMD (left) and distribution of employment by sector and type (right), by level of education 4,581 91% 81% 3,556 72% 3,260 2,719 3,084 46% 55% 2,299 44% 46% 34% 29% 20% 26% 12% Some lower… Some lower… Some upper… Some upper… Post-secondary Post-secondary No education Average No education Average Some primary Some primary No education Average Some upper Some lower Some primary Post-secondary secondary secondary Average earning Services sector Wage employment Source: Authors’ estimations based on IHS 2015 6. The estimates of the returns on an additional year of schooling broken down by gender and sector of activity are positive and indicate better returns for women and high productivity sectors. At the national level, an additional year of schooling yields a 6 percent return (figure 5). For the same amount of education, women have a higher rate of return (7 percent) than men (5 percent). The rate of returns on education in the services sector is higher than in the other sectors suggesting that productivity in the services sector is higher. Figure 5: Rate of returns on additional years of schooling by gender and employment sector 7% 7% 6% 5% 4% 3% National Male Female Agriculture Industry Services Source: Authors’ estimations based on IHS 2015 7. Higher level of education is also associated with higher rate of returns as the rate of returns on education increases with each successive level of education attained, confirming that education is a key determinant of livelihoods. At the national level, the rate of return ranges from 15 percent for lower basic education to 87 percent for higher education. Higher education is the level that leads to remarkably high returns in all categories: 79 percent for male, 97 percent for female, 100 percent for those that work in the private sector and 101 percent in the services industry (Figure 6). Page 75 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Figure 6: Rate of returns on education by level of education, gender, sector of employment and employer type 97% 100% 101% 94% 87% 88% 79% 76% 71% 73% 68% 63% 61% 53% 56% 48% 49% 44% 45% 42% 40% 39% 35% 34% 34% 35% 31% 30% 30% 20% 20% 21% 20% 15% 13% 14% 14% 4%4% 0% National Male Female Private Public Agriclture Industry Servivces Some primary Some lower secondary Some upper secondary TVET Some post secondary Source: Authors’ estimations based on IHS 2015 8. Lastly, education is a strong predictor of wage employment and employment in more productive sectors. It increases the chances of employment in sectors with high returns and gaining contract employment which offers greater stability (Figure 7). An additional year of education increases the probability of working in wage employment and in non-wage-non-agriculture employment by 37 percent and 21 percent, respectively. Similarly, an additional year of education increases the likelihood of working in industry and services by 20 percent and 31 percent respectively, compared to the agricultural sector. Additionally, differences in employment opportunities are found by gender and the sector of activity. For instance, a man with an additional year of education has a higher likelihood than a woman of finding employment in industry (7 percent more) and services (5 percent more). Relative to working in the private sector, an individual has a 34 percent greater chance of working in the public sector with an additional year of schooling. Women are 10 percent less likely than a man to be employed in the public sector. Page 76 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Figure 7: Probability of employment in better return sector, employment type and employer for additional year of schooling 50% National Male Female 43% 39% 40% 37% 34% 34% 32% 31% 29% 29% 30% 27% 24% 20% 21% 18% 20% 17% 10% 0% Industry services Wage Non wage non agaric Public Source: Authors’ estimations based on IHS 2015 Cost-benefit Analysis Economic Analysis 9. The project was implemented as planned except for a few modifications as described earlier. As such, the CBA consider the steps and assumptions used at the project preparation stage and applies any relevant adjustments that have been made to the project. Key changes to the project include: LEARNET was revised, core subject textbooks were provided, and an AF enabled the project to pay teachers’ salaries and cover the cost of school grants for all public LBS, UBS and SSS for a short period of time. 10. Similar to the project preparation stage CBA, the CBA undertaken for the ICR focuses on the quantifiable benefits and provides the economic rationale for the non-tangible benefit portions using finding from global assessments. The quantifiable benefits of the project are divided into three categories: quality, equity and capacity building, but the CBA only captures the economic benefit stream of the project beneficiaries realized in terms of lifetime earnings as the result of better learning, and intermediate benefits of the interventions. Semi-tangible benefits are benefits that are mostly associated with the quality component of the project and estimated using standard benefit findings from similar interventions in other countries. Hence, the CBA uses three different approaches of benefit stream estimates to align to the project’s design, intervention targets and beneficiaries. 11. The first approach focuses on the impact of interventions on quality improvement. In particular, this examines both the direct and indirect impact of interventions. The impact of direct interventions looks at student learning improvement through, learning material provision, reading and ICT initiatives, and learning assessment, while the indirect channels of student learning improvement include curriculum development, incentivizing teachers in hardship areas, teachers training and ECD development. The analysis builds on various studies showing the relationship between Page 77 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) education interventions and student learning outcomes (mainly test scores) 10 and impact of learning outcomes (particularly cognitive skills) on labor market outcomes (mainly earnings) 11 . The summary results of the relevant interventions to this project are presented below in detail. 12. The second approach of the CBA focuses on quantity and access intervention aspects of the project. This section captures the benefit interventions composed of construction of multi-grade schools, ECD sheds, investment in transportation, and school grants. It is worthwhile to mention that some of the investments affect both the quality and quantity aspects of the beneficiaries’ outcomes and should not be assumed as double counting of the benefit. For example, the benefit of ECD sheds under the quality aspect refers to the benefit of school readiness at the start of primary schooling while the quantity aspect refers to the number of students served by the sheds during the ECD program duration. 13. The third category of the CBA focuses on intermediate benefits occurring through the provision of teacher training and certification, as well as trained opportunities offered for capacity building. In particular, this focuses on interventions associated with The Gambia College for teacher training, in-service teacher training and capacity development in the Ministry. 14. Since each intervention of the project varies in terms of the target population and implementation mechanisms, calculation of the cost-benefit for each approach is done separately. But all three estimates are combined for the cost- benefit estimates based on plausible assumptions, including the costs associated with partially tangible benefits to present the project’s benefits at the lower spectrum.12 Assumptions for CBA 15. To derive the CBA, the model makes several assumptions about the project and the associated costs (opportunity and running costs) and benefits and the assumptions remains the same. It should be also noted that the earning at project preparation stage was estimated based on the 2010 Household Survey and the earning estimate based on the 2015 IHS is slightly higher after inflation adjustment but for efficient comparison project preparation stage, the analysis used the earning estimate from 2010 HHs. As most of the assumptions are common for the three estimation modules, assumptions common to all are presented first and the rest later in the sequence. 10 The first relationship (impact of intervention on leaning outcomes) is based on findings from international expert interview on 40 education interventions. The analysis uses research by Schiefelbein et al. (1998 and 2007) on the impact of education interventions, and research findings on learning outcomes and labor market relationships from developing countries. Although average measures are used to calculate the base scenario estimates, the estimated values can be affected by country-specific contexts, implementation efficiency and the school management system which require alternative scenarios to validate the analysis as robust. For example, Schiefelbein et al.’s study shows that out of the 40 interventions assessed, a divergence of one standard deviation from the mean had an average impact of 10.3 percent on the learning outcome (test score). They further estimated the probability of adequate implementation of the intervention to be at 62.8 percent - leading to an overall actual impact of 6.5 percent (10.3*62.8 percent). Findings from the labor market indicate that one standard deviation from the mean in cognitive skills (learning outcome or test score) results in 0.17 to 0.22 proportional increases in wages. Therefore, the ideal intervention with these profiles increase earnings by 1.1% (6.5 percent*0.17) to 1.4 percent (6.5 percent*0.22). 11 There is body of literature on the impact of cognitive ability (as measured by test scores) on earnings (Patrinos and Sakellariou 2007; Green and Riddell 2001; Moll 1998: Glewwe 1996; Murnane 1995, Jolliffe (1998), Vijverberg (1999), Boissiere, Knight and Sabot (1985); Knight and Sabot (1990), and on average, incremental earning ranges from low 13 percent to 20 percent per one standard deviation o test score. 12 This labeled as partial tangible innervations just to indicate that the model only captures the effect on learning outcomes while in realize the investment on capacity building could have several other impacts including on poverty reduction, health outcomes, fiscal space and other none economic benefits and potential economic benefits accrued other than from cognitive skills. Page 78 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 16. The assumptions relevant to all components: -  Since the analysis takes into account only quantifiable benefits, it is assumed that the internal rate of return (IRR) is the lower bound for the impact of the project.  A 12 percent discount rate is used to derive future returns and for the sensitivity analysis a higher discount rate (15 percent) is considered under different scenarios.  The official exchange rate is set at 33.0 GMD per US$ for this analysis while the latest exchange rate stands at 44.0 GMD per US$ for comparison purpose to project preparation stage.  A wage rate is estimated for different levels of education and age, using IHS 2010 for the benefit and opportunity cost estimate.  The earning rate per level of education and cognitive skills remains the same over the benefits time span except for inflation adjustment.  Average earnings by age grow by 5 percent (annual inflation rate) between the data year (2010) and the project end year and thereafter continue to grow by 5 percent for incremental benefit determination.  Primary school starting age is 6 and ECD starting age is 3.  Unemployment rate by level of education will remain the same although quality improvement might increase the chance of employment to take the lower bound of the benefit. 17. The assumption for quality intervention involves three types of costs:  Project cost of about 66 percent of the total cost is financed by the project and Government contributions.  The direct cost paid by households such as uniforms and transportation. The school fees fully abolished under READ project.  Some costs associated with the running and maintenance of the project during and after project intervention cycles remain the same, so the system upholds to the assumed end of the benefit stream. 18. The benefits for the quality interventions are based on various findings on the relationship of educational interventions and learning outcomes, as well as learning outcomes and labor market outcomes. In particular, as stated above, it uses research by Schiefelbein et al. on the impact of education interventions, and research findings on learning outcomes and labor market relationships from developing countries. However, although average measures are used for the baseline estimate, the estimated values can be affected by country specific context, implementation efficiency and school management system which require several alternative scenarios to be considered as robust. For example, Schiefelbein et al’s study shows, that of the 40 interventions assessed, a divergence of one standard deviation from the mean had an impact on average of 10.3 percent on the learning outcome (test score). They further estimated the probability of adequate implementation of the intervention to be at 62.8 percent - leading to an overall actual impact of 6.5 percent (10.3*62.8 percent). Similarly, findings from the labor market show that one standard deviation from the mean in cognitive skills (learning outcome or test score) results in 0.17 to 0.22 proportional increases in wages 13 . Therefore, the ideal intervention with these profiles increase earnings by 1.1% (6.5 percent*0.17) to 1.4 percent (6.5 percent*0.22). The specific assumptions are as follows:  Take the lower bound of the IRR and impose secondary completion for benefit estimates  Although secondary completion rate can be improved over the project’s life-time, it is assumed as the current rate of 30 percent to take the lower bound of the IRR  The IHS 2010 and EMIS data are used to estimate the number of beneficiaries. 13Based on the meta-study “Returns to Education in Developing Countries” by Patrinos and Psacharopoulous (2010) the average estimated return to a one standard deviation increase in cognitive skills across different countries is between 0.17 and 0.22. Page 79 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079)  Beneficiaries are assumed to be in the labor market for 30 years  Labor market entry age is 18.  Unemployment rate of beneficiaries is estimated from the IHS2010. 19. The assumption for quantity and training related interventions has also three cost categories:  Project costs (about 34 percent of the total cost)  Opportunity cost during training: Since the targeted intermediate and direct beneficiaries are older than 18 years, there will be an opportunity cost associated with program participation and is estimated based on earning levels of subsequent education levels.  Running cost or maintenance costs: there are costs associated to providing services including building maintenance fees throughout its life span. 20. The benefits for this component assume incremental earnings of beneficiary students and the following assumptions are used:  The age of entry into the labor market is 18 and estimate earnings by age and by education level (no education category, grade primary, secondary, TVET and higher education) using HIS 2010 to determine the incremental wage at each educational level attainment after leaving school. An incremental earning benefit is the earning difference between the earning associated to lower level of education and the earnings associated with the education level at the time of school leaving for those in school and training for those upgrading their status.  The probability of employment remains the same and is estimated by level of education taken from the IHS 2010  The beneficiaries of the programs work for a period of 40 years except for those benefiting from different terms for capacity building  The schools provide services for 40 years for the base scenario and reduced to shorter periods for sensitivity analysis, and  Classroom size and student teacher ratio are taken from the Country Status Report (CSR) 2011 21. Cost-Benefit Analysis (CBA). Table 4 below compares findings from the cost-benefit analysis carried out during project preparation and from the analysis carried out at project closing. The internal rates of return (IRR) and net present value (NPV) of costs and benefits for all components combined shows that the project was economically viable and sound both at the project preparation stage and project closing based on data from actual number of project beneficiaries. Notwithstanding the adjustments introduced over the life of the project (financing, new activities, etc.), the results from the CBA continue to show that the project remains economically viable and sound. The lower bound of present discounted value of benefits for the overall project is estimated to be US$65.85 million compared with US$58.28 at project preparation stage. The increase is a result of the fact that the AF allowed more teachers and students to benefit from the project specifically from the quality and training activities supported under the project. As such, the present discounted value of costs is estimated to be US$37.93 million (US$25.99 million project cost and US$11.94 million opportunity/running costs) including other unquantifiable portions compared to US$31.5 million (US$19.63 million project cost and US$11.87 million opportunity/running costs) at the project preparation stage. The corresponding NPV of program benefits is US$27.69 million compared with US$26.56 million at the project preparation stage. The IRR associated with this NPV is 21 percent which is slightly higher that the estimated IRR (20 percent) at project preparation. Therefore, although some benefits cannot be fully quantified in the model, the NPV from the quantifiable benefits is larger than the NPV costs, providing strong evidence of the efficiency of investments undertaken under the project. Page 80 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Table 1: Net Present Value (NPV) in millions of USD and Internal Rate of Return (IRR) (a) At project preparation stage Access & Quality Training Average equity IRR 23% 16% 21% 20% Discounted cost (present value of costs) $10.20 $17.48 $3.82 $31.50 Present value of direct project costs $7.72 $8.09 $3.81 $19.63 Present value of opportunity/other costs $2.48 $9.39 $.01 $11.87 Present value of incremental benefits $32.80 $20.08 $5.41 $58.28 NPV $22.60 $2.60 $1.36 $26.56 Benefit/cost ratio 3.2 1.1 1.4 1.9 (b) Based on actual data at ICR stage Access Quality Training Average & equity IRR 23% 16% 24% 21% Discounted cost (present value of costs) $10.30 $23.81 $3.82 $37.93 Present value of direct project costs $7.77 $14.40 $3.81 $25.99 Present value of opportunity/other costs $2.53 $9.41 $.01 $11.94 Present value of incremental benefits $32.79 $26.94 $6.12 $65.85 NPV $22.49 $3.13 $2.07 $27.69 Benefit/cost ratio 3.2 1.1 1.6 1.7 22. Cost-effectiveness of provision of textbooks and teachers’ guides. Provision of textbooks and teachers’ guides under the Project was cost-effective when evaluated in comparison to figures taken from a number of other countries in Africa as well as those from the final procurement of textbooks and teachers’ guides under the previous project in The Gambia in 2013. The READ Project provided a total of 1,407,558 subject textbooks and 33,436 teachers’ guides for grades 1-10 at a unit cost ranging from US$0.24 (G1 Integrated Social and Environmental Studies) to US$0.57 (G9 Science) for textbooks and from US$0.19 (G1 Integrated Social and Environmental Studies) to US$0.81 (G9 Science) for teachers’ guides. The previous education project in The Gambia had provided a total of 565,058 textbooks and teachers’ guides for grades 5-6 and 180,146 for grades 7-9 at a unit cost of US$0.60 in 2013. In the Democratic Republic of Congo (DRC), the unit cost of textbooks was US$1.327, in Sierra Leone the unit costs ranged from US$0.97 to US$1.66, and in Niger, the unit costs ranged from US$1.10 to US$1.40. The low-cost of textbooks under the READ project was in part attributable to the selected firm’s experience in providing similar textbooks under the previous Bank-financed education project in The Gambia and the larger number of textbooks and guides procured under the READ project which allowed economies of scale to be realized. As such, the firm had experience producing similar texts – being familiar with the book size, inks, types of paper to use, etc. 23. Cost-effectiveness of classroom construction. The cost of classroom construction per square meter under the READ Project ranged from US$229 to US$276. The READ project used National Competitive Bidding (NCB), as mandated by the World Bank. Even without accounting for inflation between the previous project and READ project, the costs under this Project were highly consistent with those observed under the previous Bank-supported education project. Page 81 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Under the previous project, the cost per square meter ranged from US$191 to US$268 (2012-2013). It is also important to highlight that The READ project contributed significantly to improved quality standards for school construction. For example, more durable, child friendly and hygienic materials were and are used in construction. Such constructions provide a safer and more hygienic teaching and learning environment and these structures are expected to last for more than 30 years assuming regular maintenance is undertaken. Despite some logistical challenges (e.g., inaccessibility of sites due to the rainy seasons and the large expanse of project sites, 12 contracts were finalized during the project with an average of 368 days between the contract signing date and the provisional hand-over of the sites. 24. Efficient execution of school grants. The project financed school grants in UBS in Regions 3-6 while the government financed such grants in LBS and UBS in Regions 1-2 and all SSS. Under the AF, for one year – the project supported such grants to all LBS, UBS and SSS. Following this, the project supported school grants in UBS in regions 3-6 while the government supported all LBS and UBS in Regions 1-2 and all SSS. The project contributed US$3.3 million to this activity, of which US$1 million was provided by the original project and US$2.3 million from the AF. The government’s contribution was also significant – the annual cost of providing school grants to LBS in all regions, UBS in Regions 1 and 2, and SSS in all regions together was approximately US$4.3 million, which means that over the course of the project, the Government contributed almost US$13 million to the school grants. Under the project, the MoBSE was able to transfer the first tranche for the school grants to every LBS and UBS by the time school started on September 15 (DLIs 2 and 3). This was a difficult task as it was the first time that school grants were provided to every public school in the country and given that the amount of the grants varied according to school depending on the number of students. The MoBSE also made significant efforts to inform all head teachers and teachers in the country of the school grants, making sure that the eligibility criteria for the grants was well understood (and incorporated in the school grant guidelines). Following project closing, the Government is continuing to provide school grants. 25. Strong Financial Management Performance and Judicious Use of Project Resources. With support and oversight of the FM specialist, fiduciary tasks under this operation were effectively carried out. The Bank also provided significant support to strengthen the Government’s capacity in this key area. Throughout the life of the operation, external annual audits were conducted, which had overall satisfactory ratings with a few suggestions, they revealed no irregularities, and were of unqualified opinion. Interim unaudited financial reports (IFRs) were also submitted in a timely manner and in accordance with Bank procedures. In instances in which minor issues were identified, corrective actions were taken by the Bank and the Government. Available information from the internal and external audits provides evidence that project resources were used for their intended purposes and that no significant financial leakages or losses had occurred during the project life. 26. In light of the analysis presented above, Overall Efficiency under the project is rated Substantial. Page 82 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The following provides a summary of the Government’s completion report for the Results for Education Achievement and Development Project (P133079). The full report can be found in WBdocs. A. RELEVANCE OF PROJECT OBJECTIVES AND DESIGN Relevance of the Project Objectives 1. The project objectives were consistent with the Joint Assistance Strategy (JAS, 2013 – 2016). The READ project directly contributed to strengthening public service delivery by supporting learning in basic education as well as focusing on providing education services to out of school children. In addition, the READ project was fully aligned with both the World Bank Africa Regional Strategy (Africa’s Future and the World Bank’s Support to it) and Education Strategy 202 (Learning for All – Investing in People’s Knowledge and Skills to Promote Development). The project was also aligned with GPE strategic plan. The project further supported the targeted outcomes laid out in the ESSP of components 1, 2 and 5 respectively. 2. The project goals were aligned with the twin goals of the World Bank’s Strategy for poverty reduction and shared prosperity. The READ project aimed was to improve access to education for the poorest children and as well as improve the quality of education through provision of inclusive education and learning for all. The READ project embedded an important international dimension of education by improving the reading skills of children to help reduce poverty and provide opportunities for families to increase the ability of the bottom 40 percent to become empowered and informed citizens. 3. The READ project supported the implementation of the Government ESSPII (2014-2022) and MTP (2014-2017), which was in the tuned of US$18.08 Million investment project with part of it as financing grant in the sum of IDA US$11.9Million and GPE US$6.9Million). The READ project was executed through a traditional IPF approach for components 1 to 3, and results based approach for component 4. The first three components were financed using essential investment activities, advisory, technical, capacity building and M & E support. The results based component which was component 4 only was critically designed to help strengthen the education system because it was geared towards changing critical behaviors that were impeding the education system towards achieving the planned investments that could help impact changes. Component 4 was financed using Eligible Expenditure Program (EEP) and tagged with the education budget line item. An MOU was signed between the MOBSE and MOFEA which helped to specify how the funds were utilized. Additional education policy was supported under the DPO which was aligned with the READ project which helped reduced the burden on household expenditures by providing schools with grants which helped to take care of school fees charged by lower basic schools and the expansion of programs targeted for the poorest or vulnerable children. Page 83 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Relevance of the Project Design 4. The READ Project was designed with considerations of the recommendations highlighted from various studies and survey conducted to determine the actual challenges confronting the education system, especially with regards to access of out of school children in under-served communities and quality of teaching, learning and teachers training program at the Gambia College. The formula used was calculating the IRR and the NPV to determine if the investments will provide the solutions to those challenges highlighted in the ESSP and the studies conducted. Two particular evidences were the studies on ‘Out of School Children’ and ‘School Infrastructure’. Also the survey on the competencies of the teachers which found out that many teachers do not have the competency knowledge to teach in a school warranted the Ministry with its partners to come up with reform of the teaching practice as Gambia College. Also, due to previous experience of project implementation delays, a ‘Result Based’ financing was introduced as a component of the READ project to help address implementation delay and make the Ministry more focused and committed to the implementation of its program. The READ project disbursements were tied to satisfactory achievement of the DLIs. Overall, that design helped strengthened systems and institutional performance. Furthermore, the design of the READ project has contributed to the sustainability of previous implemented projects funded by World Bank and GPE, such as the FTI Catalytic Funded project and the Third Education Project Phase II. The implementation of the READ project was more satisfactory than previous funded WB and GPE projects which further measures were put in place to strengthen implementation arrangements. The guidance of a committed SMT has contributed to the steady incremental of implementation progress. B. PROJECT IMPLEMENTATION AND RESULTS Project Implementation 5. Senior Management Team (SMT) meetings were conducted regularly with an inter-ministerial sittings arranged and an invitation extended to MoFEA for a senior staff to represent the ministry and enhance discussion on the education budget preparation and execution. MOBSE was the main implementer of the project and MoHERST served as coordinator for all activities related to teachers training but on few instances both Ministries carried joint responsibilities for the implementation of the pre-service teachers training. The PCU of MOBSE was entirely responsible for procurement and financial management of the project activities and funds respectively. The M & E unit of MoBSE was tasked with the provision of timely M & E report, which was always prepared and presented to stakeholders including partners. With the support of consultant as construction expert funded by the READ project, the Construction Unit of MoBSE was responsible for the coordination, execution and supervision of the school construction activities. Although among all the critical activities of the READ project, the construction activity has experienced some delay. Among the various consultancies the READ project support was supporting MoBSE to hire the services of an Assessment Specialist who worked closely with WAEC and SQAD directorate to help improved the standards in the schools. The PPARBD Directorate provided the data and served as an oversight body in supporting the consultant with the assessment of examination results and other educational outcomes to arrange for better methodology of improving the quality of assessment, data collection and analysis that would serve as a tool for quality enhance of teaching and learning practices in the schools. One of the requirements of the READ project for effectiveness was the development of a project implementation manual. The Page 84 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) project implementation manual was used by the PCU particularly the Finance and Procurement Units for control of financial budgets, transparency in procurement activities and accountability by the MoBSE leadership for the entire operations of the project. Project Results 6. The ‘READ’ Project has completed and handed over 80 multi-grade schools, rehabilitated significant number of lower basic schools and provided water-points, sanitary facilities to all the new schools. In the rehabilitation package, it has included ramps for the physically challenged to access the facilities with ease. Internal assessment has now included Grade 5 and the attendance rate has been monitored over the period to enable improvements in attendance. This has been achieved over the period by having over 90% attendance rate. The NAT results have been analyzed and are part of EMIS to enable policy makers use the data to monitor learning outcomes, identify changes in learning conditions for students, and helped identified factors contributing to learning achievements. 7. Cumulative targets for access have been met with significant increase in enrolment, transition and completion at all levels. Although the relevance of the curriculum to meet national and global needs has somersaulted due to over two decade of non review and development (revision). This could be one of the major factors affecting learning outcomes/quality education. The education sector has expanded over the years with increased in the number of teachers and classrooms constructed, therefore, policy changes in the areas of decentralization and equity is critical. There should be policy changes in areas of ‘Coordination Mechanism’ to factor the expansion, reforms, and changes in directions (thinking), scope of the strategic plan, partnership and sustainability. The school infrastructure has supplementary values through interventions of other partners, like the EU who is providing solar lighting system to all the currently built and old schools. 8. Improvement in the reading strategies is showing up positively through direct observations of children reading ability and the impact from the NAT, GABECE and WASSCE English examination speaks for itself. 9. EMIS has been strengthened through the provision of statistical data, yearly collection and analysis of data. The EMIS has now been consolidated with HR data which includes the number of qualified teachers per region, sex per region and number of TTs per region. 10. Attendance of teachers has been strengthened through weekly review of teachers’ attendance using mobile phones. This has helped to reduce time lost of pedagogic by prompt replacement of absent teachers or through special arrangements for temporary replacement of displaced teachers. This arrangement and monitoring could help inform policy and regulation. The advantages could vary and not limited to strategic decisions, such as mutual consent placement, priority hiring pool, use of the hiring cycle plan, transfers and termination. 11. Submission of project progress report has been done annually, but it could be strengthened to semi- annually to coincide with the semi-annual joint donor review and supervision mission. The progress report could also serve as a monitoring tool for the quality, inputs of project activities, constraints, challenges, way forward, bottlenecks and overall M & E report of the PDOs. Page 85 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 12. SMT has been strengthened to include ministerial committee (joint SMT) between MoHERST and MoBSE with special invitation extended to the directorate of Budgeting at MoFEA to help in the education sector budget formulation and approval. 13. The project implementation went smoothly without hindrance from the political, social, legal, technological and environmental. Just few cases of economical constraint and two months of political impasse after the legislative election for a President in which the previous president suddenly declined the voting results and called for a ‘fresh election’. But the stand-off was maturely and diplomatically resolved quickly. The government requested for an extension of disbursement. 14. One major achievement of the ‘READ’ is the institutional memory due to leadership and senior management stability. Outputs by Component 15. The ‘READ’ Project has achieved almost 99.8% of its desired outputs based on the numerous activities carried out and the type of inputs gathered and realized on a timely manner. Table 1: Project Components and Associated Outputs Component Outputs Increase access to basic i. Consultancy design reviewed and bidding documents education (Increasing access and ii. Consultancy for boreholes and wells equity) iii. Supervision of works boreholes & wells iv. Furniture for multi-grade schools v. Furniture for ECD Sheds vi. Complete mult-grades schools vii. Multi-grade schools viii. Borehole solar pump & reservoir ix. Well with hand pump x. ECD shed xi. School Transport – Donkey Carts xii. Reviewed design xiii. Fabrication of Carts xiv. School Grants for UBS xv. Grants to UBS (77 UBS and BCS) xvi. Provision of stationary to LBS hardship areas xvii. Procured School stationary (exercise books, pencils, etc.) Improve Quality of Teaching & i. Assessment Learning ii. EGRA iii. NAT 3 and 5 iv. NAT 8 v. EGMA vi. Conditional Cash Transfer to Majalis Page 86 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) vii. CCT Initiative Students viii. Facilitators ix. Evaluation of the ongoing program x. Continuous professional development for LBS xi. Printing of Modules xii. Vehicles xiii. Training of LBS teachers, fuel etc. xiv. Development of Modules xv. Development of assessment data analysis xvi. Support for reading program G1 to 3 xvii. Training on Early Grade Reading (food, transport, etc.) xviii. Develop English Language Materials G1 – 3 xix. Printing of Pocket Readers xx. Vehicle xxi. Maintenance xxii. Impact Evaluation Study xxiii. Support to National Languages xxiv. Training of teacher for the implementation of the reading strategy xxv. Revision of scripted lessons assessment xxvi. Supervision of coaches and data etc. xxvii. Printing of scripted Lessons xxviii. Stationery xxix. Vehicles xxx. Hardship allowances for LBS xxxi. Allowance for teachers of 3 – 8km LBS xxxii. Hardship allowance for LBS teachers of 9km and above xxxiii. Data collection and monitoring xxxiv. Monitoring of teachers’ attendance xxxv. Data collection using mobile phones xxxvi. Training of mobile phone users (teachers) xxxvii. Procured Mobile Phones xxxviii. Support to access to Broadband xxxix. Procured Laptops xl. Develop content xli. School level monitoring cluster RED & HQ xlii. Vehicles (Replaced old ones) xliii. Fuel & Insurance for vehicles and motorcycles xliv. TA for development & refine, classroom observation tool xlv. Quarterly monitoring and supervision for RED & HQ xlvi. Piloting & validation of classroom observation tool xlvii. Training of head teachers xlviii. Training of Cluster Monitors, SQAD Page 87 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) xlix. Continuous monitoring by cluster monitors l. Annual Impact Evaluation of classroom Observation tool li. Support to ECD program lii. Provision of teaching and learning materials liii. Procured Laptops liv. Office Furniture and Equipment lv. Printing of ECD Content Materials lvi. Supervision & Monitoring of ECD program lvii. Orientation of Gambia College lecturers on the new ECD curriculum lviii. Monitoring and Supervision of ECD centres lix. Training of ECD Facilitators lx. TA Review & Development of ECD Curriculum lxi. TA Development of 2nd level Goal PPHRC lxii. Impact Evaluation Study lxiii. Study tours to Kenya University, etc. lxiv. Support to WAEC lxv. Procured printing equipment for WAEC lxvi. Procured laptops for WAEC lxvii. Procured Vehicles for WAEC lxviii. TA for WAEC lxix. Support to curriculum improvement lxx. Curriculum development and review textbooks lxxi. Curriculum audit and research lxxii. Support to GC teachers training lxxiii. Printing of teaching and learning materials lxxiv. Procured laptops lxxv. Procured photocopiers lxxvi. Vehicles lxxvii. Maintenance lxxviii. TA Review existing content etc lxxix. TA Development on ICT competency, etc lxxx. Teacher Professional Examination lxxxi. Teacher supervision and monitoring lxxxii. Incorporation of ECD teachers training lxxxiii. Training of Teacher Trainers lxxxiv. Testing & ICT Certification lxxxv. Stipends for PTC 3 lxxxvi. Stipends for HTC 1 to 3 lxxxvii. Stipends for HTC 1 – 3 Maths Special lxxxviii. TA support training teachers on Reading lxxxix. Publishing textbooks xc. Publish G1 – 4 textbooks and teachers guide xci. Publish G5 – 6 textbooks & teachers guide Page 88 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) xcii. Reprint G7 – 8 English & Maths textbooks, etc xciii. Reprint G7 – 9 science and SES textbooks, etc. xciv. Procured SSS textbooks xcv. Procured supplementary reading materials xcvi. Procured library materials Technical and Institutional i. Support to data systems, EMIS, HR data Support (Improving Sector ii. Development and implementation of communication Management) strategy, etc. iii. Annual data collection iv. Strengthen management and monitoring, etc v. Support to school mapping vi. PCU operating costs vii. Salaries to project management staff and technical staff, etc viii. Support to intra and internet and networking system ix. Procured computers and accessories x. Procured printers and toners xi. Procured furniture and fixtures xii. Procured office provisions (tea, coffee, etc.) xiii. Office telephone services and internet services xiv. Maintenance of vehicle and motorcycles xv. Fuel xvi. Building statistical capacity of MoBSE and RED xvii. Long-term training (1 year) xviii. Workshops xix. Short-term training (1 – 3 weeks) xx. Development of safeguard framework (ESMF RFF) xxi. Environmental safeguard and management framework xxii. Resettlement policy frameworks xxiii. Short-term training, and environmental management Strengthening the Education i. Consumable for RED & HQ System with a focus on ii. Procured photocopiers governance and management iii. Maintenance of Photocopiers results iv. Procured Digital Cameras v. Procured Laptops vi. IT consumables, including printers vii. Project Management Software viii. Development and distribution of MoBSE Newsletter ix. Printing of MoBSE Newsletters x. Audit xi. TA to MoHERST xii. Consultancy services NQF xiii. Strengthening Education Systems (DLIs) Page 89 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Table 2: Level of Achievement on Project Intermediate Results Indicators Output Indicators Status 1. Increased access to - Stationery packages provided to students in Achieved and improved equity lower basic schools n hardship areas. within basic, - Number of additional classrooms built, or Achieved secondary, technical rehabilitated at the primary level resulting and vocational from project interventions. education and - Communities with >3 kilometers to nearest Achieved training, tertiary and lower basic school Region 3, 4, 5 and 6. higher education achieved. 2. Improved delivery of - Number of additional qualified primary Achieved quality and relevance teachers resulting from project interventions. of basic, secondary, - Student teachers enrolled in revised PTC Achieved technical and program. vocational education - Externally controlled end of year examination Achieved and training, tertiary in place for students completing second year and higher education of revised PTC and HTC programs at GC. achieved. - Average teachers’ attendance on time. Achieved - ECD monitoring tool developed by ECD Unit of Achieved MoBSE. - Public Upper Basic and Senior Secondary Not achieved Schools connected to broadband. - Reading curriculum guidelines developed by Achieved Directorate of Curriculum at MoBSE based on evaluation results of early reading strategies. - Head Teachers skilled to interpret NAT Results. Achieved 3. Research in the critical - Annual formal and informal school fees paid Achieved fields of development, by parents to public lower basic schools. mainly health, - UBS and SSS English teachers benefitted from Achieved agriculture, basic training programs using the newly developed sciences and human e-curriculum. resource development - Basic Education (MoBSE) Communication Achieved and management Strategy developed by STED. promoted and - HR supplementary data (personnel details, Achieved strengthened qualifications, promotion and postings history) updated in HR database at Directorate of HR at MoBSE. - STI National process as envisaged in The Achieved Gambia’s development plan to ensure sustainability of all STI interventions harmonized, coordinated and integrated. 4. Sector Management - Effective and Efficient delivery of education Achieved and training services. Page 90 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) 16. Cumulative targets for access have been met with a significant increase in enrollment, transition and completion rates at all levels. The quality of the teaching and learning environment has also significantly improved. Table 3. Level of Achievement on PDO-level Indicators Type Objective Indicator Results PDO Indicator 1 Project Beneficiaries Direct project Direct project (Core Sector Indicator) beneficiaries (number), beneficiaries: of which female 330,442Female (percentage) Percentage: 52% PDO Indicator 2 Increase Access to Basic Additional students in Public ECD: 12,259 Education including public ECD, Lower Basic Lower Basic ECD Schools (LBS) and Upper Schools: 68,827 Basic Schools (UBS Upper Basic number) Schools: 6,455 PDO Indicator 3 Improve Learning Mean Scores in Grade 3 Mean Scores in NAT Outcomes English) Grade 3 English: 47.8% PDO Indicator 4 Improve Quality of Teachers in grades 1 – 3 A survey has Teaching showing good teaching already been performance as conducted by the demonstrated through Standards and the new classroom Quality Assurance observation tool (%) Directorate on the indicator. The report is currently being finalized for dissemination. PDO Indicator 5 System Strengthening Head Teachers in Lower 95% of Head Basic Schools adjusting Teachers in the School Development Lower Basic Schools Plans in response to NAT have adjusted their results and approved by Development plans Regional Directorates in response to NAT using an agreed upon results and it was checklist (%) approved by RED. C. SUSTAINABILITY OF PROJECT SUPPORTED ACTIVITIES AND INTERVENTIONS 17. The likelihood of sustainability of the ‘READ’ project and beyond the ‘READ’ is premised on four factors, which are: (a) strong commitment of government as demonstrated that education is a key priority; (b) government took a leading role in preparing the project with support from World Bank, GPE and other bilateral partners; (c) organization of bi-monthly CCM and other coordination Page 91 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) mechanisms; (d) development, implementation and coordination of a communication strategy to support internal and external elements as well as attempt to build and improve upon strategic partnerships with external players facilitating broad ownership for sustaining reforms. 18. Nonetheless, sustainability as mentioned earlier, the READ and beyond READ can be realized through at least ten cardinal principles (1) Access – construction of child friendly and excellent facilities; (2) Relevance – curriculum reform, review and development in line with national and global education, knowledge and skills labor needs; (3) Equity – decentralization of the education system with planning and budgeting provided by REDs; (4) Quality – placement of quality teachers at both urban and rural areas, provision of quality teaching and learning materials (aid); (5) Public Consultation and Engagement – frequent holding of public debates like Bantaba, Coordinating, Management, Review and Supervision gathering; (6) Sound Policy and Realization – realistic thinking (in and out of the box) to help turn key within a short span and achieve the desired results within cost and quality not compromised; (7) Innovative Strategies and Impact – using or harnessing technology and research, especially action research into teaching and learning curriculum; (8) Results Based Budgeting – provide task on budget to enhance more committed delivery of services, works and goods, transparency and accountability of results; (9) Community Empowerment and Partnership Trust – strategic partnership with communities, NGOs and Philanthropies to facilitate ownership of the school management; and (10) Government and Leadership Strong Commitment – government timely and progressive released of funds from the national budget, i.e. 20+% of the EFA benchmark to education, and country leadership efforts and interest in provision of quality and relevant education for all. D. Lessons Learned and Recommendations i. All the stakeholders including the Local Education Group (LEG) were fully involved in the preparation of the project. Per the GPE requirements, they were involved in every step of the project preparation, including the Concept Note Review, Decision Review and Appraisal, and endorsed the project documentation at each stage. During the implementation and supervision of the project, they were also actively engaged through the bi-monthly CCMs and monthly LEG meetings. This mechanism has worked very well and the stakeholders had a very good understanding of the project and how it was supporting the sector. As such, the full engagement of stakeholders should continue in the new project. i. The principle and goals of DLIs were aligned with the sector policy. It was the first time for the education sector to use the DLIs and all the relevant officials and PCU staff worked very hard as a team. As a result, the sector was able to improve its capacity. However, a fiscal crisis meant that DLI funds were received but did not flow to the education sector, which constrained the capacity to achieve the objectives of the project. To resolve the issues, 5 out of 9 DLIs were reclassified as intermediate results indicators so that these activities could be funded through the traditional investment approach. For future projects, the funding mechanism of the DLIs should be reviewed to ensure that there is a robust mechanism to ensure the flow of funds to the sector. ii. An evaluation of the impact of the school grants activity should be conducted. The school grant has been a mechanism to provide recurrent funding to schools, and has been a Page 92 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) mechanism to remove informal school fees. However, calibrating the correct size of the school grant is a challenge. Schools need more resources and seek an increase in the grant amount. However, there is limited budgetary space to finance larger grants in a sustainable manner. A review of the scale of school grants may be needed to help the Ministry make informed decisions in this regard. iii. The intensive teacher trainings have been successfully conducted. Some improvements of teachers’ content knowledge and pedagogy skills as well as students’ learning outcomes were seen. However, there is still room for improvement for both. In the new project, the Government plans to develop a comprehensive and strategic teacher training framework for all types of training from pre-service to in-service along a continuum. In addition, it will be important to review the quality of pre-service and in-service trainings and the impact of teacher training on classroom teaching. Page 93 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) ANNEX 6. SUPPORTING DOCUMENTS Project Documents - Project Appraisal Document, February 10, 2014. Report No. 83958-GM - IDA Grant Agreement, April 9, 2014 - GPE Grant Agreement, April 9, 2014 - Project Paper, Additional Financing, August 18, 2016. Report No. PAD1954 - Agreement Providing for the Amendment and Restatement of the Financing Agreement, October 8, 2016 - Agreement Providing for the Amendment and Restatement of the GPE Grant Agreement, October 8, 2016 - ELP Grant Agreement, October 8, 2016 - Restructuring Paper, January 19, 2018, Report No. RES21576 Implementation Status and Results Reports - ISR Sequence 1-10 Aide Memoires - AM April 2013 - AM August 2013 - AM June 2014 - AM December 2014 - AM August 2015 - AM December 2015 - AM February 2016 - AM July 2016 - AM March 2017 - AM June 2017 - AM December 2017 - AM May 2018 - AM August 2018 Page 94 of 95 The World Bank GAMBIA - READ: Results for Education Achievement and Development Project (P133079) Additional Documents - The Gambia: Education Sector Public Expenditure Review – An Efficiency, Effectiveness, Equity, Adequacy and Sustainability Analysis. The World Bank, October 2017. Report No. ACS23503. - Second Joint Partnership Strategy (JPS) (FY2013 to FY2016), World Bank. Report No. 72140. - Systematic Country Diagnostic (SCD) - Early results from PSI-PMI Impact Evaluation - Education Sector Strategic Plan Page 95 of 95