Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Report Number: ICRR0021615 1. Project Data Project ID Project Name P133079 READ Project Country Practice Area(Lead) Gambia, The Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D1370,IDA-H9160,TF-16496,TF- 28-Feb-2018 26,813,346.15 A2125 Bank Approval Date Closing Date (Actual) 06-Mar-2014 31-Aug-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 11,900,000.00 7,900,000.00 Revised Commitment 27,300,000.00 7,900,000.00 Actual 26,866,553.65 7,891,298.87 Prepared by Reviewed by ICR Review Coordinator Group Salim J. Habayeb Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objectives of the project were to “increase access to basic education, improve quality of teaching and learning in lower basic schools, and strengthen education systems” (Financing Agreement, 4/9/14). The statements of objectives in the PAD and ICR were identical. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Note on a revision of one associated outcome target. A restructuring associated with additional financing on 9/14/16 revised two outcome indicators upwards, and another one (scores in English) downwards to reflect increased enrollments from disadvantaged groups who traditionally have lower performance in language tests. Since both original and revised targets were exceeded by project closing, neither the ICR nor this ICR Review applied a split evaluation, as it would not add value to outcome assessment in this case. Note on basic education: Basic or primary education consists of six years of primary education known as lower basic (grades 1-6, ages 7-12); three years of upper basic education (grades 7-9, ages 13-16); and Early Childhood Care and Development (ECD). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 14-Sep-2016 c. Will a split evaluation be undertaken? No d. Components The original components and associated revisions are described below. The revisions were associated with additional financing in response to a national financial crisis brought on by external and internal shocks, including significant public sector debt, overvaluation of currency, delayed rainy season, and Ebola Virus Disease. The revisions provided temporary financing for teacher salaries, allowances, and school subgrants to ensure that progress and service delivery were maintained. Concurrently, ECD development strategy was modified from separate facilities to annexation of ECD classrooms to primary schools, based on evidence indicating that integration had higher impact. 1. Increase Access to Basic Education (Appraisal US$6.0 million; Actual US$6.0 million). (a) School construction: Construction of about 40 multi-grade schools and water points in remote areas; and completion of construction for two schools of 9 grades, including 18 urban classrooms in each school. (b) Donkey carts: Provision of about 100 redesigned donkey carts for transportation for early grade children. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) (c) Community-based ECD improvement: Construction of ECD classrooms; training for facilitators; development of a program for facilitator certification; provision of teaching and learning materials; supervision and monitoring of the program; and development of a monitoring tool. (d) School subgrants supporting subprojects, including the provision of school materials and stationery. 2016 Revisions:  Based on evidence from an impact evaluation that annexation had the greatest impact, the project opted to annex ECD classrooms, staff quarters, and toilet blocks rather than building separate community-based ECD structures as originally planned. In addition, the component provided for attaching ECD classrooms to 27 existing multigrade schools (out of 48 multigrade schools) along with the provision of furniture, teaching and learning materials, and additional training to ECD teachers.  As part of the emergency response to the fiscal crisis in the country, the component provided school subgrants to subprojects in all public lower basic schools and upper basic schools, and stationery to all students in hardship areas. The incremental support was planned for seven months (ICR, p. 17). 2. Improve the Quality of Teaching and Learning (Appraisal U$$6.2 million; Revised US$18.0 million; Actual US$17.3 million). (a) Enhanced teacher training: Restructuring of the Primary Teacher Certificate and the Higher Teacher Certificate programs, through the provision of advisory services to relevant education staff to strengthen teacher training, including assessment and redesign of the existing programs and development of a professional examination. (b) Stipends for student teachers: Provision of in-service teacher support, through the provision of student teacher stipends to approximately 1,600 Higher Teacher Certificate students per year and 700 Primary Teacher Certificate students in their third year. (c) Hardship allowances for public lower basic school teachers in remote areas. (d) Extended teacher continuing professional development: Design and implementation of a demand- driven in-service teacher professional development course and monitoring tools. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) (e) Teacher supervision: Provision of advisory services for the development, piloting, and validation of standardized teacher observation tools. (f) Reading in early grades: Improving early grades reading and foundational skills, including: carrying out studies on reading programs; review of the reading curriculum and transition to full English proficiency; strengthening teacher capacity on teaching reading skills; development and distribution of reading materials and library books; assessment of early grade reading and math skills; and support to student reading progress monitoring (g) Developing evidence on improved reading performance through support of conditional cash transfers to Koranic schools (Majalis or Daaras), benefiting about 1,500 pupils in literacy and numeracy. (h) LEARNET: Support the phased connection of schools to LEARNET, a high-bandwidth Information and Communications Technology (ICT) learning network. 2016 Revisions:  The revisions provided US$5.2 million in teacher salaries and allowances as emergency assistance in the form of temporary funding to avoid teachers being unpaid during the fiscal crisis, after which the Government would resume payments in June 2017.  Rather than providing broadband connectivity, the revision scaled up the Progressive Science Initiative/Progressive Math Initiative or PSI-PMI approach, specifically for the development of curriculum-linked digital content of mathematics, science and English. The reasons for this change included cost-effectiveness and long-term sustainability, as the public-private partnership necessary to provide broadband could not function effectively in a resource-constrained environment (ICR, p. 19). 3. Technical and Institutional Support (Appraisal US$3.0 million; Revised US$1.7 million; Actual US$2.4 million). (a) Communications strategy: Development and implementation of a communications strategy, including dissemination of information, training, outreach campaigns, and surveys. (b) Institutional support and capacity building through continuous implementation of learning outcome assessments; strengthening education data systems; strengthening the capacity of education staff in Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) assessing learning outcomes, statistics, and data collection; completing a national qualifications framework for higher education; and supporting project implementation, supervision, monitoring, and evaluation. 4. Strengthening the Education System with a Focus on Governance and Management Results (Appraisal US$3.6 million; Revised US$1.6 million; Actual US$1.6 million). Improving access to education, and teacher and student performance, through the achievement of Disbursement Linked Indicators (DLIs). 2016 Revisions: Challenges related to fund flows under the DLI mechanism were encountered (ICR, p. 16 and p. 23). Upon meeting DLIs, funds were made available in the consolidated account of the Ministry of Finance, but onward transfers to the Ministry of Basic and Secondary Education (MOBSE) were constrained by a difficult fiscal situation. Hence, the project reverted to the use of traditional investment approaches for this component, and DLIs were reclassified as intermediate results indicators. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost and Financing. The original project cost was estimated at US$34.8 million, consisting of: (a) US$18.8 million external financing (an IDA grant of US$11.9 million and another grant of US$6.9 million from the Global Partnership for Education); and (b) a Borrower/Recipient contribution of US$16 million equivalent. Traditional processes for Investment Project Financing were used, except for component 4 (strengthening the education system), which was carried out using a results-based approach. Additional financing on 9/14/16 provided an IDA grant of US$7.5 million equivalent and a grant of US$1.0 million from the Early Learning Partnership Trust Fund to support temporary support for teachers’ salaries, allowances, and school grants as an emergency measure in response to the fiscal crisis in the country, and to further develop ECD activities (ICR, p. 15), all aggregating at US$27.3 million. No direct financing contribution was provided by the Borrower/Recipient in view of the financial crisis. The actual cost was US$26.9 million, reflecting a disbursement of about 99% against the revised cost of US$27.3 million. Dates. The initial design planned for a four-year implementation period. The project was approved on 3/6/14 and became effective on 4/25/14. A Mid-Term Review was carried out on 12/5/15. The project closed on 8/31/18, six months beyond the original closing date. The project underwent two restructurings: (a) the first restructuring on 9/14/16 was associated with additional financing in response to the financial crisis. It modified the financing plan, disbursement arrangements, and results framework (see Section 2a); and (b) the second restructuring on 2/9/18 extended the project closing date by six months to 8/31/18 to complete activities and to ensure full handover of the constructed schools. Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) 3. Relevance of Objectives Rationale The project objectives were responsive to key education challenges in the country. At appraisal, the education sector, which is heavily donor-dependent, was facing multiple challenges, and improving students’ learning outcomes was a national priority. Access to basic education was inequitable, and scores of Early Grade Reading Assessment were low. The National Assessment Test conducted annually in grades 3, 5, and 8 revealed in 2012 that average scores in English ranged between 42% and 45%, and in mathematics between 37% and 42%. Out-of-school and dropout rates represented considerable inefficiencies in the system. A World Bank study in 2013 showed that the factors most reported by families were cost of access (46%), distance to school (15%), and work and family activities (7%). The cost burden included informal school fees and participation costs such as uniforms, shoes, school bags, and lunches. The country was pursuing efforts to improve teaching quality, but more efforts were needed to improve teacher training programs, and to provide an added focus on relevant subject content, pedagogic skills, ICT competencies, professional values and attitudes, and a higher quality intake of teacher candidates. At appraisal, the objectives were in line with Gambia Second Joint Assistance Strategy FY2013-2016 (World Bank and African Development Bank), Bank Education Strategy, and priorities of the Global Partnership for Education. At project closing, the objectives remained fully consistent with the current Country Engagement Note (CEN 2018-2020) that followed a Systematic Country Diagnostic in 2017. The CEN had three focus areas of engagement: macroeconomic stabilization, restarting growth, and protecting the poorest from the impact of macroeconomic instability. Project objectives remained aligned to the third focus area, given that the project was centered on equitable access to quality learning opportunities and on improving the quality of education that ultimately promotes the likelihood of greater earnings. The project was in line with Government’s Education Sector Strategic Plan (ESSP) II 2014-2022 that aimed at similar improvements, including for increasing equitable access to quality education and strengthening sector management. The objectives were also consistent with the twin goals of the Bank for poverty reduction and shared prosperity. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Increase access to basic education. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Rationale Rationale It was reasonable to expect that school construction, expanded ECD offerings, provision of school grants that alleviate school fee barriers to schooling, stationery packages to hardship schools, and transportation of young school children via donkey carts in remote communities would improve schooling availability, enhance school readiness and its environment, and reduce parents' financial burden, which in turn would plausibly contribute to increased access to basic education. Outputs and intermediate results The project built 84 additional primary level classrooms and 29 ECD classrooms, meeting the target for each. It provided 54,611 stationery packages to lower basic school students in hardship areas, exceeding the target of 36,500 packages. The project provided school grants on a regular basis. The number of communities located more than three kilometers away from a lower basic school was reduced from a baseline of 17% (339 communities) in 2013 to less than 5% (97 communities) by project closing in 2018, exceeding the target of 14% (however, this indicator was attributable to the collective support of development partners, the Bank-supported project, and the government, ICR, p. 51). All intermediate results targets were met or exceeded. School grants were aimed both at reducing informal school fees and at quality improvement expenditures, mainly covering recurrent school costs such as stationery, chalk, material for posters, extra teacher training, learning excursions, and minor school maintenance. Schools were precluded from using the grants for major civil works or additional pay to teachers (ICR, p. 52). Outcomes Under the project, 79,541 additional students were enrolled in public basic schools and ECD by 2018, exceeding both the original target of 35,000 and the revised target of 48,000 additional students. The breakdown of this increase was as follows: in ECD, the baseline of 19,590 in 2013 increased to 31,848 students in 2018; in lower basic education from 184,279 to 245,106 students; and in upper basic education from 69,372 to 75,827. The total aggregated baseline for basic education was 273,241 students, increasing to a total of 352,782 students in 2018. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Direct project beneficiaries reached 330,442 in 2018, exceeding both the original target of 290,000 and the revised target of 305,000. The percentage of female beneficiaries was 52%, exceeding the target of 50%. Direct beneficiaries included basic primary education students, including ECD students, teacher trainees receiving stipends, and teachers receiving hardship allowances. A noteworthy associated development was that the project resulted in the reduction and quasi-removal of the burden of school fees on parents. In 2013, the annual average school fees paid by parents for their children in lower basic schools was US$12 equivalent. A third-party firm conducted an evaluation of formal and informal fees paid by parents and showed that only a small proportion of parents (2.9%) paid an average of US$0.3 equivalent in 2018, resulting in a total average of only US$0.01. In public upper basic schools, 6.7% of parents paid US$2.7 on average, reflecting a total average of US$0.18 against US$17 at the baseline. Rating High OBJECTIVE 2 Objective Improve the quality of teaching and learning in lower basic schools. Rationale Rationale In addition to the school improvements described under objective 1, the theory of change in both the ICR and the PAD (p. 51) illustrated a combination of incentives and improved professional development, where it was reasonable to expect that improved initial teacher training, expanded in-service courses, stipends for student teachers, hardship allowances in remote areas, teacher supervision and support, improved reading curricula, and provision of textbooks to students would improve teacher skills, qualifications, and deployment, and student opportunities to learn, which in turn would plausibly contribute to improving the quality of teaching and learning. Outputs and Intermediate results 1,906 additional qualified teachers emerged from the project, exceeding the target of 1,750. The reading curriculum guidelines were developed by Directorate of Curriculum of MOBSE based on evaluation results of early reading strategies. Human resources supplementary data (personnel details, qualifications, promotion Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) and postings history) were updated in the human resources database at Directorate of Human Resources in MOBSE. Average teacher on-time attendance increased from a baseline of 90% in 2013 to 95% in 2018, attaining the target. An ECD monitoring tool was developed and used. 176 teachers benefited from training using the newly developed e-curriculum, short of the target of 200 teachers. Externally reviewed and overseen end-of-year examinations for students completing the second year of revised Primary Teacher Certificate and Higher Teacher Certificate programs were not continued under the project because the firm revising the Gambia College curriculum indicated that the review of the new curriculum had to be conducted prior to completing the development of the end-of- year examination. According to the ICR (p. 45), this task was planned to be pursued under the follow-on project (Education Sector Support Program, P162890, US$30 million, 2018-2022). A new classroom observation tool was developed and consisted of direct in-class observation of the teaching process by cluster monitors and Head Teachers (TTL clarifications, 6/5/19). Its aim was to: (a) provide developmental feedback to teachers; (b) guide supervisors and schools in conducting supervision; and (c) provide a mechanism for systematic recording and reporting of teaching quality. The five areas of analysis in the classroom observation tool included: (i) time on task, (ii) preparation and planning; (iii) presentation/lesson delivery; (iv) classroom management; and (v) student engagement in learning. The distribution of textbooks improved the pupil-textbook ratio of core textbooks in grades 1-9 in public schools from 4 to 1:1 by project closing in 2018. Outcomes The percentage of teachers in grades 1-3, from a randomly selected 15 percent of schools, who showed good teaching performance through the newly introduced classroom observation tool reached 82.7%, exceeding the target of 60%. Mean scores in English in grade 3 in National Assessment Tests increased from a baseline of 41.5% in 2012 to 47.8% in 2017, exceeding both the original target of 46% and the revised target of 43%. The ICR stated that a verification method was put in place to confirm the presence of the majority of children during testing days and that attendance related to all students rather than the better performing ones (ICR, p. 22) The Borrower comments also noted (ICR, p. 93) that, under the follow-on project, the government intends to develop a comprehensive and strategic teacher training framework for all types of training, from pre-service to in-service, along a continuum. Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Rating Substantial OBJECTIVE 3 Objective Strengthen education systems in basic education. Rationale Rationale It was reasonable to expect that system-strengthening activities in governance and management results, enhancement of evidence-based decision-making and accountability, and technical and institutional support, including the implementation of National Assessment Tests; collection and analysis of test results; distribution of test results to regional directorates and to schools; review of National Assessment Test results to inform School Development Plans; and evaluation of the degree to which School Development Plans were adjusted would plausibly contribute to strengthened education systems. Outputs and intermediate results Annual training, promotion, and postings were linked with annually updated personnel records in interfaced electronic management information systems and human resource systems. Training on School Development Plans was conducted four times during project implementation by the Standards and Quality Assurance Directorate and Planning Directorate of MOBSE. In 2015, 800 head teachers and cluster monitors were trained. In 2016, 230 additional head teachers were trained, and a second training was conducted for all head teachers, Regional Education Directorates staff, and cluster trainers. In 2018, third-party consultants evaluated School Development Plans and confirmed that 95.3 percent were implemented. The Gambia College curriculum was revised. The number of student teachers enrolled in the revised Primary Teacher Certificate program reached 1,074 in 2018, exceeding the target of 900. An annual school census was carried out. A Basic Education Sector communication strategy was developed by MOBSE. Citizen feedback was included in the development of the school improvement grant manual. The average National Assessment Test attendance of grade 3 and 5 students in all public schools for two full days increased from a baseline of 89.1% in 2013 to 94.6% in 2018, close to the target of 95%. The average National Assessment Test attendance of grade 8 students in all public schools for two full days increased from a baseline of 83.7% in 2013 to 91.5% in 2018, short of the original target of 95%, but exceeding the Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) revised target of 90%. The percentage of Head Teachers skilled to interpret National Assessment Test results reached 100% in 2018, exceeding the target of 95%. Outcomes The percentage of Head Teachers in lower basic schools adjusting School Development Plans in response to National Assessment Test results and approved by Regional Directorates using an agreed checklist increased from a 0 baseline to 95.3% in 2018, exceeding the target of 80%. According to the ICR, this achievement built on the collective and systemic strengthening activities shown in the rationale section, above. Rating Substantial OVERALL EFF TBL OLD Rationale The aggregation of one fully achieved objective with two almost fully achieved objectives is indicative of Substantial overall efficacy. Overall Efficacy Rating Substantial 5. Efficiency Both the PAD and ICR carried out adequate economic analysis, and the PAD (pp. 26-27) referred to wage regression coefficients showing increasing returns to education as levels of education go up, after controlling for other determinants. Similar to results found in other developing countries, the regression estimation for Gambia showed that one additional year of education resulted in 6.1% higher wages, that people with higher levels of education prefer to work in salaried jobs, and that people who attain a higher level of education receive greater returns from education investments. The cost-benefit analysis carried out by the PAD focused on quantifiable benefits and provided an economic rationale for the non-tangible benefit portions using findings from global research (PAD, p. 107). The analysis captured the economic benefit stream realized in terms of lifetime earnings resulting from better learning, and Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) intermediate benefits of interventions. A 12% discount rate was used to derive future returns. The cost-benefit analysis estimated a net present value (NPV) of benefits at US$26.6 million with an associated internal rate of return (IRR) of 20%. The benefit/cost ratio was estimated at 1.9 (PAD, p. 111). A sensitivity analysis used a higher discount rate of 15% under different scenarios (PAD, p. 109). Using a lower-bound discount rate of 18% and an upper-bound discount rate of 10%, the sensitivity analysis estimated IRRs at 18% and 24% respectively (PAD, p. 13). The ICR carried out a similar cost-benefit analysis and compared its findings at project closing with those carried out by the PAD at appraisal (ICR, pp. 24-25). The corresponding NPV of benefits was estimated at US$27.7 million, compared with US$26.6 million at appraisal. The IRR associated with this NPV was 21%. The benefit/cost ratio was estimated at 1.7. While some benefits could not be quantified, the NPV from quantifiable benefits was larger than NPV costs, suggesting the efficiency of project investments and indicating its economic viability, according to the analysis. In addition, several unit costs for project-supported activities, such as for textbooks and classroom construction, were favorable (ICR, pp. 25-26). Unit costs ranged from US$0.24 to US$0.57 for textbooks, and from US$0.19 to US$0.81 for teacher guides. In the Democratic Republic of Congo, the unit cost of textbooks was US$1.3, and ranged between US$0.97 and US$1.66 in Sierra Leone, and between US$1.10 and US$1.40 in Niger. The low cost of textbooks under the project was in part attributable to selected firm experience and the larger number of textbooks and guides procured, allowing economies of scale to be realized. No shortcomings of significance were reported during implementation. The ICR (p. 26) described the efficient execution of school grants as confirmed by audit findings. The ICR noted that 12 construction contracts were finalized during the project with an average of 368 days between contract signing date and the provisional hand- over of sites, and that construction used durable and child friendly materials, contributing to improved learning environments with structures that were expected to last for more than 30 years assuming regular maintenance was undertaken. The government pursued a highly participatory and consultative process to facilitate implementation through its regular regional visits and Coordinating Committee Meetings, which were rotated through the regions (ICR, p. 29). The rate of disbursement was high throughout implementation and reached nearly 50% after the first year of implementation (ICR, p. 32). There were some occasional delays related to elections and financial constraints, and the project closing date was extended by six months to complete activities (ICR, p. 86). Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  20.00  Not Applicable 100.00 ICR Estimate  21.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated High, as the objectives remained fully aligned with the Country Engagement Note at project closing and with the country’s Education Sector Strategic Plan. Efficacy is rated Substantial, as the project almost fully achieved its objectives. Efficiency is rated Substantial in view of favorable returns and largely efficient implementation. Therefore, the overall Outcome is rated Satisfactory, indicative of essentially minor shortcomings in the project’s overall preparation, implementation, and achievement. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Main risks to sustaining development outcomes are related to macroeconomic challenges and vulnerability to external shocks. Hence, in the near term, continued financial support from development partners remains critical. This was evident when temporary relief for salaries was needed during the fiscal crisis. However, given that project interventions emanated from the ESSP and that government commitment remains high, it is reasonable to expect that activities will continue, and outcomes will be sustained. School grants were fully integrated into the MOBSE budget (ICR, p. 35). Capacity building was notable, including for improved technical capacity to deploy and manage human resources, and for M&E. A follow-on project, the Education Sector Support Program (P162890), became effective on 8/31/18 and built on the progress made under this project (TTL clarification, 6/5/19). 8. Assessment of Bank Performance a. Quality-at-Entry The project design was aligned with the national education sector strategy. The government demonstrated high commitment and played a major role in project preparation (ICR, 91). Stakeholders were fully involved in the preparation of the project, as required by the Global Partnership for Education (ICR, p. 92). The project's association with the Global Partnership further increased the quality of preparation. Stakeholders included development partners, officials from different ministries, and the Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) private sector. Their core group was known as the Local Education Group. Past lessons learned were considered in design and implementation, including for improving access and equity, and the importance of hardship allowances and teacher training. Institutional and implementation arrangements were adequately prepared and included a senior inter-ministerial management team for oversight with periodic representation from the Ministry of Finance and Economic Affairs. A pre-existing Project Coordination Unit was assigned to the project, as it was familiar with Bank processes and guidelines. An environmental and social safeguards focal point was assigned to the unit. Although stakeholders were engaged, a stakeholder workshop was organized to foster ownership (PAD, p. 23). Fiduciary aspects were well planned and were in line with the Government Budget Management and Accountability Act. Risks were adequately identified, and mitigation measures were planned, including the risk of introducing a new disbursement mechanism with DLIs for one of the project components. Quality-at-Entry Rating Highly Satisfactory b. Quality of supervision Supervision missions were regular, and there was continuity in task team leadership. According to the ICR (p. 34), documentation of progress made was "extremely thorough" throughout the project with diligent follow up, including monitoring compliance with safeguards. Regular financial management and procurement missions were undertaken by related specialists based in neighboring Senegal. During implementation, stakeholders led by the government were actively engaged through the bi-monthly Coordinating Committee Meetings and the monthly Local Education Group meetings. All six regions were visited each year by senior MOBSE officials. The Bank team was reportedly proactive and mobilized technical assistance as necessary to support MOBSE. The team was also responsive to sectoral needs during the financial crisis and in project restructuring that facilitated continued progress towards the achievement of intended objectives. No supervision and implementation support issues were reported by the ICR. Quality of Supervision Rating Highly Satisfactory Overall Bank Performance Rating Highly Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objectives were clearly specified, and the theory of change was sound and reflected in the results framework. The indicators encompassed all outcomes of the objectives statement. Both intermediate and outcome indicators were suitable for tracking progress and in assessing outcome achievements. M&E was closely aligned to the ESSP. Baselines for relevant indicators were available. The fact that M&E Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) arrangements were embedded in existing systems, rather than a standalone arrangement, provided added strength to the design. MOBSE officers who managed the Education Management Information System were responsible for M&E. Additional surveys and assessments were planned (ICR, p, 31) for verification of DLIs and for technical aspects, including data collection for an ECD impact evaluation, evaluation of the classroom observation tool, assessment of reading performance in Majalis, evaluation of the early reading program, and evaluation of the effectiveness of the hardship allowance program. b. M&E Implementation M&E activities were implemented as planned through the government’s own systems of collection and with supplementary methods of collection and analysis as noted above in Section 9a. Additional data were collected through an integrated household survey. Additional studies were developed to supplement data in areas where interventions were new, or in new areas, such as for further analysis of ECD approaches, early reading, and foundational skills development in Majalis (ICR, 31). Supplemental training was undertaken in areas that required additional capacity building. According to the ICR (p. 32), reporting was timely and transparent. c. M&E Utilization In addition to routine use of M&E for project monitoring and identification of capacity building needs, M&E findings were used to change ECD approaches from developing separate facilities to school- annexed classrooms where a higher impact was witnessed. Also, human resources data were integrated with the Education Management Information System to better address teacher deployment aspects (ICR, p. 32). M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified under Environmental Category B - Partial Assessment. At entry, the project triggered two safeguard policies: Environmental Assessment OP/BP 4.01 and Involuntary Resettlement OP/BP 4.12 (ISDS-Appraisal Stage, and PAD, p. 32). At project restructuring in 2016, a third safeguard policy was triggered: Physical Cultural Resources OP/BP 4.11 (ISDS-Restructuring Stage and ICR, p. 33). It was unknown if subproject sites contained physical cultural resources, but, as a measure of prudence, potential impacts were assessed in the context of the Environmental and Social Management Framework (ESMF). An ESMF and a Resettlement Policy Framework were prepared and implemented, including institutional arrangements, roles and responsibilities of stakeholder groups involved, and implementation and monitoring Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) of mitigation measures. A safeguards focal point in the Project Coordination Unit at MOBSE worked in collaboration with the National Environmental Agency to complete initial safeguards screening under national regulations, and both parties were actively engaged throughout the project. Environment and social safeguards checklists were not always systematically used for site visits and screening before the construction of schools started, and in the final stages, there was hazardous disposal of construction waste and inadequate development of water towers. These issues were rectified before project closure, and school sites were cleaned. Two compensation issues for land donations were observed and resolved. According to the ICR (p. 33), overall compliance was fairly adequate, but with some implementation challenges having been encountered, safeguards compliance was rated Moderately Satisfactory throughout the project period. b. Fiduciary Compliance Financial management. The ICR (p. 26) noted that financial management was strong and rated satisfactory by the Bank Team throughout project implementation. Audits were undertaken biannually and were unqualified (ICR, p. 32). Reporting was adequate, and interim financial reports were produced in accordance with established guidelines and procedures and submitted in a timely manner. The ICR (p. 26) stated that information from internal and external audits showed that project resources were used for their intended purposes. Procurement. Procurement performance was also rated satisfactory by the Bank Team throughout implementation, as procurement followed guidelines and its performance remained on track (ICR, p. 33). The ICR reported minor delays in the procurement of textbooks during the final year of the project, and which were related to funds flow issues at the Central Bank. c. Unintended impacts (Positive or Negative) None reported. d. Other -- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) No shortcomings were identified Bank Performance Satisfactory Highly Satisfactory in the quality at entry or supervision. Quality of M&E Substantial Substantial Quality of ICR --- High 12. Lessons The ICR (pp. 35-36) offered several lessons and recommendations, including the following lessons restated by IEG: The Bank’s proactive engagement during periods of economic crisis can be instrumental in ensuring continued service delivery and sustained progress towards the achievement of development objectives. The Bank’s strategic decision to provide additional financing to the Government of Gambia when it faced a severe fiscal crisis ensured that project progress was maintained and subsequently scaled up. High participation of senior officials in project supervision facilitates informed decision- making. The Minister, Permanent Secretary, Deputy Permanent Secretary, and Directors of the Ministry of Basic and Secondary Education visited one out of the six Regions every two months for one week. Hence, all regions were visited every year. The high-level officials became familiar with both physical and learning challenges through interactions with school heads, teachers, and students, and the findings were shared at the Coordinating Committee Meetings. The ministry’s senior management team became familiar with the challenges faced by the schools and their needs, thus facilitating decisions in response to those needs. Donkey carts can provide a viable though unconventional solution to schooling access in difficult areas. In order to address access barriers in difficult areas, the project used donkey carts for transporting young children to school, as the children would not have been able to walk long distances. The government provided the donkeys, and the project funded the carts. This approach strengthened community engagement and accountability: communities had to care for and feed the donkeys, and in return they could use the animals for other tasks during the remainder of the day, after transporting the children to and from schools. This arrangement constituted a sustainable and low-cost transport solution for some of the most remote villages serving 2,000 school children daily and was continued after project closing. 13. Assessment Recommended? Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review READ Project (P133079) No 14. Comments on Quality of ICR The ICR was highly clear and consistent, both internally and with the guidelines. It was results-oriented, and the theory of change was aligned to development objectives. The evidence and analysis were also aligned to the messages outlined in the ICR. Both narrative and evidence supported the ICR's main conclusions. Its overview of the sector context and its thorough discussion of M&E were noteworthy. Lessons were derived directly from project experience as outlined in the ICR. No shortcomings were identified in this high-quality ICR. a. Quality of ICR Rating High Page 18 of 18