Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) Report Number : ICRR0020867 1. Project Data Project ID Project Name P117662 LR - Education GPE Program (FY11) Country Practice Area(Lead) Liberia Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-97456 30-Jun-2013 40,000,000.00 Bank Approval Date Closing Date (Actual) 27-Aug-2010 31-Oct-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 40,000,000.00 40,000,000.00 Revised Commitment 40,000,000.00 40,000,000.00 Actual 40,000,000.00 40,000,000.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz John R. Eriksson Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components a. Objectives According to the Grant Agreement (page 5) and the Project Appraisal Document (PAD, para. 49), the original project objectives were as follows: • To increase access to basic education with a particular focus on poor areas; • To improve conditions of teaching and learning; and • To improve school management and accountability. Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) In September 2012, the project objectives were revised as follows (Project Paper, 2012, page 6): •To improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school based management. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 06-Sep-2012 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The original project components were as follows: 1. Increasing Access and Equity in Rural Areas (Appraisal: US$ 15.5 million); This component aimed to increase enrollment in primary education by constructing and/or rehabilitating classrooms, with a focus on underserved areas. Activities included: construction of 24 primary schools comprised of six classrooms each and auxiliary facilities; construction of up to 20 junior secondary schools; and construction of up to 16 primary schools using a community driven development approach in particularly hard-to-reach areas. Given the lack of local construction management capacity within the country, a qualified firm of civil works consultants was to be contracted to manage and supervise this component. 2. Improving the Conditions of Teaching and Learning (Appraisal: US$ 14.0 million); This component aimed to ensure the availability of learning materials and supportive services for school children to address poor health, the lack of academic preparedness of children, and the lack of finances for operating expenses. Activities included: provision of textbooks and teaching guides; implementation of an early childhood development model; de-worming activities; and provision of small grants to schools to help cover operating costs. Teacher training had also been considered under this component, but was not included, owing to support from other donors (PAD, p. 27, ICR p.55). 3. Strengthening Central and Local Levels to Operate in a Decentralized Education System by Development Management, Monitoring and Evaluation Capacities (Appraisal: US$ 10.5 million): This component aimed to develop institutional capacity of the central Ministry of Education (MOE) personnel as well as district-level education officers. Activities included: establishment of a project support team in the MOE to assist with fiduciary management; development of a school-level assessment tool; development of a national student assessment test; and support to studies to improve personnel and payroll management. The components were revised in September 2012 as follows: 1. Improving Management Capacity and Accountability at the Central and School Levels (Appraisal: US$ 40.0 million; Actual: US$ 40.0 million): This single project component was divided into four sub- Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) components, encompassing several of the activities listed above. 1.1. Strengthened institutional management capacity and accountability at the central and school levels (Appraisal: US$ 5.5 million; Actual: US$5.5 million). 1.2 School construction in rural areas (Appraisal: US$ 16.2 million; Actual: US$ 18.8 million). 1.3 Procurement and distribution of teaching and learning materials (Appraisal: US$ 11.0 million; Actual: US$ 13.2 million). 1.4 School-based management through school grants (Appraisal: US$ 7.3 million; Actual: US$ 2.4 million). The following activities were dropped, due to weak capacity: • Early childhood development centers • School de-worming • Learning assessment tools • Education monitoring and evaluation system • Personnel and payroll management e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost • The appraised and actual project cost was US$ 40.0 million. Financing • The project was financed entirely by a Global Partnership for Education grant of US$ 40.0 million, also referred to as a “Catalytic Fund” in the PAD (p.24) and ICR (p.65) Borrower Contribution • There was no planned Borrower contribution. Dates • November 2011: A Level II restructuring took place, to include the construction of early childhood learning centers, in addition to renovation of existing buildings, to serve as such. This change was due to the lack of existing buildings that could be used for that purpose. • September 2012: The project was restructured due to significant capacity weaknesses and implementation challenges. The project objectives and results framework were modified. The scope of project components was also significantly reduced. • February 2013: The project closing date was extended from June 2013 to June 2015, to allow completion of activities. • August 2014: The country is hit by the Ebola virus outbreak. The President declares a three-month Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) state of emergency that includes a curfew, school closures, and restrictions on movement and gathering of people. • June 2015: Due to the outbreak of the Ebola virus and the impact on project implementation, the project closing date was extended from June 2015 to June 2016. US$ 2.7 million was also reallocated from school grants component to school construction component due to cost overruns (general cost escalation due to the outbreak). • June 2016: The project closing date was extended from June 2016 to October 2016, to allow completion of activities, particularly school construction. 3. Relevance of Objectives & Design a. Relevance of Objectives Following the end of a 14-year civil war in 2003, Liberia made progress on multiple fronts to restore political stability and basic services for its population. A 2007 Demographic and Health Survey indicated that one- third of the population had no formal education and that 31.8% had only a primary level education, conditions which posed significant constraints on the country's overall productivity and economic growth potential. The main challenges to the primary education sector included: a shortage of qualified and trained teachers; inadequate payroll management, teacher management and deployment; low quality of education; significant above-age student population; inequitable access; high household costs for education; and weak sector monitoring capacity (PAD, paras. 18-35). These factors contributed to an overall gross enrollment rate (GER) of 94% and net enrollment rate (NER) of only 35% (2010). The project objectives respond to these long-term challenges as well as the immediate needs of the education sector, as the sector transitions from emergency service provision to strengthened quality services within a more robust education system. The original and revised objectives are consistent with the government's ten-year Education Sector Plan (2010-2020), which includes an objective to provide quality basic education. The Bank's Country Partnership Strategy for FY2013-17 more explicitly identifies improved conditions for learning and management capacity in basic education as a key outcome. The revised objective remains highly relevant albeit notably less ambitious and less focused on a development outcome (increased access to education) compared to the more intermediate outcome focused on improved management and accountability. Rating Revised Rating High High b. Relevance of Design The original project design was somewhat ambitious, although the results chain was logical, with key project outputs likely to lead to the intended outcomes. Construction of classrooms and provision of educational Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) materials were likely to increase access to primary education, given the significant number of destroyed school facilities and the high pupil-classroom ratio. Other outputs including school grants, early childhood education programs, and school health programs were also likely to improve learning conditions and enable increased access. However, the relevance of the original project design is rated Modest due to significant changes in the institutional and political environment soon after project was approved (2011). The government passed a new Education Reform Act which "substantially altered the institutional structure of the basic education system by decentralizing many functions" and therefore there existed a significant disconnect between the original project design and the MOE's reform priorities (Project Paper, page 5). The likelihood of achieving outcomes in such an institutional environment was modest. In addition, the lack of selectivity in the project design contributed to complexity and low feasibility of being completed within the project time frame. The relevance of the project design increased for the revised design, taking into account the new decentralized system as well as the low implementation capacity. The project's scope was narrowed to focus on capacity building in the central Ministry of Education and at the school level. Key intervention areas that were retained from the original design included management capacity and accountability mechanisms, school construction in rural areas, procurement and distribution of teaching and learning materials, and school grants. As noted in the Project Paper (page 7), while the project design remained somewhat ambitious, this was in response to the need to retain policy focus on high priority activities. Rating Revised Rating Modest Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To increase access to basic education with a particular focus on poor areas Rationale Outputs • Construction and/or rehabilitation of 303 primary classrooms (original target: 108). There is no information on whether these were constructed in "poor" areas, only that they were constructed in rural areas. Outcomes • According to the education management information system for the school year 2015/16, the schools were staffed with 273 teachers and enrolled nearly 11,000 students, of which 4,591 were in early Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) childhood education, 5,349 were in primary school, and 875 were in junior high school. There were no accurate baseline figures available for number of students and number of teachers prior to the project interventions, according to the project team (some schools existed but were dilapidated, some were non- functional, some had students but the number was unclear), nor were any targets set for enrollment. • The indicator measuring the basic education completion rate was dropped, as the project period was deemed too short for this outcome to materialize. Achievement is rated Modest. Although there is data reflecting increased enrollment, there is no evidence provided that this took place in poor areas. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To improve conditions of teaching and learning Rationale Outputs • Provision of one million textbooks, 20,000 teacher guides for grades 5-9, one million supplementary readers for grades 1-4, and other instructional materials, as planned. However, • Development of an early childhood development model was dropped. • De-worming activities for school children was dropped. Outcomes • The indicator measuring the textbook:student ratio at the basic education level was dropped, due to the poor quality of data on student enrollment. • The indicator measuring whether teachers had at least one teaching guide was dropped and instead, replaced by indicators on the procurement and distribution process. Achievement is rated Modest due to significant shortcomings in implementation. While teaching and learning materials were procured and distributed as planned, other key activities related to early childhood Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) development and school health were not implemented. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To improve school management and accountability Rationale Outputs • Development of school grant guidelines, followed by the provision of school grants to 2,579 schools (based on a process of developing grant utilization plans and opening school bank accounts) (target: 1,000 schools). • Construction of a new Ministry of Education (MOE) building to consolidate all central MOE staff in one location. As noted in the ICR (page 21), in the long term, the building is expected to enable the "creation of a conducive working environment for MOE staff, with positive implications for improved synergy and efficiency." • Conducting of two cycles of a Rapid Results Approach to increase implementation capacity of MOE staff. However, the following outputs were not implemented due to weak capacity; • Development of a unified Human Resources database with other sector Ministries. • Development of a pay and grade scale for teachers. • Publication of annual sector statistics. • Development of a learning assessment system at primary school level. Outcomes • 72% of school grants were implemented in accordance with guidelines. This surpassed the target of 30%. • The indicator measuring schools managed by a school management committee or parent-teacher association was dropped, due to lack of measurements available for this indicator. Achievement is rated Modest due to significant shortcomings in implementation. While there was evidence of achievement in implementing the school grants program in support of improved school management, Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) other key activities were dropped. Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 3 Revision 1 Revised Objective To improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school based management Revised Rationale Outputs School construction • Construction/rehabilitation of 303 primary classrooms (revised target: 272) in 41 schools in rural areas. • Development of standardized school construction guidelines for site selection, design, contract management and construction quality, as well as a school construction database and school maintenance guidelines. Teaching and learning materials • Provision of one million textbooks, 20,000 teacher guides for grades 5-9, one million supplementary readers for grades 1-4, and other instructional materials, as planned. • Updating of national curriculum to align with West African Senior School Certificate exam. • Support to distribution chain, and development of social accountability framework for distribution of teaching/learning materials. School-based management • Development of school grant guidelines, followed by the provision of school grants to 2,579 schools (target: 1,000). • Development of social accountability framework for the school sub-grants programs. Also, • Training of MOE staff on M&E, procurement, education planning and management. • Conducting of annual Joint Education Sector reviews each year (except the year of the Ebola epidemic in 2014). • Conducting of functional review of MOE's strategic alignment, work structure, administrative efficiency, and resource utilization, which identified gaps in capacity. Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) However, • A teaching and learning materials utilization assessment was not completed as planned, although at the time of ICR preparation, fieldwork for the assessment had taken place using alternate sources of funding. • The transfer of financial management capacity from the project's financial management unit to the MOE did not take place, due to continued weak capacity. The ICR (page 22) does report that measures to support knowledge transfer from project staff to MOE staff were included in terms of reference for international consultants; for example, as a condition for the Bank extending a no-objection to the international procurement specialist's contract extension, the specialist was required to prepare training materials and a lesson plan and hold regular training sessions with MOE procurement staff. • The Center for Education Management, which was to absorb project staff (and help retain capacity) was not established due to lack of funding. Outcomes • 100% of schools were constructed according to standardized school construction guidelines under the project. This surpassed the target of 75%. • Procurement of textbooks and teacher guides for grades 5-9 was verified at 100%. This achieved the target of 100%. Of these, 98% were also verified as having been distributed. This surpassed the target of 85%. • 72% of school grants were implemented in accordance with guidelines. This surpassed the target of 30%. 92% of sampled schools in the beneficiary survey confirmed they had received the school grants, and schools were "largely satisfied" with the grants although there were widespread concerns regarding delayed disbursement and the lack of continuity in the school grants program. • 2,579 schools were signatories to their own bank accounts, signifying increased capacity. This surpassed the target of 1,000. Achievement is rated Substantial due to evidence of increased capacity, including implementation of measures to increase accountability that achieved targets, at both the central MOE level and at the school level. Revised Rating Substantial PHREVISEDTBL 5. Efficiency The PAD (Annex 9) provides a discussion of economic issues in the education sector, including the impact of Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) schooling on children's health and nutrition outcomes, the distribution of public spending on education, and household spending on education. However, the discussion is primarily focused on sector-wide issues, rather than a project-specific analysis of efficiency in the use of project resources. The ICR (Annex 3) provides more project-specific data analysis, including a comparison of the cost of the key project outputs (school construction, textbook procurement, and school grants) versus the cost of similar outputs for other donor projects within the country. For school construction, the project adopted the high-quality standard for school design set by the government for the entire country; this standard had been set in response to previous experience in which lower-cost schools using local materials did not meet quality standards and were found to have incurred high maintenance costs. The per-classroom unit cost was US$ 37,166 under the project, compared to US$ 40,000 for schools constructed under the government's Education Pooled Fund. However, again, due to the higher school design quality standards, this per-classroom unit cost was higher than the US$ 20,000 to US$ 25,000 reported for similar low-capacity, fragile countries including Benin, Burundi, Cote d'Ivoire, and northern Uganda. More detailed information to verify comparability of outputs is not provided. Similarly, textbooks procured for this project were of relatively high quality (color printing, better paper quality). The per-textbook unit cost was US$ 4 under the project, which was similar to the US$ 4.58 reported for other government-procured textbooks, but higher than the per-textbook unit cost reported for similar efforts in other sub-Saharan countries: Sierra Leone (US$ 0.97 - 1.66), Niger (US$ 1.10 - 1.40), Eritrea (US$ 0.60) and Mauritania (US$ 0.73). The ICR also suggests efficiency in implementation due to the textbook distribution system that distributed materials directly to schools (hence bringing down the unit cost of learning materials and lessening risk of illicit sales), although no specific data are provided. However, in light of the overall context of a significantly reduced project scope despite the same project funding envelope, in addition to a project period extension of more than two years beyond the original project period, overall project efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) Project under original objectives - Unsatisfactory The outcome is rated overall Unsatisfactory due to significant shortcomings in the relevance of project design and in the achievement of the three project objectives. The relevance of the project objectives is High while the relevance of the project design is rated Modest due to the significantly changed policy environment and the complex design that made it unlikely to be achieved within the project time frame. Achievement of the objective to increase access to basic education with a focus on poor areas is rated Modest due to lack of sufficient evidence particularly with regard to reaching the poor. Achievement of the objective to improve conditions of teaching and learning is rated Modest due to significant shortcomings in implementation of key outputs. Achievement of the objective to improve school management and accountability is also rated Modest due to significant shortcomings in implementation of key outputs. Project under revised objectives - Moderately Satisfactory The relevance of the project objectives is High while the relevance of the project design is rated Substantial. Achievement of the objective to improve management capacity and accountability at the central and school levels is rated Substantial due to evidence of increased capacity to ensure provision of education materials, provide financing at the school level, and the completion of key outputs that likely contributed to improved management and accountability. Efficiency over the entire project period is rated Modest. Although unit costs for key outputs were comparable to other government-funded outputs, the overall context of a significantly reduced project scope despite using the same amount of project resources and over a longer project period indicate Modest efficiency in the use of resources. Overall outcome - Moderately Satisfactory According to harmonized OPCS/IEG guidelines, the outcome of a restructured project is assessed according to the amount of the grant that disbursed before and after the restructuring. In the case of this project, US$ 7.26 million, or 18.2%, of the grant had disbursed t the time of restructuring in September 2012. The Unsatisfactory rating has a weighted value of .364 while the Moderately Satisfactory rating has a weighted value of 3.272. Therefore, the combined outcome rating has a value of 3.636, or Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating The government remains strongly committed to the objectives presented in the project, as reflected in its updated sector strategy and the increased share of education recurrent expenditure as a share of the overall recurrent expenditure. The latter included funding to the MOE for a second tranche of school grants, although the amount provided was less than anticipated due to the Ebola virus epidemic. A new Global Partnership for Education grant is being prepared that will continue the school grants and MOE capacity building activities, while also addressing the teacher and personnel management issues. However, fiscal constraints are likely to continue due to the overall weak macroeconomic situation. There is limited, or no, budgetary allocation for Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) maintenance of schools and limited human resource capacity to maintain schools. Supervision visits also revealed that some schools lacked qualified teachers, in part due to lack of resources. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance a. Quality-at-Entry The project design built upon comprehensive analysis from the country sector report (2010) as well as experience from the donor-supported Education Pooled Fund (of which, the latter addressed poor accountability and management of the textbook distribution chain). The project design was also highly consistent with the government's medium-term education sector plan. The weak human resource capacity was recognized as a significant risk factor and was to be addressed with institutional capacity building activities; however, capacity was even weaker than expected and led to the first year of the project period being focused on staffing the Project Support Team (PST) and not on project implementation. The feasibility of contracting NGOs, which were to be used for community-driven school construction and textbook distribution, was not identified as a risk factor. The ICR (page 5) notes other shortcomings in project preparation which were detrimental to the project's implementation: for example, the ICR (page 4) notes that with the multiplicity of activities, "the PST, and those responsible for the implementation of subcomponents, were not always aware of their responsibilities"; also, there was substantial overlap in donor-supported projects including in the areas of early childhood development, school health, systems development for M&E, and payroll/personnel management. The results framework and M&E arrangements were overall adequate. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The Bank provided intensive supervision support (with twelve supervision reports prepared over the project period and weekly/bi-weekly conference calls with the Project Support Team during the Ebola crisis period) in response to the numerous implementation challenges arising from the project. Implementation support was provided primarily through supervision missions (staffed by education, fiduciary, safeguards, and architect specialists), rather than through field-based staff, due to the minimal Bank presence in the country for certain portions of the project period. The mid-term review meeting was moved up from the original date due to implementation concerns; and the Level I restructuring paper (Project Paper, 2012) provided an extensive description of implementation status and risks for the revised project. Fiduciary matters as well as safeguards were managed effectively. However, while the results framework was revised in accordance with the project restructuring, including dropping of indicators that were no longer relevant, the ICR (page 15) notes that the revised key indicators were more focused on measuring completion of outputs rather than capturing outcomes in the form of actual improvements in capacity, which the ICR suggests could have been measured via Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) qualitative surveys. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance Given the long preceding period of civil war, the project period was marked by relative stability in the political environment. Presidential elections took place in October 2011, which led to political transitions in key Ministry positions and, as noted in the ICR (page 12), "multiple reshuffling of political appointees in government disrupted project implementation." The ICR (page 11) notes that the lack of selectivity in the project design - which also led to complexity in implementation- may have been a reflection of the less conducive political environment: the country had been previously rejected as a Education For All grant recipient and was therefore under pressure to present a wide-ranging and ambitious program to leverage further donor support. Also, the tightened fiscal context brought on by the Ebola crisis led to less financial support for the school grants program. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Project Support Team within the Ministry of Education was the primary implementing agency. This project represented the first operation in the education sector since the end of the civil war that would be financially and operationally managed by the government. However, there was weak implementation capacity that, while fully recognized as a project risk, was "even weaker than initially assessed at appraisal " (ICR, page 4) and therefore the first year after project effectiveness focused primarily on capacity building while project disbursements remained low. Although implementation performance improved with the reduced project scope and efforts were made to ensure skills development and knowledge transfer from international consultants to local MOE staff, there remained some shortcomings in achieving capacity improvements (i.e. establishing of Center for Educational Management). Fiduciary and safeguard responsibilities were overall satisfactorily conducted. Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The project results framework was extensive, having been directly derived from the broader education sector plan. Indicators were overall measurable and relevant, including identified baseline and target figures, but were numerous given the capacity constraints. An existing education management information system (EMIS) was expected to be able to provide data, and the project design included development of a school construction database and various studies planned to provide additional data and qualitative evidence. b. M&E Implementation The ICR (page 15) reports that up-to-date EMIS data was not available as expected at the project start, thereby complicating planning for key outputs and leading to implementation delays. The results framework and key project indicators were modified during project restructuring, although the ICR (page 15) suggests that the revised key indicators were focused more on measuring completion of outputs than on capturing actual improvements in capacity, which might have been measured through qualitative surveys. c. M&E Utilization Data from the school construction database was utilized to track progress and in some cases, amend or cancel contracts. Textbook procurement and distribution was regularly reported on and monitored. However, several end-of-project studies to measure project impact, for example, textbook utilization, were not completed in time to inform the ICR. M&E is rated Modest overall due to shortcomings in choice of revised indicators (i.e. more output- rather than outcome-oriented), lack of up-to-date data, and shortfalls in implementing evaluative studies. M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was classified as a Category "B" project due to potential negative impacts from school construction. Safeguard policies on Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12) were triggered. An Environmental and Social Management Framework as well as a Resettlement Framework were prepared as part of project appraisal. Recommendations included capacity Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) building activities and consultations with potentially affected persons. When the construction of early childhood centers was added to the Grant Agreement, the Frameworks were also amended correspondingly. According to the ICR (page 17), the MOE obtained official land deeds and tribal certificates for all construction sites. The supervision team confirmed in a November 2013 mission that construction activities were in compliance with safeguard policies. No further safeguard problems were reported thereafter. b. Fiduciary Compliance Financial management: The ICR (page 17) reports that the government was in full compliance with all financial covenants, and audits were submitted regularly and in a timely manner. However, auditors noted in the post- audit report that the fixed assets register had not been adequately maintained and updated and that reconciliation items identified in bank account reconciliation statements had not been resolved in time. There was also one allegation of misuse of funds in October 2014, involving fraudulent and corrupt practices by the Project Support Team's junior project accountant. The Bank team reported the incident and also took measures to strengthen controls over such activity. The amount in question, US$ 34,810, was returned to the Bank, while the staff member was dismissed from the project and as of the time of the ICR the case was pending judgment from the government courts. Procurement: The ICR (page 18) reports that procurement at the central level was adequately implemented and documented, although there were significant delays in filling the procurement staff positions in the Project Support Team and in hiring a management firm to oversee school construction. Bank supervision visits confirmed that school grants were largely in compliance with guidelines, although record keeping was not comprehensive. c. Unintended impacts (Positive or Negative) None noted. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) Risk to Development Substantial Substantial --- Outcome Moderately Moderately Bank Performance --- Satisfactory Satisfactory Moderately Moderately Borrower Performance --- Satisfactory Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons Lessons drawn from the ICR: • Direct distribution of education materials to schools can enhance accountability and effectiveness (and perhaps efficiency) of the delivery chain but can also be hindered by lack of timely access to EMIS data. In the case of this project, decentralized education officers and school principals were mobilized to participate in textbook distribution; however, there were discrepancies in the number of schools and enrollment data which led to inaccuracies in delivery of materials. • School construction outputs are less likely to lead to sustained education outcomes (i.e increased enrollment) without a corresponding maintenance plan. In the case of this project, the lack of budget allocated to school maintenance and the lack of human resource capacity to maintain schools is contributing to significant risk in the development outcomes. 14. Assessment Recommended? No 15. Comments on Quality of ICR The quality of the evidence is overall satisfactory (for the revised objective). Given the challenge in measuring improvements in "management capacity" and "accountability", the ICR provides both quantitative data from the project indicators and qualitative information in the narrative to supplement the evidence. There is also extensive discussion of implementation challenges, which is useful for similar fragile contexts. There was some inconsistency in the assessment of government's impact on project achievements/shortcomings, for example with regard to the political environment (stable macro-political environment (ICR, page 12) vs. lack of enabling environment for selectivity in project design (ICR, page 11) and fiscal support (the government took Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR - Education GPE Program (FY11) (P117662) over full financial responsibility of the school grants program a year ahead of planned (ICR, page 14) vs. due to fiscal constraints, the government could not support the school grants program without external financial assistance (ICR, page 19). a. Quality of ICR Rating Substantial Page 17 of 17