Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) Report Number: ICRR0021714 1. Project Data Project ID Project Name P123315 Strengthening Institutional Capacity and Country Practice Area(Lead) Djibouti Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H7770 31-Aug-2017 5,449,394.61 Bank Approval Date Closing Date (Actual) 12-Jun-2012 31-Aug-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 6,000,000.00 0.00 Revised Commitment 6,000,000.00 0.00 Actual 5,449,394.61 0.00 Prepared by Reviewed by ICR Review Coordinator Group Anthony Martin Tyrrell Eduardo Fernandez Joy Behrens IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project objective as stated in the Financing Agreement (p. 5) of August 21, 2012 is to strengthen the institutional capacity of the Ministry of National Education and Vocational Training (MENFOP) for improved management of the education system. The same objective is repeated in the Project Appraisal Document (PAD, p. 3) and the Implementation Completion Report (ICR, p. 1). Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) There was no change to the project objective or PDO indicators during implementation; however, the target for one of the PDO indicators was revised upwards as part of a May 2017 restructuring, which also extended the closing date by 12 months. The ICR provides the following rationale for the upward revision: (a) despite initial delays, MENFOP had made significant progress after the mid‐term review in the development of Annual Work Plans (AWPs) across all its departments, and was confident of enhanced performance by the closing date; and (b) MENFOP also saw the higher target as confirmation of its sustained commitment to a results‐based way of working. A split evaluation is not performed here as the altered target was made more ambitious to reflect the additional time made available for project completion. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Reinforcing institutional capacity and performance of the education system (Planned: US$4.0 million; Actual US$3.6 million) – this component reinforced the capacity of the MENFOP at the central, regional, and school levels to improve performance and management of the education system through technical assistance, training, and the provision of furniture, vehicles, and equipment, as detailed in the Procurement Plan. Component 1 was implemented through four sub-components as follows:  Subcomponent 1.1. Improving management and accountability provided technical assistance, training, and equipment to various departments at the central, regional and school levels to help the MENFOP in: (a) defining roles and accountability structures and developing procedures that introduce performance‐based management tools; (b) strengthening planning and management of financial, material, and human resources; and (c) supporting the systematic collection, analysis, and sharing of information, and harmonization of the various information systems to monitor and evaluate the performance of the MENFOP in delivering education services.  Subcomponent 1.2. Supporting the establishment of a new governance structure for TVET provided technical assistance, training, and equipment to the new Directorate for TVET that was established before project effectiveness. Through support to this directorate, the project promoted the establishment of a national authority for TVET with public and private sector participation that facilitated the links between the education system and the labor market.  Subcomponent 1.3. Strengthening efficient management of textbooks. This subcomponent supported the National Education Research and Information Production Center (CRIPEN) in the management of textbooks through technical assistance and training, in planning, costing, budgeting, procurement, stock control, distribution and maintenance of textbooks, and in implementing a Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) sustainable system of cost recovery which does not exclude the rural poor from access to learning materials. It provided technical assistance to conduct studies to assess the use of textbooks before and after the project and sought to conduct a pilot experiment on the use of digital textbooks through e‐tablets at the secondary level. The project also provided IT equipment and minor rehabilitation for the offset printing and storage facility.  Subcomponent 1.4. Enhancing project management, which strengthened the capacity of MENFOP to manage education projects, in general, and this project, in particular, by providing technical assistance and training to the Project Implementation Unit (PIU) in procurement, financial management, and monitoring and evaluation. It also provided technical assistance to support the Project Steering Committee. The project financed furniture, equipment, vehicles and incremental operating costs for the PIU. Component 2: Increase the quality of the learning environment (Planned US$2.0 million; Actual US$1.7 million) This component supported activities to improve the quality of the school environment by ensuring minimum learning and working conditions through: (a) the upgrading and rehabilitation of the technical colleges and provision of didactic equipment; and (b) the completion of the Hodan 3 lower secondary school and provision of equipment and learning material. We note that this second Component is unrelated to the PDO, which focuses on improved system capacity. This activities supported in this component are more closely related to improving education quality and/or conditions. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Estimated cost at appraisal was US$6 million of which US$5,449,394.61 was disbursed at project close. Financing: This Specific Investment Loan was supported with IDA funding of US$6 million. Borrower Contribution: There was no Borrower contribution planned or made. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) Dates: The project was approved June 12, 2012 and became effective seven months later on January 15, 2013. The Mid-term Review was completed March 7, 2016. The project closed on August 31, 2018, exactly one year after the original planned closing date. The project underwent two level two restructurings:  First restructuring (May 31, 2017): Change in Results Framework; Change in Components and Cost; Change in Loan Closing Date(s); Change in Legal Covenants; Change in Institutional Arrangements; Change in Implementation Schedule; Other Change(s). The restructuring extended project closing date by one year (to August 31, 2018) to allow for completion of project activities (mitigating early delays), and a new activity was introduced under component 1 (i.e., the provision of school grants to 30 schools to extend the results‐based approach (RBA) down to the school level) to empower schools to identify key constraints to learning (e.g., remedial programs, minor repairs, equipment, supplies, etc.), develop and manage improvement plans and to be accountable for these activities. The schools were selected among the poorest communities, and with the lowest education performance. The restructure also involved revision of the results framework to shift the end target dates for all indicators in line with the revised Closing Date and to adjust some end targets, again reflecting the longer time frame for delivery. Adjustment to the overall funds available and component costs were made to reflect losses due to exchange rate fluctuations. Funds were reallocated between activities to reflect support from other donors or the government for certain activities and the cost of revised activities: the cost of Component 1 was reduced from US$4 million to US$3.6 million equivalent and the cost of Component 2 was reduced from US$2 million to US$1.7 million equivalent.  Second restructuring (November 14, 2017): Change in Results Framework (RF); Reallocation between Disbursement Categories Change in Disbursements Arrangements; Change in Institutional Arrangements. This restructuring was to permit an increase in the school grants funding envelope from US$50,000 (as stated in the first restructuring) to US$150,000, and was necessary due to the government’s decision to include lower secondary schools that had more significant resource constraints - 21 primary schools and 9 lower secondary schools were to benefit from school grants. The additional US$100,000 came from the planned household survey for which other funding had been identified through the Resident Mission. The RF was amended to add secondary schools to the target. 3. Relevance of Objectives Rationale The project contributed to the priorities and objectives highlighted in Djibouti’s National Development Initiative (INDS), the National Education Strategy 2010-19, and the associated National Education Action Plan (PAEFPT) for 2011-2016. The PAD notes that Djibouti acknowledges the important role of education – in 2010, 19.8 percent of the national budget went to education. The INDS highlights improvement in management of the sector to increase access, quality and relevance of education as a priority. The National Education Strategy focused on improving the efficiency of the education system and the quality and Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) relevance of education with a focus on results. A series of diagnostic studies that supported the strategy included in-depth analyses of all levels of education as well as study of the administrative structure and its management capacity, including an organizational audit of the Ministry of Education. The project was designed to support the National Education Strategy’s emphasis on access, equity, quality, focus on results and good governance as major priorities for the sector. The project was also in line with the 2009-12 Country Assistance Strategy (CAS) that highlighted improvements of the education sector as one of the key national priorities. The Performance Learning Review of the 2014-17 Country Partnership Strategy (CPS) brought about a change in focus for Pillar 2 from “strengthening the business environment” to “strengthening public and private sector capacity for service delivery”, thereby reconfirming the relevance of the project objective. The PDO remains consistent with the World Bank CPS and the Government's Education Sector Strategy 2010‐2019. Relevance was reconfirmed in the Systematic Country Diagnostic (2018) that identified capacity and governance constraints in Djibouti and recommended continued focus on these areas. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective to strengthen the institutional capacity of the Ministry of National Education and Vocational Training (MENFOP) for improved management of the education system Rationale The theory of change (ToC) focused on improving education sector management by providing support to several priority areas in the sector including training of staff at various levels, purchase of equipment to strengthen the institutional capacity for improved management of the education system, and a focus on instituting results‐based annual work plans, a governance structure for technical and vocational training, more efficient management of learning materials and increasing the capacity to manage education projects. The results framework was built on output and activity type indicators, including the PDO indicators. This was in recognition of fragile country context and limitations in capacity. The indicators suggest activities and output that one would expect to be associated with enhanced capacity in the Ministry - the introduction of a certain discipline in planning - but they do not demonstrate enhanced capacity i.e., the production of a work plan is no guarantee that the plan contains the appropriate content and, even if it did, no guarantee that required actions are implemented to a satisfactory standard. However, taking the context and conservative build of the project as given, targets for the two PDO indicators - percentage of departments have Annual Work Plan (AWP) developed and agreed and approved; and, percentage of AWP implemented – were exceeded as were those for six out of the eight intermediate indicators. The ICR also presents information regarding other project achievements relating, for example, to institutional strengthening and private sector participation as set out below. Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) Intermediate Indicators and Outputs Timely production and dissemination of semi‐annual reports on school performance. Original target (8) increased to 10 at the first restructuring and actual was 12. Annual statistical reports with key indicators. Original target (4) increased to 5 at the first restructuring, and that target was achieved. Number of partnerships with private sector firms to train and/or recruit students has increased from 1 in 2011 to at least 6 (increased from 4 at first restructuring) by the end of the Project. The actual achievement was 15. The level of satisfaction of TVET students and firms has increased by at least 10 percent (decreased from 30 percent at first restructuring) by the end of the Project. The original indicator included the qualifier, (compared to baseline); however, there was no baseline available. Actual rates of satisfaction 85.5 percent for firms, and 89 percent for students. Number of schools receiving school grants (which was added at the second restructuring) had a target of 30 that was achieved. The National Education Research and Information Production Center (CRIPEN) consistently collected information on number of books sent, delivered and received at the primary, lower secondary and secondary levels, and tracked the textbook stock at the beginning and the end of the school year, both at the center as well as in schools. A total of 204,477 textbooks distributed in 2017/2018. Number (stock) of books available in schools at the beginning of the school year (June 2017): 83 198 Number (stock) of books available at the end of the school year (June 2018): 13 6039 Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) Total number of students using new computer rooms in Hodan 3 schools. The original target (2,200) increased to 2,500 at the second restructure. Actual achievement was 2,861. Total number of students using didactic equipment in technical colleges increases. Actual achievement was 818 against a target of 800. Under Component 1 (Reinforcing Institutional Capacity and Performance of the Education System) the following outputs are noted: 626 Ministry staff trained, including, for example, 97 trained on planning, management, budgeting and MTEF, 75 staff were trained on applying a RBA and management, 9 statistics staff at central and regional levels, and 130 school heads trained; administrative manual with clear roles and responsibilities, and performance management tools produced; dashboard for Minister established; online depository of workplans established; geo‐mapping included in all schools; Management information system harmonized; National directorate for TVET supported through capacity building and equipment; over 600 TVET students trained as part of the partnerships; 132 private firms participated through the 15 partnerships; 204,477 textbooks distributed 2017/2018; annual data on textbooks printed, delivered and received by schools, as well as stock at beginning and end of school year available. Under Component 2 (Increase the Quality of the Learning Environment) the following outputs are noted: 5 technical colleges rehabilitated and equipped; 6 computer rooms constructed; 1 multipurpose room constructed; 2 housing quarters for school director and assistant school director constructed Outcomes / PDO The target for the first PDO indicator - percentage of departments with Annual Work Programs (AWP) agreed and approved by the beginning of the fiscal year – was increased from 75 percent to 90 percent at the first restructuring (May 2017) to reflect the extended timeframe for the project. At project close, 96 percent of departments had an AWP agreed and approved. By 2017, the AWPs were based on results and their introduction helped the Ministry to identify and be accountable for clear targets directly linked to the Education Sector Plan (ESP, 2017-20) indicators, ensure that the sector priorities of increased access, improved equity and retention were embedded across all ministry departments’ work plans, and strengthen its capacity to coordinate and manage different interventions by donors and government agencies to avoid duplication and therefore maximize synergies . In 2018, a total 48 out of 50 departments produced results based AWPs. Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) The target for the second PDO indicator - departments with an AWP that was implemented - was 70 percent, and the actual achievement was 90 percent, which suggests an upward trend in the ministry’s capacity to implement activities it set out to achieve at the beginning of the year. The ICR reports that the Ministry was able to complete 572 activities out of 697 in 2018 across all areas, including human resource management (professional development, deployment of teaching and non‐teaching staff); student and sectoral performance assessments (administration and analysis of key exams, monitoring and evaluation, development of annual statistical reports harmonization of information systems); planning and policy making (development of a pre‐school strategy, introduction of ICT into schools, review and update of curriculum, textbook management, project management, school management, governance of TVET, improvement of the learning environment etc.). The ICR also notes that the Ministry’s strengthened capacity to manage the education sector is also supported by undertaking two early grade math assessments (EGMA) for Grade 2 in 2017 and 2018. The 2018 assessment revealed that most teachers applied the new teaching practices and CRIPEN developed supplemental math booklets based on the 2017 results, which led to children’s increased performance. Gross enrollment rates went from 78 percent to 89 percent 2016-19, and the number of schools built increased from 133 to 141 over the same period. Training and expert support to the Ministry to engender a mentality of RBA formed a significant part of the project and that units within the ministry now focus AWPs on results rather than (as in the past) completed actions, thereby becoming more transparent and accountable. The M&E system was enhanced with online tools for better monitoring of sector performance and the development of a dashboard helped to focus units on results and link their work plans to the ESP. The ICR also notes that the quality of materials coming from CRIPEN has improved significantly and the different trainings and capacity building activities strengthened assessment and exam capacity of relevant directorates and units which has led to increased interest in rigorous student evaluation approaches. Strengthening the TVET directorate helped create “incubator” centers where young people choose a profession/skills, receive three‐month training on business development and life skills and 9 months of center‐based practical experience. The ICR also reports that, although the project did not mobilize private sector financing, it did mobilize private sector participation, which was facilitated by the establishment of and support to the National directorate for TVET, which includes private sector representatives. Private firms were also involved in improving the quality and relevance of TVET. The project also supported MENFOP in its data collection and analysis such that all major reports can be provide gender disaggregated data. The ICR reports that several of the key department heads were female and activities supported through the project were led by women (the directorate for inspection, the evaluation department and teacher training center), 27 percent of all school heads receiving inputs on RBA were female, and 40 percent of students benefitting from school grant interventions were girls. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) Rating Substantial OVERALL EFF TBL OLD Rationale Overall Efficacy Rating Substantial 5. Efficiency The PAD (p.12) stated that the project was not amenable to a cost-benefit analysis as the principal benefits from the project are not quantifiable noting the successful implementation of the project is expected to improve efficiency in the management and use of resources. The ICR (p.17) states that all activities were more or less implemented within the extended timeframe and indicators were met or exceeded by project closing notwithstanding the availability of a reduced budget due to exchange rate fluctuations. On the other hand, implementation progress was mixed. Rated satisfactory for the first two of 14 ISRs, implementation progress dipped to moderately satisfactory for the next three ISRs and moderately unsatisfactory for the subsequent three ISRs before recovering to moderately satisfactory for all but one – where implementation progress dipped once more to moderately unsatisfactory - of the remaining six ISRs. The ICR suggests that project outcomes and outputs point towards more efficient use of resources, that the project leveraged additional donor funding and promoted greater sector alignment, which, taken together points to increased efficiency. The ICR recognizes that delays in implementation prohibit a high efficiency rating. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) 6. Outcome The relevance of the objectives is High given alignment with country context and strategy and with the Bank’s country strategies over time. Efficiency is rated Substantial reflecting solid achievement of outputs with a reduced budget, but also reflecting delays in implementation. Efficacy is rated Substantial given the achievement of almost all targets. The overall outcome rating is therefore Satisfactory, consistent with only minor shortcomings in the project's preparation and implementation. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Notwithstanding some ongoing implementation challenges, the results based approach (RBA) is now used throughout the ministry, which is the only ministry in Djibouti to either apply a RBA or have such detailed information made publicly available. The ministry has also invested to further improve the ICT‐enabled system so that it can become fully web‐based and show information in real time. The ministry has also requested to continue and expand the results based approach under a planned basic education project, which will allow for further consolidation of achievements made under this project. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank worked closely with the government and other donors to support the implementation of government strategy and action plans (as referenced above under Relevance). The Bank agreed with government that it would support strategic components of reform, although certain activities under Component 2, such as finishing and equipping the Hodan 3 appear more pragmatic than strategic in intent. In fact, Component 2 is more closely tied to an objective such as 'improving learning conditions' rather than the PDO as stated which focused on development of system capacity. The project was also designed to respond to findings from an organizational audit completed in 2009 under an earlier Bank project (Second School Access and Improvement Program (P086994)). The audit had identified weaknesses in the decision‐making system process, and in the process of planning and conducting reforms and in implementing corrective actions. The project also built on lessons and achievements of the earlier multi‐donor funded Primary Education Support Program, EFA‐FTI Catalytic Fund. The project was conservatively designed taking into account the country context and local capacity. Indicators were output and activity oriented and were unambitious as evident by the fact that targets were later raised despite reduced budget (given exchange rate movements) albeit with an additional year of Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) implementation agreed through restructurings. The ICR notes some shortcomings in design of the results framework i.e., some of the results indicators were not clearly specified and that ambiguity meant the PIU and the ministry were not fully geared up to measure the results of some project activities relating to TVET student and firm satisfaction. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank maintained close contact with the project and counterparts throughout. An early change in ministerial leadership slowed implementation, but the Bank team continued to promote dialogue and, as noted in the ICR, find common ground that would assuage new political priorities while supporting the project’s objectives. Fourteen ISRs were implemented and there was consistent reporting on financial management and procurement progress. The Bank also hired a consultant (July 2017) who was based at the Djibouti Country Office and who worked with the PIU team to build capacity - frequent staff turn‐over and prolonged vacancies had weakened PIU effectiveness to implement project activities. The two restructurings were well managed and the additional time for project implementation meant that planned activities were completed e.g., full completion of Hodan 3 lower secondary school, procurement of a used Offset Printing Machine for CRIPEN, 30 pilot schools received increased financing for their school improvement projects. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The Theory of Change set out how activities and intermediate outcomes would lead to outcomes for strengthening the institutional capacity of Ministry for enhanced management and planning. Design included measures to ensure effective M&E such as the development of an internal M&E framework, guidelines, manuals and related capacity building and training of respective staff, as well as strengthening information management systems with clear linkage to AWPs. The project also envisaged a pre‐ and postsurvey of schools to confirm receipt of textbooks by students and satisfaction by TVET students and firms. That said, PDO indicators related to Component 1.1 were framed only with reference to the development and implementation of AWPs noting, as in the Quality at Entry Section above, that indicators were relatively low level and intermediate reflecting the context and timeframe of the project. The ICR notes Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) that the indicator for TVET satisfaction rate was ambiguous and difficult to measure in the absence of a baseline. b. M&E Implementation The Ministry maintained regular reporting of project progress and efforts were made between the Ministry and the M&E consultant to resolve challenges relating to design of surveys for TVET students, firm satisfaction and textbook distribution. The ICR notes that project M&E benefited from the fact that capacity was supported by other projects for which the PIU was responsible. The results framework was strengthened with the addition of a dashboard to facilitate monitoring and reporting. The PIU produced detailed semi‐annual progress reports on project activities. c. M&E Utilization Improved M&E provided the government and the Bank with information to support a reasonable expectation that a one‐year extension of the project closing date would lead to the successful completion of remaining actions. The system also allowed calibration of targets to reflect the additional time and allowed for rebalancing of funding across activities to support additional funds to extend grants to more schools. The ICR notes that there continues to be areas for improvement in terms of data gathering, analysis and reporting, although it is proposed to address these as part of a new Bank/GPE‐supported project under negotiation. Semi‐annual progress reports were instrumental in updating the results framework, developing the restructuring proposals and measuring progress towards project objectives achievements. Changes to the survey instrument (pre/post) meant that results could not be compared. Survey information on textbooks received by students was not available at the time of the ICR writing but was provided by the TTL during IEG's review process and relevant results are reflected in the Efficacy section. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The PAD (p.15) notes that, based on an assessment by the team's safeguards specialists, and taking into account the limited financing amount as well as experience from earlier projects – School Access and Improvement Program (P044585), and Second School Access and Improvement Program (P086994) - the environment safeguards category for the project has been downgraded from B to C. The project did not trigger any social safeguards. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) b. Fiduciary Compliance Project preparation was informed by the organizational audit of the ministry that was supported by a previous Bank‐supported project and that set clear priorities and recommendations to improve education sector management. The current project focused on technical assistance and training for sector staff to strengthen implementation capacity through knowledge transfer and fiduciary functions of the PIU. The project used the existing project implementation unit which already had experience in managing Bank financed projects. The World Bank team assessed the financial management system for the PIU and identified risks associated with staff capacity, certain fiduciary management procedures, and the fact that the PIU was entrusted with several projects financed by other donors, which might have made external audit complex. Actions were agreed to address these potential shortcomings. Field‐based and Washington‐based staff carried out regular supervision missions and found that, overall, the system was robust. Financial covenants were complied with by and large in a timely manner, an adequate FM system and internal controls were in place, and the action plan was being implemented. Audit reports and Interim Financial Reports were submitted in a timely manner. An internal auditor performed independent and objective internal audit activities including inspections and verification of the use of funds at the sub‐project level. Procurement guidelines were generally followed, and procurement processes completed in a satisfactory manner albeit with some delays. Procurement decisions were transparent, but at times slow. The ICR reports one instance where the Bank procurement specialist was concerned about a potential conflict of interest on a bid for a construction project, which was referred to the Bank’s integrity unit but was found to be unsubstantiated. Notwithstanding, the Bank team strengthened its support to the PIU, and procurement performance remained acceptable through the end of the Project. c. Unintended impacts (Positive or Negative) No unintended outcomes or impacts are identified. d. Other N/a 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- High Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) 12. Lessons The ICR provides useful lessons of which IEG notes, in summary, the following: Sound analytical underpinning is critical to identifying key areas of intervention: the institutional audit of the Ministry carried out at the end of an earlier project was key to laying out priorities that led to the main design features of the current project. To effect progress towards results‐based management, it is important to align Annual Work Plans (AWPs) with the national strategy and the minister’s management needs. Such alignment gives a sense of clarity, importance and accountability to the staff who are responsible for the AWPs. Putting in place a dashboard for the Minister provided him with timely information on the status of activities on a timely basis, which helped improve accountability and the focus on results. Continued support is critical especially in a low capacity context: the hiring of a local, independent consultant who was located in the Bank’s country office in Djibouti was a key factor in ensuring end‐of‐project implementation success. IEG adds the following: This project demonstrates how management of the political economy is critical to successful project implementation. The project started out with strong political and administrative support; however, shortly after preparation, the project’s champion was transferred out of the ministry and this impacted progress. For example, regular donor meetings at the Ministry ceased, project progress and disbursement slowed. However, the Bank team remained in contact and dialogue with the new administration and, ultimately, worked with the Government to request a Level 2 restructuring that extended the project closing date by twelve months and allowed for the achievement of almost all results targets. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well written, and consistent in its internal logic. The ICR presented a concise presentation of what was a relatively uncomplicated project that included a succinct background piece and rationale for Bank Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Strengthening Institutional Capacity and (P123315) involvement together with a suitably simple theory of change that reflected the largely output-type focus of the project indicators. The ICR was candid in its assessment of the project, noting, for example, limitations in M&E design. The ICR also provided adequate evidence to assess achievement with reference to relatively simple output-type indicators, and provided additional information on related contextual developments, for example in relation to gender. Finally, the ICR presented a significant number of valuable and well-articulated lessons that were based in the project experience a. Quality of ICR Rating High Page 15 of 15