Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) Report Number : ICRR0020823 1. Project Data Project ID Project Name P125799 MA-Judicial Performance Enhancement Country Practice Area(Lead) Morocco Governance L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-81750 31-Dec-2016 15,800,000.00 Bank Approval Date Closing Date (Actual) 12-Jun-2012 31-Dec-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 15,800,000.00 0.00 Revised Commitment 2,387,139.73 0.00 Actual 2,088,396.37 0.00 Prepared by Reviewed by ICR Review Coordinator Group Malathi S. Clay Wescott Lourdes N. Pagaran IEGEC (Unit 1) Jayawickrama 2. Project Objectives and Components a. Objectives The Project Development Objective (PDO) is to strengthen the capacity of the Borrower's justice sector to deliver efficient, timely and transparent services to citizens and businesses through: (a) piloting of a participatory reform process involving judges, administrative staff, judicial auxiliaries and users in selected courts in the project area; and (b) strengthening the institutional capacity of the central functions of the Ministry of Justice and Liberties (MdJL) to better support and monitor the court system (Project Appraisal Document, PAD, p. 6-7 and Loan Agreement, p. 5). Page 1 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components Component 1: Improving Court Performance (appraisal, US$7.77 million; actual US$0.66 million). This component supported: preparing court performance improvement plans (PAPJs) following a participatory approach involving key stakeholders in the project area; organizing a functional separation between a “front office” and a “back office”; and upgrading and developing critical ICT tools that would allow systematizing and automating jurisdictional and administrative processes in the pilot courts. Component 2: Upgrading the Strategic Planning and Management Capacities of the MdJL (appraisal, US$3.89 million; actual US$0.52 million). This component was to improve the MdJL’s capacity to monitor the performance of the courts and provide them with adequate support, to attain the strategic objectives of Component 1. Overall, this Component was to provide support to the MdJL's Directorates entrusted with a cross-support function, and to the Superior Judicial Training Institute. Component 3: Project Management, Monitoring and Evaluation (appraisal, US$0.91 million; actual US$0.82 million). This component supported MdJL's capacity to manage, monitor and evaluate project activities. As such, this Component was to provide support to the Project Management Unit (UGP) responsible for coordinating day-to-day project activities and administering loan funds e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: At appraisal, the total project cost was estimated at US$15.8 million or EURO 12 million. The actual disbursement was US$2.09 million (ICR datasheet). Total project cost is alternatively given as $12.63 million and actual expenditure $2.03 million (ICR, Annex 1, p. 22).　The remaining amount was cancelled upon request by the Borrower in a letter from the Bank dated June 21, 2016 (ICR, p. 11). Project Financing: The IBRD Loan was approved for Euro 12 million (US$ 15.8 million equivalent). There were no other external sources of financing or Borrower contribution. Dates: The project was approved on June 16, 2012. Effectiveness was delayed until March 22, 2013 because of challenges in meeting effectiveness conditions. The project underwent a Level 2 restructuring on December 9, 2015, to move the location of one of the pilot courts of the program from Sidi Kacem to El Jadida, as the Sidi Kacem First Instance Court was to be included in a development program of the Council of Europe. The PDO, budget, scope of beneficiaries, and the locations of the other pilot courts remained the same. Page 2 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) 3. Relevance of Objectives & Design a. Relevance of Objectives The project’s objectives were relevant to country conditions at the time of appraisal and remained so at project closure. In 2011, in response to a call for curbing corruption and a more inclusive development process, Morocco adopted a new Constitution strengthening the Parliament and the Judiciary, empowering local government, and promoting a more open governance system and public administration. The objectives were relevant to the Government’s 2012-2016 program, which, under its second pillar, reflects consolidating the rule of law, strengthening good governance, democratic participation, and advancement of regionalization and decentralization, in the context of accountability and true citizenship. The objectives were pertinent to the World Bank’s Country Partnership Strategy (CPS, 2014-2017), which proposed to scale up its support for Morocco's open governance and green growth agendas.　The CPS is built around three results areas: promoting competitive and inclusive growth; building a green and resilient future particularly with respect to the third results area of strengthening governance and institutions for improved service delivery to all citizens. Rating Substantial b. Relevance of Design The project’s components and planned activities were broadly linked to the PDO. Preparing PAPJs involving a participatory approach, where each PAPJ included modernizing court management, standardizing application procedures to reduce delays, and increasing regular interaction with court users were intended to enhance court performance in the project areas. Upgrading the strategic planning and management capacities of the MdJL was expected to improve MdJL’s capacity to monitor the performance of the courts. However, the activities were not sufficient to achieve the multi-dimensional objectives of efficient, timely and transparent services to citizens and businesses. 　 The design was envisaged to be kept flexible to support implementation of the National Dialogue, which was ongoing at the time. The pilot approach was also appropriate given the evolving justice reform agenda of the government. However, for the project’s bottom up approach to be effective, broad consultations with pilot courts and other stakeholders at the local level should have been started at preparation and not left until implementation. In short, the approach was unrealistic and overly ambitious, given that the sector had no experience with bottom-up approaches to justice reform and involvement of civil society organizations, and in the context of a highly centralized operating environment. Page 3 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Strengthen the capacity of the Borrower's justice sector to deliver efficient, timely and transparent services to citizens and businesses. Rationale Morocco’s efforts in modernizing its court system were motivated in large part by the Association Agreement (2000) creating a free trade area between Morocco and the EU, and requiring Morocco to harmonize its legal framework with that of the EU. In justice reform, the country had established specialized courts. Despite these improvements, the justice sector was perceived as being inefficient and lacking trust of the citizens. Improving efficiency remained a key challenge. This included reducing the backlog of cases, staffing the justice sector and using existing resources efficiently, and strengthening organizational management, including increasing MdJL’s capacity to monitor court performance and follow modern management techniques. To achieve the objective, the project intended to pilot reform processes in selected courts, and strengthen the institutional capacity of MdJL to support and monitor the court system. 　 Outputs and Outcomes Except for the analytical work, none of the outputs were delivered. As a result, none of the four PDO outcome indicators were achieved. One intermediate outcome indicator (out of 13) on structuring two MdJL budgets around the updated Medium-Term Expenditure Framework, was achieved, but independently of the project (ICR, Data Sheet). Following effectiveness delays and a slow start, in 2014, consultants were hired to deliver analytical work (by 2015), which needed to be completed prior to launching project activities. According to the ICR, this work is unlikely to be of benefit due to weak design and client perception that the consultant profiles and their work were of little value. By project closure, no pilot courts had set up consultations. Rating Negligible PHREVDELTBL PHREVISEDTBL 5. Efficiency The project disbursed only US$2.09 million compared to the original commitment of US$15.8 million or Page 4 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) roughly 13 percent of the original project cost. Except for the analytical work that is unlikely to be of value as mentioned in Section 4, none of the outputs were delivered. However, while the project had negligible outputs, roughly 90 percent of the project management cost was disbursed compared to the appraisal estimates, which suggests inefficient use of project resources. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the PDO was high, and relevance of design was modest; efficacy and efficiency were both rated as negligible. 　The design did not include key activities that were necessary for the project’s bottom-up approach to succeed, and to achieve its objectives. a. Outcome Rating Highly Unsatisfactory 7. Rationale for Risk to Development Outcome Rating As there was negligible achievement, there is negligible risk to development outcomes. a. Risk to Development Outcome Rating Negligible Page 5 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) 8. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant and built on background analysis. However, the Bank proceeded to prepare the project before the government’s own justice reform agenda was finalized. While the project was designed to be flexible to accommodate possible changes during implementation, the flexible design approach was not adopted with agility as will be noted later in the quality of supervision section. Overall, the project was not ready to be implemented at board approval and resulted in long effectiveness delays. The project was based on the bottom-up approach in an environment with a top-down institutional culture. 　 This approach, which was intended to drive project implementation, was overly ambitious, given the sector had no previous experience with bottom-up approaches to justice reform and civil society organizations in this process. At a minimum, the project could have laid the foundations for this approach by building reform momentum from the bottom-up during project preparation. This was left until project implementation, and did not happen. The lesson identified from a previous operation, Bank-supported Legal and Judicial Development Project (P063918) to involve a wide range of stakeholders as early as possible in the project, was not taken on board. During project preparation, the government’s priorities shifted away from this project (with the change of leadership in MdJL in early 2012), and focused on implementing the National Dialogue to develop the Charter of Reform for the judicial system. The PAD identified overall implementation risk as moderate, which may have been overly optimistic. The stakeholder and capacity risks were correctly identified as high and did materialize; however, the rating of the program and donor risk as low turned out to be unrealistic. Overall, the design was strategically relevant, and could not have anticipated the more attractive support offered by the EU during implementation. However, there were major shortcomings in quality of entry, including insufficient consultation with stakeholders during preparation, and other design features not helpful in adapting to the evolving political environment. Quality-at-Entry Rating Unsatisfactory b. Quality of supervision The Bank did not make much effort to resolve project issues and risks. The Task Team Leader (TTL) was changed between Board approval and effectiveness. According to the ICR, there were no regular follow-up video conferences, the supervision reporting was not done in a timely manner- for example, all seven Implementation Status and Results Reports were late, with three being significantly late (i.e. one over 20 weeks). As mentioned above, the Bank was not proactive in revising the project. For example, although implementation problems were already identified early in 2014, a mid-term review could have beeen carried out to address issues in a systematic manner. The team could have proceeded with a major restructuring even without undertaking an MTR. While a simple and very limited restructuring was submitted on June 24, 2015, it took over five months to be approved. The project dragged on without any progress for three years until the closing date. The availability of grant-based EU funding for the justice sector was a significant factor in the Page 6 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) declining Government commitment to the project. The limited government ownership could have been mitigated if the Bank made the effort to build synergies with the EU funded operation. The ICR states that by the fall of 2015, it was evident that the project had completely lost support from the government. In September 2015, the Moroccan authorities requested the Bank in a letter to the Country Director to cancel the project which the Bank concurred. The decision to close the project was appropriate in light of limited government ownership and to minimize the cost of keeping the project open when it was not likely to achieve its objectives. Quality of Supervision Rating Highly Unsatisfactory Overall Bank Performance Rating Highly Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance Although the Government of Morocco was supportive of the judicial reform effort in general, it appears that the authorities’ commitment was limited and diminished during implementation. For example, the project was approved on December 6, 2012; but, budget was allocated only following the Prime Minister’s intervention in November 2013 and the first request for disbursement took place only in January 2014. Overall, the authorities were not proactive in addressing factors that were causing implementation delays. The Government also did not proactively respond to the World Bank’s proposal to restructure the project that may have addressed shortcomings. As mentioned below, when other donor support to implementing the Charter became available (i.e. by mid-2014, the EU was also preparing a new project with grant funds), commitment to this project seems to have diminished further. Yet the Government only requested that the project be closed in December 2015 (ICR, para 31, page 11). Overall, the government’s performance toward ensuring quality of preparation and smooth implementation, and progress toward achieving the PDO had major shortcomings. Government Performance Rating Unsatisfactory b. Implementing Agency Performance MjDL’s was initially committed to the project; however, its commitment and ownership weakened over time. Change of leadership in MdJL in early 2012 also resulted in a change in the institution’s priorities, as mentioned above. Consultations with local stakeholders for this project commenced only in 2013, but stalled for 14 months while the consultation process for the development of the Charter was ongoing. One of the key factors in MdJL’s limited interest in this project was the EU was providing grant money. The grant money offered by EU was attractive for the Moroccan authorities and presented advantages compared to the funds available under the Bank project, as they did not require the MdJL to carry out procurement and financial management activities, and provided the authorities with what they wanted: experienced practitioner experts (ICR, p. 10). Page 7 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The Technical Steering Committee and the UGP were responsible for supervising M&E activities at the PDO level and at the component level, respectively. The design also included perception-based indicators, which were to be detailed after conducting the initial surveys aimed at establishing baseline indicators among the public and users of public courts. A contractor selected by the UGP was to oversee the development of the M&E framework for Component 1. The selection of project indicators and their measurement were kept modest, given the capacity and the time expected to follow the bottom-up approach. The targets for the first two years were also conservative to allow for MdJL and the pilot courts to commit to the targets. Despite the project’s focus on improving the performance of the judicial system in terms of service delivery, there were no baselines for any of the key performance indicators. This project could have developed a system in pilot courts at the preparation stage to collect data on key judicial performance indicators that could have be used as a baseline for the project; however, this was not done (ICR, p. 11-12). b. M&E Implementation The arrangement for the Technical Steering Committee and UGP to monitor activities, and for the contractor to develop an M&E system never became operational. There were delays in the procurement process for the contractor for designing and implementing surveys and project M&E. Analytical work was finally delivered in late 2015, two and a half years into implementation. However, the Directorate of Research, Cooperation and Modernization did not give the consultants access to the relevant data, citing security reasons. c. M&E Utilization M&E data were not collected or generated. Hence, the M&E framework was never utilized. M&E Quality Rating Negligible Page 8 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) 11. Other Issues a. Safeguards This was an environmental category "C" project. According to the ICR, no safeguard policies were triggered nor did any safeguard issues arise during implementation. b. Fiduciary Compliance Financial Management (FM). At appraisal, the FM assessment concluded that MdJL had sufficient capacity to manage project finances and administer loan funds, and FM risks were low. During implementation, the FM ratings were satisfactory until they were downgraded to moderately satisfactory in November 2014 due to a late audit. The rating remained until closing. Procurement: The PAD considered the overall risk for procurement as substantial due to lack of experience in Bank procurement procedures, and the absence of training provided to MdJL staff working on project implementation. Mitigation measures included training and recruitment of a procurement specialist. The ICR reports that although procurement was rated satisfactory throughout implementation, progress was slow and the quality of procurement documents very low (ICR, p. 13).　 c. Unintended impacts (Positive or Negative) None d. Other None 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Highly Unsatisfactory Highly Unsatisfactory --- Risk to Development Negligible Negligible --- Outcome Major shortcomings in quality Bank Performance Unsatisfactory Highly Unsatisfactory of entry and severe shortcomings in supervision. Page 9 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR High --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR offers several good lessons, including on the need for client ownership.　These are summarized below using somewhat different wording. 　 Client ownership is a critical factor in successful project implementation. The implementing agency (MdJL) initially requested budget support; however, the Bank proposed this modest investment operation as a first step towards a sector reform program that might lead to future budget support to the sector. Although the agency agreed, to be consistent with the government, they were not fully on board, and ownership decreased further following a change in leadership and priorities at MdJL. The modalities offered by the World Bank in support of reform efforts need to be aligned with the country’s institutional absorptive capacity and preferences. In this case, the loan amount was limited, implementation required a major effort to follow applicable World Bank procurement and financial management rules, and there was increasing perception that this effort and the amounts spent had low value added in comparison to the support to the sector from the EU. Project design should not be disconnected from the reality on the ground and should adequately reflect political economy factors. For a bottom-up approach to work effectively, it would have required both the client and the Bank to ensure effective involvement with pilot courts, local stakeholders, and civil society organizations at the earliest stages and certainly during preparation. These efforts could not be left to the implementation stage. IEG adds another lesson: Institutional development support requires an agile approach that can adapt to changing circumstances to ensure ownership. This project’s flexible design could have been easily adjusted to align with the government’s National Charter for Judicial Reform as it evolved. Instead, the project went on for too long without necessary decisions by the Bank or the borrower to make changes to improve its performance. Given the diminished ownership, a joint decision could have been reached earlier to close the project, and thus minimize costs of processes not leading to expected results. 14. Assessment Recommended? Page 10 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA-Judicial Performance Enhancement (P125799) No 15. Comments on Quality of ICR The ICR is clear, candid, and suitably critical of the project’s shortcomings, especially with respect to its readiness for implementation. The ICR team also presents useful lessons. It would be helpful to have an explanation of the different amounts for project cost at appraisal and closing in the ICR Datasheet, and in Appendix 1. Overall, the ICR is of good quality and the ICR team is commended. a. Quality of ICR Rating High Page 11 of 11