IEG Report Number: ICRR14889 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 02/05/2016 Country: Malawi Is this Review for a Programmatic Series ? Yes No How many operations were planned for the 3 series? How many were approved ? 1 Series ID: S133663 First Project ID : P133663 Appraisal Actual Project Name: Malawi Economic Project Costs (US$M): 50.0 50.0 Recovery Development Policy Operation 1 L/C Number: Loan/Credit (US$M): 50.0 50.0 Sector Board: Economic Policy Cofinancing (US$M): Cofinanciers: Board Approval Date : 05/28/2013 Closing Date: 12/31/2013 12/31/2013 Sector(s): Central government administration (67%); General energy sector (17%); Other social services (16%) Theme(s): Public expenditure; financial management and procurement (50%); Vulnerability assessment and monitoring (17%); Regulation and competition policy (17%); Debt management and fiscal sustainability (16%) Evaluator: Panel Reviewer: ICR Review Group: Coordinator: Nestor Ntungwanayo Clay Wescott Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: "The overarching development objectives of the Economic Recovery Development Policy Operation (DPO) series are to (i) strengthen the macroeconomic and public finance management and (ii) lay the foundation for stronger growth and protection of the poor [Program Document (PD), para. 49]". The DPO specific objectives are to: (i) strengthen fiscal discipline to entrench macro-stability, (ii) enhance efficiency and transparency of the public finance management, (iii) improve the energy efficiency and the regulatory environment; and (iv) improve efficiency in the provision of social safety nets, and (v) strengthen the country’s statistical capacity (page 2-29 of the PD) b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The DPO supported reforms in two policy areas (pillars in the PD) as follows: (i) Strengthening Macroeconomic and Public Finance Management : Key policy objectives under this area were to: (i) strengthen fiscal discipline to entrench macro-stability; and (ii) enhance efficiency and transparency of the public finance management. Four prior actions were completed before the program's approval: (i) the recipient, through the Ministry of Finance, has validated its outstanding payment arrears and has developed a plan for the full clearance of said arrears for State Owned Enterprises and Ministries, Departments and Agencies, (ii) the recipient, through the Ministry of Finance, has adopted a detailed budget formulation calendar, (iii) the recipient, through the Ministry of Finance, submitted Treasury Minutes for fiscal years 2005/06 and 2006/07 to Parliament, (iv) the recipient, through the Ministry of Agriculture and Food Security, has revised the bidding documents for the annual procurement of fertilizer under the Farm Input Subsidy Program to align the allocation of tonnage demand with unit price ranking for the purpose of increasing the recipient's value for money. (ii) Laying the Foundation for Stronger Growth and Protection of the Poo r: Policy objectives under this area were to: (i) enhance agriculture productivity and diversification; (ii) improve energy efficiency and the regulatory environment; and (iii) improve efficiency in the provision of social safety nets and (iv) strengthen the country's statistical capacity. The two following prior actions were completed under this area: (i) the recipient, through the Malawi Energy Regulatory Authority, undertook monthly reviews of the Automatic Tariff Adjustment Formula for electricity covering the period May to November 2012; and (ii) the recipient approved the National Social Support Program. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Costs: Total costs of the DPO supporting the reform program for Malawi amounted to US$50.0 million, Financing: The DPO program was funded by an IDA grant. Dates: The operation was approved on May 28, 2013, in the amount of US$50.0 million, made effective on June 10, 2013, and closed on schedule on December 31, 2013. Disbursed amount was equal to the approval amount. The operation was prepared as the first in a programmatic series of three operations. After the first operation was completed, the Bank suspended preparation of the second operation at an advanced stage, due to a large scale theft of public resources revealing serious weaknesses in the country's public financial management (PFM) system. The second and third operations in the series were subsequently dropped and the ICR was prepared on the basis of the first and only operation. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial The operation was consistent with the priorities of the country's development strategy and plan, and with the pillars of Bank's CAS. It was relevant to the country's needs at appraisal, and remained so until its closure. The operation's goal was to support the implementation of the Second Malawi Growth and Development Strategy (MGDS II) for 2011-16 approved in April 2012, as well as the Economic Recovery Plan adopted in September 2012, which provided for short-term measures to restore macroeconomic stability and to mitigate the effects of structural reforms on the poor. The program was built on policy dialogue and reforms undertaken under the earlier Rapid Response Development Policy Grant approved in July 2012 and intended to consolidate reforms supported by the three-year arrangement for Malawi under the IMF's Extended Credit Facility program. The program was geared towards consolidating macroeconomic policy reforms implemented under the stabilization effort and preparing the ground for longer term structural reforms. The operation was also an integral part of the World Bank Group's FY13-16 Country Assistance Strategy (CAS) for Malawi, contributing in particular to CAS objectives of promoting more diversified, competitive, shock-resilient economic growth and mainstreaming governance for enhanced development effectiveness. The CAS aimed to: i) promote sustainable growth; ii) enhance human capital and reducing vulnerabilities; and, iii) mainstream governance for enhanced development effectiveness. The proposed DPO sought to bridge the gap between the short term reforms of consolidating macroeconomic stabilization, and the longer term reforms that will improve efficiency in public expenditure, remove constraints to growth and economic diversification, and promote efficiency in the provision of social safety nets. b. Relevance of Design: Modest During preparation, key dimensions of the macroeconomic framework were reviewed, including by a joint IMF/World Bank Debt Sustainability Analysis for low-income countries in early 2012 as part of the Article IV consultations. The macroeconomic framework was considered adequate for the operation if government made progress on structural reforms to improve controls and efficiency of government expenditure, and to maintain a flexible exchange rate (PAD: 11-12,14). The objective statement was not clearly articulated. Two pillars were defined as objectives (para 49), and in the results matrix in Annex 2, there are different objectives. In addition, the focus of the reforms supported by the program could have been improved if identification of country priorities and risks was done in a more careful and comprehensive way, and if sharing of responsibilities among donors was more strategic. In the hindsight, fiduciary weaknesses and risks in the management of public funds pinpointed by the 2011 PEFA were not sufficiently understood, and correctly addressed by the DPO and the donors, and ended up hurting the country's public finances. For both overarching objectives, the causal chain between the prior actions and the objectives was weak. The four prior actions associated to the first pillar provide a plausible link to the objectives but had a very limited impact on the challenges of country's fiscal discipline, and transparency of public finance. Validating internal arrears stock and having a plan to clear them is very far from effectively clearing them. Instead of clearing them, more arrears were accumulated when the DPO was under implementation. Similarly, the two prior actions associated with the second pillar provides a causal chain to the objectives, but it was even weaker, because they were preliminary and first generation reforms, and their impact toward the objectives was marginal. A number of risks associated with the operation were correctly identified and included (i) external shocks, political risks, and technical capacity risks. In each case, mitigation measures were proposed in the PAD. However, fiduciary weaknesses unearthed by the 2011 PEFA and acknowledged by the PAD were not considered as a source of major and immediate risks, and were not sufficiently mitigated. 4. Achievement of Objectives (Efficacy): Strengthening Macroeconomic and Public Finance Management : Negligible (i) Strengthen fiscal discipline to entrench macro -stability. Reduction in the stock of domestic payment arrears owed by State Owned Enterprises : The performance indicator was to clear 50 percent of the stock of arrears. When the operation closed in 2013, about 26 percent of the stock of domestic payment arrears had been cleared, far below the program target indicator. While initial stock of arrears identified at appraisal was being cleared, a significant amount of new arrears were built up by ministries, departments and agencies (MDAs) by the end FY13/14. This was the result of the negative fiscal impact of the large scale theft of public funds in 2013, which triggered the withdrawal of external budget support and resulted in lower government budget resources. As of end FY13/14, the total stock of arrears owed by MDAs had ballooned, and dwarfed earlier attempts to clear arrears. Reduction in the variance in primary expenditure between approved and actual budget : The program target was to reduce the variance from 12 percent to less than 8 percent. The sharply lower grants in FY13/14, and the significant overspending on recurrent expenditures led to a substantially larger budget deficit than anticipated. The deficit was financed in part by higher domestic borrowing, but the fiscal year also closed with substantial outstanding payment arrears owed by the Government, and the FY13/14 variance in primary expenditure was 13 percent, above the program target indicator of less than 8 percent. Despite attempts to improve planning and budgeting, the link between activities supported under the program and observed outcomes was overshadowed by events surrounding the financial scandal, the suspension of budget support, and underlying weaknesses in expenditure management and commitment control. Zero ceiling on accumulation of new arrears by Ministries , Departments and Agencies : While the program target was a zero ceiling on accumulation of new arrears, the stock of domestic payment arrears grew substantially over the course of FY13/14 to reach MWK 30 billion (US$64 million). Transparency of PFM also deteriorated following the breakdown of public finance controls, the build-up of significant off-budget payment arrears, and significant disruption to the normal process of preparing government's annual financial statements. The audit of government's financial statements has also weakened, and at the time of ICR preparation, there was a three year backlog in the preparation of audited financial statements, and Treasury Minutes backlog has remained at above 60 months and in excess of the program target indicator. (ii) Enhance efficiency and transparency of the public finance managemen t: Reduction in the rate of recurrent Audit Committee findings : The expected result under this reform objective was a reduction in the rate of recurring audit findings from 90 percent in 2012, to 50 percent by the end of the program. The medium term expected outcome was strengthened external oversight of public finances. Five Audit Committees were set up with independent membership, each of which entrusted with four MDAs. The Audit Committees met irregularly and could not make enough progress in addressing findings. At the time of the ICR, a total stock of 93 of 114 audit issues (82 percent) remained unresolved, reflecting a progress that was short of the program target of 50 percent. Reduction in the fertilizer share of the agriculture budget public investments in agriculture equally : The program target was to reduce the fertilizer share of the Agriculture budget from 85 percent to 80 percent. The share of fertilizer costs in the agricultural recurrent budget has slightly decreased, following a decrease in international fertilizer prices, and a change in the procurement of fertilizer, but fell below the program target. In broad terms, the Farm Input Subsidy Program (FISP) continues to dominate public expenditure on agriculture. Efforts to see a more balanced distribution of public investments in agriculture and reduce the share of fertilizer in the agriculture budget have been slow. The Bank has contributed to this FISP reform dialogue by producing policy briefs to help summarize the findings and evidences of various research papers and to help formulate clear reform options. Reduction in the backlog of Treasury Minutes : There were no Treasury Minutes prepared during the period from appraisal of the operation until the preparation of the ICR, and the backlog has remained at above 60 months and in excess of the program target indicator. Laying the Foundation for Stronger Growth and Protection of the Poo r: Modest (iii) Improve energy efficiency and the regulatory environment : Increase in the number of power generation licenses for Independent Power Producers (IPP): The expected result under this reform was to issue at least one power generation license. At operation closure, there was no IPP generation license granted. However, reforms undertaken by the authorities have removed some of the policy restrictions that prevented market access for IPPs. At the time of the ICR there were no longer any policy restrictions preventing market access for IPPs and while one license was close to being granted, there are as yet no formal approvals of IPPs in Malawi. Increase in legume production : The program target was an annual increase of 20 percent in legume production. The ICR found (p.12) that legume production in the 2013/2014 cropping season grew by 19.9 percent compared to the previous 2012/2013 campaign, practically meeting the program target indicator. This increase was mainly attributable to good weather conditions, although the distribution of subsidized legume seeds contributed to greater access to seeds for smallholder farmers. The DPO was instrumental in ensuring that the private sector, especially seed companies, was associated with the proposed legal reform agenda of the Seed Act. Finally, a draft Seed Act was finalized and is planned to be submitted to Parliament's approval in 2015. (iv)Improve efficiency in the provision of social safety nets : Increase in the number of National Social Support Program beneficiaries with savings of at least 30 percent of public works wage one year after participation : Achieved. The expected result under this reform was an increase in the number of National Social Support Program beneficiaries with savings of at least 30 percent of the public works wage one year after participation from 13,000 in 2012 to 49,000 by the end of the program. The ICR found that a total of 99,153 people have joined the Community Savings and Investment Program of the National Social Support Program. Of this total, 78,330 (79 percent) meet the savings criteria at operation closure, substantially exceeding the program target indicator. Of this total, 47,781 (61 percent) of the beneficiaries are female. The operation supported the approval of the National Social Support Program as part of efforts to create an enabling policy environment for the implementation of social protection interventions in Malawi. (v) Strengthen the country’s statistical capacity . Increase in the number of quarterly Malawi Growth and Development Strategy progress reports published : The expected result under the program was an increase in the number of MDGs quarterly reports published from zero in 2012 to four by 2016. The expected outcome was the production of high quality, timely and accessible statistical information. Current status is that quarterly progress reports are being produced on average once per year, below the program target indicator. Efforts to increase the availability of MDGs reporting have had mixed impact, with no sanctions for failure to submit quarterly reports. Only one sector working group out of the 16 is operational and covers Education, Health, Trade, Democratic Governance, Decentralization and Agriculture. 5. Efficiency (not applicable to DPLs): 6. Outcome: Unsatisfactory The objectives of the DPO were substantially relevant, but the relevance of design was modest. While the operation was in line with the country's priorities and the pillars of the CAS, the results chain was weak, inter alia because of the cancellation of the two last operations of the series. Efficacy was modest for the two objectives. Performance was weak toward fiscal discipline, transparency in public finance management, energy efficiency and statistical capacity as all indicators were missed. Satisfactory achievements were recorded in the area of legume production and the provision of social safety nets, and limited progress was made regarding fertilizer procurement, and energy pricing. a. Outcome Rating: Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Malawi's economic and governance institutions remain fragile, as illustrated by the recent embezzlement of about US$50 million of Government resources following the collusion of civil servants operating in the public finance administration. The event was a testimony of weak checks and balances arrangements in the management of public resources. Unless there is a political will to tackle this matter, progress in the management of public resources will be hard, and the limited reforms that were accomplished will not be sustained. Unless the Government shows reform commitment and brings the donor community to reinstate their programs, achieved reforms under the DPO might not be sustained. a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: The design of the operation was informed by the Bank's sustained dialogue, budget technical assistance, and large analytic work covering macroeconomic policy, public financial management, and agriculture and energy sector reform. Preparation took place in a context of good coordination among the Bank and other development partners within the Common Approach to Budget Support (CABS) group, five of which provided budget support alongside the Bank in FY13. The CABS group and the Government of Malawi agreed on a Joint Framework agreement for budget support cooperation designed to formalize their relationship and which provides for a Performance Assessment Framework on which to base disbursement decisions. During FY13, five other donors (IMF, the European Union, the DFID, Norway and the African Development Bank) supported public finance reforms and provided budget support to Malawi. While there was coordination among donors and an agreement on a joint performance framework, the ICR did not explicitly indicate how responsibilities in supporting specific reforms were assigned to donors in order to avoid duplication, and to make sure that critical areas of reforms are not overlooked. One of the consequences of this lack of clarity was that fiduciary controls remained weak. The Bank and the donor partners would have benefitted from taking better into account knowledge of how the PFM system was operating in a weak governance environment. In particular, the 2011 PEFA report underscored that there were major and longstanding weaknesses in the areas of internal audit, accounting, recording and reporting. These weaknesses, combined with the program’s assessment that the country's fiduciary risks were high, should have sent a warning sign that the country's PFM situation was in crisis. In the light of this knowledge, the reform program should have identified more stringent prior actions in the areas of treasury and cash management, accounting, procurement, and internal budget audit and controls. More policy dialogue was needed to identify and build momentum for such reforms, and then focus on the areas where there was support, setting appropriate prior actions and achievable targets, and providing technical assistance as needed. Quality-at-Entry Rating: Moderately Unsatisfactory b. Quality of supervision: Supervision was cut short by the unravelling of the reform agenda following the embezzlement of Government resources. However, monitoring of the results indicators continued to take place as part of the preparation of the originally planned DPO-2, and the regular economic policy dialogue with the Government. Despite challenges associated with the fall out of the treasury theft, efforts were continued to complete the preparation of the second operation, with an increased focus on remedial measures aimed at restoring budget basic controls and oversight. However, preparation of the second operation was later suspended and the series terminated following a cabinet reshuffle and a reorientation in the focus of Government policy priorities. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: IEG concurs with the ICR assessment (p.17) that weaknesses of PFM systems in Malawi had existed for several years, but the gravity of these weaknesses was not factored into the dialogue that took place during the design of the reform agenda. As a consequence, there was a crisis in the country's public finances, and development partners withdrew their resources, basic budget planning and commitment controls broke down, and macroeconomic and fiscal instability undermined the development objectives of the operation. The negative impact of systemic institutional weaknesses in the integrity of Malawi's public finances, together with a lack of oversight and culture of impunity outweigh the few DPO reform achievements. Government Performance Rating : Unsatisfactory b. Implementing Agency Performance: The Debt and Aid Division of the Ministry of Finance was the primary agency responsible for coordination during the preparation and implementation of the operation. Malawi's coordinated approach towards development policy financing under the CABS group was noteworthy, although results were lower than expected. The Division displayed good management and coordination throughout the preparation and implementation of the program. However, concerns over the appropriateness of design for a program that went off track so soon after approval weigh negatively on the performance of the implementing agency Implementing Agency Performance Rating : Unsatisfactory Overall Borrower Performance Rating : Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E matrix in the Annex II of the Program Document was well crafted. For each expected outcome, there was a prior action for the first operation, and indicative triggers for the two subsequent operations. For each objective, there was a baseline and a targeted outcome indicator. However, there were shortcomings: the objective statement was not well articulated, with two pillars defined as objectives. The causal chain between prior actions and the objectives was weak (see section 3b). The Ministry of Finance had experience in implementing DPOs, and was responsible for overall implementation of the DPO series as well as for reporting progress and coordinating actions among other concerned ministries and agencies. The institutional arrangements for the preparation and execution of the operation fell within the framework of the CABS. On the Government side, CABS was served by a secretariat which was under the purview of the Debt and Aid Division of the Ministry of Finance. The Secretariat was in charge of mobilizing sector experts in various ministries and departments, and coordinating the preparation of Performance Assessment Framework Priority Action Plan, implementation, assessment and revision. Data and information collected in this context was deemed sufficient to monitor the reform program implementation. b. M&E Implementation: The ICR assessment under this section was comprehensive (p.8). Monitoring of the operation took place as part of the preparation process for DPO-2, and involved keeping track of the implementation of the prior actions for DPO- 1 and policy dialogue in relation to the proposed triggers for DPO-2. The Bank team held regular meetings with the authorities, including through sessions of the CABS group, to monitor implementation of the program. The team also worked closely to track progress towards the achievement of expected outcomes of the series of DPOs as presented in the results matrix of the program document for DPO-1. Monitoring of the macroeconomic program was supported through preparation of quarterly economic updates and participation in IMF Extended Credit Facility review missions. However, because of shortcomings in the objective statement and design of the operation, the M&E implementation could not correct the above weaknesses. c. M&E Utilization: Following the 2013 cash theft scandal, the Bank reconsidered its engagement on governance issues. M&E exercise could not be used to prepare a follow-on operation. Working with the Government and the CABS development partners, there was an agreement on an 'extraordinary Performance Assessment Framework' to monitor progress in addressing PFM weaknesses. Progress on governance reforms was also monitored under the Financial Reporting and Oversight Improvement Project, including through the scaling up of a Multi Donor Trust Fund to support critical improvements to systems, controls and procedures to address weaknesses exposed as part of the financial scandal. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: The ICR did not provide any information under this heading. b. Fiduciary Compliance: The ICR did not report on fiduciary compliance, and did not indicate if the funds disbursed under the DPO were audited. The PD indicated that Malawi 's fiduciary risk was high and this assessment was confirmed by the 2011 PEFA. A request of an independent audit of the Government's account was scheduled in the Financing agreement. Because of the events surrounding the theft of Government's resources, and the cancellation of subsequent DPOs, the Bank should have taken this opportunity to request a timely, independently audited Government financial statement for FY13. This would have helped in supporting the country to rebuild credibility after the 2013 scandal. c. Unintended Impacts (positive or negative): The ICR did not provide any information under this heading. d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Unsatisfactory Unsatisfactory Risk to Development High High Outcome: Bank Performance: Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Unsatisfactory Unsatisfactory Systemic institutional weaknesses in the integrity of Malawi's public finances, and a lack of oversight and culture of impunity were the trigger of the operation failure. Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: IEG agrees with the three lessons identified in the ICR and rephrased below: (i) IEG concurs with the finding that there are immense risks linked to a partial reform syndrome, a situation where governments undertake reforms at a superficial level, and produce the form but not the substance of modernized institutions. The breakdown of Malawi's PFM that occurred in 2013 underscores the need for a thorough grasp of the local political economy of reform and the importance of looking beyond the formal rules-based systems, and a good understanding of the underlying informal arrangements that allowed for collusive behavior to take place. (ii) Lessons from the 2013 theft point to the need for increased use of independent verification methods to confirm the actual status of reform measures adopted, particularly with regards to PFM systems and controls, and more specifically to careful exploitation and use of the PEFA reports and recommendations. (iii) The experience of this operation points to the need for a strong focus on timeliness of fiscal information and integrity of fiduciary systems. In particular, establishing the integrity and strength of fiduciary systems and expenditure controls need to be the priorities in any future budget support operations. Furthermore, this experience suggests the need to narrow the scope of structural reforms to the most macro-critical areas where achieved progress would be harder to reverse after the completion of an operation. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was candid, concise and self-contained. It addressed the key issues of the DPO, from appraisal to closure of the operation, including the positive aspects of good coordination among donors, and the negative features, related to the insufficient quality at entry, and the weaknesses in the design of the reform agenda. In particular, there was an analytical effort to articulate the ICR around the achieved results, drawing on the policy and results matrices which were developed in the program document. However the ICR should have discussed more thoroughly the fiduciary risks identified in the PD and in the PEFA report, and how in hindsight they materialized after the operation approval. Parallel budget support was provided by five other donors (IMF, EU, DFID, AfDB, Norway) and most of them supported PFM reforms, and a clearer reporting on their contribution was lacking. Also, the story about the amounts disbursed and their contribution in achieving the outcomes were not thoroughly discussed. Relevant lessons were drawn from the DPO implementation, and were derived from the evidence, and the analysis therein. a.Quality of ICR Rating : Satisfactory