Document of The World Bank Report No: ICR2918 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-94552) ON A GRANT IN THE AMOUNT OF US$ 18 MILLION FROM THE WORLD BANK IRAQ TRUST FUND TO THE REPUBLIC OF IRAQ FOR A PUBLIC FINANCE MANAGEMENT PROJECT February 28, 2014 Poverty Reduction and Economic Management Unit Middle East and North Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 28, 2014) Currency Unit = Iraqi Dinar (IQD) 1.00 IQD = US$ .00086 US$ 1.00 = 1,150 IQD FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS BCC Budget Call Circular BSA Board of Supreme Audit EPP Emergency Project Paper FM Financial Management FMA Fiduciary Monitoring Agency FMR Financial Monitoring Report GDP Gross Domestic Product GoI Government of Iraq IDA International Development Association IFMIS Integrated Financial Management Information System IFR Interim un-audited Financial Reports ISN Interim Strategy Note ISR Implementation Status and Results Report ITF Iraq Trust Fund IQD Iraqi Dinar KRG Kurdish Regional Government M&E Monitoring and evaluation MOF Ministry of Finance MOFTAC Ministry of Finance Accounting Training Center MOP Ministry of Planning MTBF Medium-Term Budget Framework MTEF Medium-Term Expenditure Framework NCB National Competitive Bidding NDS National Development Strategy PDO Project Development Objective PMT Project Management Team SBD Standard Bidding Document USAID United States Agency for International Development Vice President: Inger Andersen Country Director: Ferid Belhaj Sector Manager: Guenter Heidenhof Project Team Leader: Arun Arya ICR Team Leader: Michael Schaeffer REPUBLIC OF IRAQ Public Finance Management Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Profile 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes ............................................................................................ 9 4. Assessment of Risk to Development Outcome......................................................... 20 5. Assessment of Bank and Borrower Performance ..................................................... 21 6. Lessons Learned ....................................................................................................... 28 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 32 Annex 1. Project Costs and Financing .......................................................................... 33 Annex 2. Outputs by Component ................................................................................. 35 Annex 3. Economic and Financial Analysis ................................................................. 46 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 47 Annex 5. Beneficiary Survey Results ........................................................................... 48 Annex 6. Stakeholder Workshop Report and Results................................................... 49 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 50 Annex 8. Iraq PFM Action Plan ................................................................................... 51 Annex 9. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 54 Annex 10. List of Supporting Documents .................................................................... 55 REPUBLIC OF IRAQ Public Finance Management Project A. Basic Information IRAQ: Public Financial Country: Iraq Project Name: Mgmt Reform Project ID: P110862 L/C/TF Number(s): TF-94552 ICR Date: 02/28/2014 ICR Type: Core ICR Lending Instrument: ERL Grantee: Republic of Iraq Original Total USD 16.00M Disbursed Amount: USD 9.78M Commitment: Revised Amount: N/A Environmental Category: C Implementing Agencies: Ministry of Finance Cofinanciers and Other External Partners: N/A B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 05/22/2008 Effectiveness: Appraisal: 11/25/2008 Restructuring(s): 01/25/2013 Approval: 06/26/2009 Mid-term Review: 09/18/2011 09/20/2011 Closing: 06/30/2013 08/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: High Bank Performance: Unsatisfactory Grantee Performance: Moderately Unsatisfactory i C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Unsatisfactory Government: Not Applicable Moderately Implementing Quality of Supervision: Not Applicable Unsatisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Unsatisfactory Performance: Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 50 50 Sub-national government administration 50 50 Theme Code (as % of total Bank financing) Administrative and civil service reform 25 25 Decentralization 25 25 Public expenditure, financial management and 50 50 procurement E. Bank Staff Positions At ICR At Approval Vice President: Inger Andersen Daniela Gressani Country Director: Ferid Belhaj Hedi Larbi Sector Manager: Guenter Heidenhof Farrukh Iqbal Project Team Leader: Arun Arya Monali Chowdhurie-Aziz ICR Team Leader: Michael G. Schaeffer ICR Primary Author: Michael G. Schaeffer ii F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) 1. The Iraq PFM Reform Project Development Objective is to support the Government of Iraq's (GoI) efforts to develop more effective, accountable and transparent public financial management. Support will be provided for strengthening budget management, improving public financial reporting and control, strengthening public procurement, and establishing the tools, systems and institutions required to build capacity for effective public financial management. Revised Project Development Objectives (as approved by original approving authority) 2. Not applicable. iii (a) PDO Indicator(s) Original Target Values (from Formally Revised Actual Value Achieved at Indicator Baseline Value approval documents) Target Values Completion or Target Years Indicator 1 : Capital budget execution rate increases by at least 10 percentage points According to Ministry of Planning (GoI) execution Value rates: quantitative or 75 85 N/a  2011 (75.39%) Qualitative)  2012 (78.45%)  2013 until 31/10/2013 (40.2%) Date achieved Not Achieved: Capital budget execution rate rose by 3.45% to 78.5% in 2012. GoI has consistently executed its operating budget at 80-90% over the past several years financing wages and salaries and Comments social benefits. According to Ministry of Planning, the execution rate falls to less than 60% in key development sectors (incl. % including Culture and Youth (49%); Water and Sanitation (52%); Education (57%), and Health and Environment (58%) . achievement) The energy and industry sectors and KRG achieved 93%, 94% and 100% capital execution rates, respectively.1 Indicator 2 : Outstanding balances in Spending Units decline by 30 percentage points, as a share of the budget Value quantitative or 38 8 N/A Est 30% Qualitative) Date achieved Undetermined:-causality of the Technical Assistance to the achievement of this indicator, poor indicator design. Technical assistance was projected to translate to a 8 percent reduction in cash balances as a percent of budget, or at the end Comments of the project cash balances be less than thirty (30%) percent of total budget. Three previous ISRs provided that the cash (incl. % comparisons of results with the baseline will be done, but there is no comparison with the baseline. A review of the current achievement) status of a number of ministerial cash balances, indicate that cash balances of line ministries are IRD 3.9Trillion (September 1 The WB-Public Expenditure Review (PER) indicates capital spending efficiency rates for the Government of Iraq at around 55%. The PER obtained their data from Ministry of Planning (MoP) data sources. The ICR Review obtained development (investment) budget spending efficiency rates directly from the Ministry of Finance. Although, there is some difference in the measurement of capital spending efficiency rates, the conclusion remains the same:- capital budget execution rates did not rise to meet the targeted rate of 85% as presented in this PDO indicator. iv 2013). Without reviewing the 640 plus spending unit bank accounts at the Central Bank of Iraq, it is difficult to determine whether Indicator 2 has been achieved. The Project’s ISR-3 and 4 indicated that there was a reclassification of accounts in 2009, due to which the Cash Balances at 31 December 2009 were reduced to 3.843 trillion (5.5% of the Budget). The cash balances stood at 5.937 trillion ID on June 2011 (6.1% of the budget) as per the new classification. Thus, there is a problem in comparing the progress with the baselines as the definition of indicator has changed. The outcome indicator and its baseline needed to be (re)adjusted. The way forward with respect to measuring this indicator was to estimate the present cash balances as per earlier classification by including provincial accounts, accounts of government trading enterprises, and purchases of vehicles. However, this was not done and cannot be measured by this ICR. While the need to change the indicators may be identified during the Mid-term review, indicators are changed through formal restructuring approved by the Board or Management (as applicable). Indicator 3 : At least 50% of contracts above threshold awarded competitively Value quantitative or Unknown 50% DROPPED N/A Qualitative) Date achieved Not Achieved (indicator dropped) Comments (incl. % DROPPED: error in baseline formulation. Details in main text. achievement) Indicator 4 : Ministry of Finance training institute providing training in all operational aspects of PFM project Value quantitative or No training program in place Y N/A N/A Qualitative) Date achieved Partially Achieved This outcome indicator is inappropriate as it measures only near-term aspects of the project and does not reflect any real Comments achievement. It is, in fact, an output indicator without any quantitative basis to measure whether or not the project’s (incl. % technical assistance was effective. A Training Needs assessment was completed and a few training of trainer sessions were achievement) provided as per the original Results Framework. There is inadequate depth of training ability in any of the key components of public financial management. v (b) Intermediate Outcome Indicator(s) Actual Value Original Target Values (from approval Formally Revised Target Achieved at Indicator Baseline Value documents) Values Completion or Target Years Indicator 1 : Component 1: Draft sector strategies completed and submitted to all relevant parties for comment/revision/etc. Value (quantitative N Y - Draft Sector Strategies Prepared None N/A or Qualitative) Date achieved Not Achieved Project proposed completed sector strategies for two sectors: Ministry of Water Resources and Ministry of Education. Drafting of Comments these two sector strategies was cancelled because these sector ministries asked other donors to provide assistance. ISR-4 asserted (incl. % that it was not the role of the PMT to assist preparation of sector strategies, but to ensure that the Budget Preparation guidelines are achievement) such that these sector strategies are well linked to the budget. In fact, the original performance measurement provides that WB would assist with the development of the sector strategies. Indicator 2 : Component 1: New procedures in place for project preparation and appraisal Value (quantitative N Y N/a or Qualitative) Date achieved Not Achieved Judging from the ISRs (2009, 2012) there was effectively little or no improvement in the implementation of new projects. ISR (9/2013) claims a 10% improvement in capital budget execution performance as measured by MOP and other relevant entities. Comments However, this did not take place. FY2012 capital execution rates as stipulated by MOP were 78.45%, barely 3.45 percent above the (incl. % original baseline estimate established in 2008/09. Project execution rates estimated up and until October 2013 are estimated at achievement) 40.3%. While new procedures and guidelines were presented to MOP, much of the on-the-ground technical assistance work and project preparation guidelines were established by USAID (which used the Project Management Institute Training Handbooks and Certification process) and Project Evaluation check-list. Indicator 3 : Component 1: New macro-economic and fiscal “team‟ established within MoF Economic Policy Division Value (quantitative N Y or Qualitative) Date achieved MTFF Team initiated in 2009. Reasonably effective MTFF analysis produced in 2011. Comments A Macro-Economic Fiscal Framework (MTFF) Team was initiated in 2009. The Iraqi MTFF uses the IMF financial programming (incl. % modeling procedures. A comprehensive database of historical fiscal information is in place including fiscal, economic and budget achievement) data and supports the estimation of fiscal forecasts and has enabled the development of top-down indicative budget ceilings. vi Actual Value Original Target Values (from approval Achieved at Formally Revised Target Indicator Baseline Value documents) Completion or Values Target Years Notwithstanding, the development of a Medium Term Expenditure/Budget Framework (MTEF/MTBF) has not been accomplished. The Ministry of Planning has developed a 5-year Project Investment Plan (PIP). This PIP is combined with a 1-year forward operational budget to produce an annual budget. The development budget and the operational/recurrent budget remain largely separate (and, independent):- There is no bottom-up Medium Term Budget Framework. Indicator 4 : Component 1: New budget instructions approved and guidelines ready Value (quantitative N Y N/A or Qualitative) Date achieved Not Available New budget call circular (BCC) designed and in use Indicator 4; and, New budget call circular (BCC) in use indicator 5 (below). 2 As GOI is required by law to produce two primary circulars (recurrent and investment) during a budget implementation period, it is difficult to determine the outcome/output impact achieved from WB TA intervention. Every year, regardless of Comments external technical assistance– a budget circular is recalibrated stipulated by Public Finance Act. At present, three sector strategies (incl. % are established with a multi-year costing of recurrent and investment expenditure framework continue to be poorly formulated. achievement) Clear linkages between investment budgets and forward expenditure estimates are not readily observable. While some projects are selected based on sector plans, and some of these have recurrent cost implications, these estimates are not generally shared during operations budgeting. Moreover, the forward estimates that do exist are not developed from such costings. Indicator 5 : Component 1: New Budget Call Circular (BCC) in use (See Discussion For Indicator 4 Above) Value (quantitative N Y or Qualitative) Date achieved Not Available Comments (incl. % achievement) See Discussion for Indicator 4 above. 2 Several criteria were initially to be used to measure these indicators as provided in earlier project documentation including: i) Introduction of expenditure ceilings; ii) Adherence to budget instructions, iii) Completeness of submissions; and iv) Submission of a multi-year budget linked to a multi-ye Several criteria were initially expected to be used to measure these indicators as provided in earlier project documentation including: i) Introduction of expenditure ceilings; ii) Adherence to budget instructions, iii) Completeness of submissions; and iv) Submission of a multi-year budget linked to a multi-year sector strategy. However, these criteria were removed in lieu of less onerous measurement indicators. ar sector strategy. However, these criteria were modified in lieu of less onerous measurement indicators. vii Actual Value Original Target Values (from approval Formally Revised Target Achieved at Indicator Baseline Value documents) Values Completion or Target Years Indicator 6 : Component 2: Standard Bidding Documents finalized The Project completed Y - Sector specific Standard Bidding Value (quantitative N - no standard bidding seven (7) standard Documents finalized for ministries of health, Seven (7) SBD or Qualitative) documents (SBDs) bidding documents, education, electricity and public works. finalized in July 2013. Date achieved Achieved (July) 2013 Comments The presentation and customization of SBDs for several key Iraqi ministries appear on the various Ministries website. In order to (incl. % achievement) make SBDs mandatory, they need to be tested or piloted. However, there has been no mandate to continue using these SBDs. Indicator 7 : Component 2: National Procurement Manual finalized Value (quantitative N Y DROPPPED N/A or Qualitative) Date achieved Partially Achieved (August 2013) Comments (incl. % The National Procurement Manual (National Implementation Manual) was prepared. However, since the Government has not yet achievement) passed a National Procurement Law and it is not foreseeable in near future, the National Procurement Manual cannot be finalized Indicator 8 : Component 2: Procurement Bulletin board being fully implemented, and housed within Procurement Regulatory Agency Value Design of single portal website N - Limited procurement Y - Procurement Bulletin Board operational (quantitative for public procurement notices N/A information available on-line and Future options for E-procurement or Qualitative) prepared Date achieved Not Achieved Comments ToR and road map for implementation were prepared in cooperation with USAID (Tarabot). Procurement bulletin board was not (incl. % completed. achievement) Component 2: Training Needs Assessment completed, training curriculum developed and first round of "Train-the-trainers‟ Indicator 9 : completed Value Y - Curricula revised based on training Training needs assessment (quantitative N outcomes N/A developed or Qualitative) Longer-term training plan in place Date achieved Achieved August 2013 Comments (incl. % A national training strategy was developed and capacity building efforts in procurement established. viii Actual Value Original Target Values (from approval Formally Revised Target Achieved at Indicator Baseline Value documents) Values Completion or Target Years achievement) Indicator 10 : Component 3: First round of training on new cash release system completed with selected pilot ministries New system demonstrates 30% improvement Value (quantitative N in the coherence between cash releases and Y N/A or Qualitative) actual needs Date achieved Partially Achieved (June 11, 2013) PFM project introduced rules and guidelines for cash management and commitment control system and improving financial Comments reporting. PMT insisted on implementing the Commitment Control/Cash Release system before providing any Capacity Building (incl. % Exercises. Eighty – seven (87) persons did complete basic training on cash release systems. Training for other line ministries has achievement) not been extended. The ICR review questions the validity of this indicator. What exactly is being measured by the indicator: the new system demonstrates 30% improvement in the coherence between cash releases and actual needs? Indicator 11 : Component 3: Detailed rules and regulations for new commitment control system developed Value New system rolled out to at least 4 total (quantitative N Y sector ministries or Qualitative) Date achieved Not Achieved The Project was able to prepare and present rules and regulations regarding commitment controls. However, the GoI’s financial Comments management operations contain a mix of financial management systems: In line ministries expenditure control mechanisms largely (incl. % rely on a weaving together of paper journal entries, excel spreadsheets, and, bespoke financial management data bases. Although, achievement) detailed rules and regulations were developed, submitted, and rolled out to four ministries, the task of ring fencing, and mainstreaming, the various channels where lack of controls occurs continues. Indicator 12 : Component 3: New financial/fiscal performance system developed, and tested in three ministries New system implemented; Value Current reporting does not 30% reduction in unused, outstanding (quantitative detail outstanding stock of DROPPED balances in Spending Units or Qualitative) advances Ongoing Training Date achieved Not Achieved (Dropped) Comments A new component 5 on Integrated Financial Management Information System (IFMIS) was added to the project to develop (incl. % financial/fiscal performance evaluation in three ministries. ix Actual Value Original Target Values (from approval Formally Revised Target Achieved at Indicator Baseline Value documents) Values Completion or Target Years achievement) Component 3: Training materials developed for financial/fiscal performance system, and training of Master Trainers Indicator 13 : carried out Value (quantitative No training program in place Ongoing training DROPPED or Qualitative) Date achieved Not Achieved (Dropped) Comments A new component 5 on Integrated Financial Management Information System (IFMIS) was added to the project to develop training (incl. % achievement) materials for the financial/fiscal performance evaluation system. Indicator 14 : Component 3: Assessment of IG system completed, and documentation provided clarifying mandate of internal controllers Overlapping functions Value between IGs and other Identification completed of needs for system (quantitative internal auditors; unclear improvement, including clarification of Y or Qualitative) mandates and reporting mandate of internal controllers responsibilities Date achieved Achieved (August 25, 2013) Comments The objective was to develop a Gap analysis of current conditions at IG, MOF, and MOP and international practices. Consultancy (incl. % report was presented but no further action was taken. achievement) Indicator 15 : Component 3:Website being regularly updated Value Website underutilized, both Website being regularly updated with new Website of Ministry of Finance (quantitative for internal and external rules/regulations/procedures/ public upgraded for enhancing or Qualitative) purposes announcements/etc. as necessary citizen's access of fiscal data. Date achieved Achieved (2013) Comments A regular update of website would not have contributed to the PDO unless the current website was upgraded to enhancing citizen's (incl. % access of fiscal data. achievement) Indicator 16 : Component 3: IT assessment completed, and functional requirements for upgraded system developed Value MoF departments fragmented New software and hardware are in place and DROPPED (quantitative in their information-sharing contributing to improved efficiency of MoF x Actual Value Original Target Values (from approval Formally Revised Target Achieved at Indicator Baseline Value documents) Values Completion or Target Years or Qualitative) ability operations. Date achieved Not Achieved (Dropped) Comments A new component 5 on Integrated Financial Management Information System (IFMIS) was added to the project to conduct IT (incl. % assessment and develop functional requirements for an upgraded system. This system ultimately could not be delivered (see achievement) Component description, paras. 69-73) Indicator 17 : Component 4: Training program fully in place Training program to provide targeted Value Limited capacity in MoF training; Long-term business plan for Training program fully in place (quantitative or Qualitative) training department sustainable operations of the MoF training in MOFATC department developed and approved Date achieved Partially Achieved A Training Needs assessment was completed, a few training of trainer sessions were provided. There is inadequate depth of Comments training ability in any of the key components of public financial management. The detailed needs assessment provided to (incl. % MOFTAC by Coffey LLC has been adopted by MOFTAC. MOFTAC has put forward a 5-year cumulative budget estimated at achievement) USD13.5 million with a detailed training plan. Indicator 18 : NEW Component 5: Prototype for IFMIS developed Value N – GOI agree on scope of IFMIS Prototype is developed after (quantitative IFMIS design prototype conducting a SRS study involving all IFMIS Prototype Developed or Qualitative) needs to be developed stakeholders Date achieved Not Achieved Comments (incl. % achievement) New Indicator as of Project Restructuring Paper, January 25, 2013 Indicator 19 : NEW Component 5: Prototype for IFMIS tested Value N - Prototype is yet to be IFMIS Prototype is tested in the Ministry of IFMIS Prototype Tested in 3 (quantitative developed. Testing will be Finance, 3 pilot Ministries and the Board of Pilot Ministries or Qualitative) done after it is developed. Supreme Audit. Date achieved Not Achieved Comments (incl. % New Indicator as of Project Restructuring Paper, January 25, 2013 achievement) xi G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/22/2010 Moderately Satisfactory Moderately Satisfactory 0.00 Moderately 2 04/30/2011 Moderately Satisfactory 0.25 Unsatisfactory Moderately Moderately 3 01/03/2012 0.99 Unsatisfactory Unsatisfactory Moderately Moderately 4 04/15/2012 2.01 Unsatisfactory Unsatisfactory Moderately Moderately 5 03/29/2013 4.13 Unsatisfactory Unsatisfactory 6 Pending Moderately Satisfactory Moderately Satisfactory 12.34 H. Restructuring (if any) 3. The Project was restructured on January 25, 2013, in order to: i) extend the closing date (to August 31, 2013); ii) add a new component (Integrated Financial Management Information System (IFMIS); and iii) to modify the Results Framework. I. Disbursement Profile xii 1. Project Context, Development Objectives and Design 1. The Project Development Objective (PDO) included supporting the Government of Iraq's (GoI) efforts to develop more effective, accountable and transparent public financial management (PFM). Cross-cutting PFM reforms were aimed at addressing the national public financial system. The primary objectives included, but were not limited to: i) Strengthening budget formulation, execution and control; ii) Improving public financial reporting; iii) Strengthening public procurement; and, 4) Developing local capacity. 2. The Iraq PFM project was driven by the GoI’s multi-year public financial reform strategy action plan (See Annexure 8) which laid out the GoI’s sequencing of PFM reforms to the international donor community. The primary target group included resource managers and their superiors in the Ministry of Finance (MoF) and Ministry of Planning (MoP), as well as line ministries. In order to maximize project effectiveness, international experts worked alongside key GoI officials in Iraq on the practical tasks that need to be supported and strengthened. 1.1 Context at Appraisal 3. Iraq is resource rich and that benefited from a substantial increase in oil revenues. Between 2004 and 2008, Iraq’s fiscal position remained strong as oil prices were on the rise. Fiscal surpluses were achieved for four consecutive years prior to project implementation due to rising oil revenues. The share of oil GDP averaged 61 percent during 2004-2008. Crude oil exports represented 98 percent of total exports and oil revenues made up 96 percent of total budget revenue (excluding grants). 4. Strategic resource allocations in Iraq suffered from inadequate attention and integration of a medium term perspective to budgeting and financial planning. Although a strategic medium term fiscal perspective existed, it was difficult to see the connection between this and annual resource allocations. The problem was compounded by the lack of effective integration between the recurrent and investment budgets both at the national (aggregate) level and at sectoral level. An ongoing issue was the relatively inadequate (low) capital investment budget execution rate. 5. Weak line ministry project evaluation and appraisal capacity led to inadequate project design and preparation. Annual implementation plans for projects were unrealistic and subject to constant revision. Contractors lacked the capacity to prepare contracts that meet the detailed specification required for the projects. The payment process was slow with the system of cash budgeting adding to problems. 6. Iraq used a predominately cash-based payments system, which provided authority to make cash payments over the fiscal year only, thus precluding the ability to make commitments when necessary. Independent audits of all Government budget 1 accounts for any recent financial year were either not undertaken, or have not been made available. 7. The final review of the second IMF Stand-by-Agreement (SBA) took place in October 2008 and was completed in December 2008. Completion of this review triggered the third and final stage of Iraq's Paris Club debt reduction agreement, for a total debt reduction of 80 percent in NPV terms. The IMF precautionary arrangement expired on March 19, 2009. Iraq received extensive assistance from the IMF under three consecutive programs (EPCA and two SBAs). Rationale for Proposed Bank Emergency Project 8. The Iraq PFMP was proposed as an Emergency Operation to be prepared and processed under OP 8.00, Rapid Response to Crisis and Emergencies, and in accordance with the Second Interim Strategy Note (ISN2) for Iraq. During this period, all World Bank Projects in Iraq were prepared under OP8.00. While PFM per se is not a sector typically associated with emergencies, it was seen that in the case of Iraq, without improvements to the PFM system sustainable longer-term reconstruction would be difficult. The WB PFM intervention sought to improve this situation. 9. The Project was seen to fit at least four of the seven objectives (OP 8.00, para. 4) for the use of the OP: a) rebuilding and restoring physical assets; b) restoring the means of production and economic activities; c) preserving or restoring essential services; and d) establishing and/or preserving human, institutional, and/or social capital, including economic reintegration of vulnerable groups. Finally, client demand to-date indicated interest in the Project. 10. As late as 2008 (See Concept Review) there was a high degree of optimism (supported by very good background work and planning by the World Bank team. There had been several encouraging developments: inter alia, Prime Minister Allawi had visited with the World Bank and encouraged a broader Public Financial Management project, a constitution had been ratified (2005), and serious discussions were underway of a system of asymmetrical decentralization that would give a high degree of fiscal autonomy to Kurdistan, Baghdad and the governorates to the South. And local elections were set for early 2009 (and were held that January). Furthermore, as this ICR report notes, at Appraisal “Iraq’s fiscal position remained strong”. With that information, the Concept Note was approved with a vision that Iraq would not only soon re-establish itself in the middle income category, but also go from “lower” to “upper” middle income to join the ranks of, for example, South Korea. BANK RESPONSE: THE PROJECT Brief Description of Bank’s Strategy for Emergency Support 11. The Second Interim Strategy Note for Iraq (ISN 2) was presented to the World Bank Board of Directors on September 15, 2005. The Bank Group's work program was organized into four pillars, supporting government efforts to: (i) restore 2 basic services; (ii) enable private sector development; (iii) strengthen social safety nets; and (iv) improve public sector governance. The endorsement of this strategy paved the way for a US$500 million IDA 14 allocation for lending in priority sectors, and for the provision of analytic and advisory work clustered around the four pillars. 12. The Third Interim Strategy Note for Iraq (ISN-3) was presented to the World Bank Board of Directors on March 19, 2009. Under ISN-3, the Bank's assistance to Iraq was organized around three thematic areas of engagement: (i) continuing to support ongoing reconstruction and socio-economic recovery; (ii) improving governance and management of public resources, including human, natural and financial; and (iii) supporting policies and institutions that promote broad-based, private-sector-led growth, with the goal of revitalizing the private sector and facilitating job creation. The preparation of this Project straddles both ISN-2 and ISN-3. 13. The primary financing instrument to achieve these objectives was the International Reconstruction Fund Facility for Iraq (IRFFI). The IRFFI assisted donors in channeling and coordinating resources for reconstruction and development in line with Iraq's priority program. This was accomplished primarily through two trust funds: i) the World Bank Iraq Trust Fund (ITF), and, ii) the UN Development Group Trust Fund. The ITF financed eighteen grants totaling US$481.6 million. 14. From the perspective of the GoI, improving PFM was a key strategic goal for long-term improvement in public sector management. The Bank had substantial previous involvement in assisting the GoI by delivering a number of analytical pieces. The framework document for overall reform of the PFM system is the Public Finance Management Action Plan (see Annex 8), the preparation of which was supported by the Bank in collaboration with the IMF. The PFM Action Plan was part of the conditions for the IMF’s Stand-by Arrangement. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 15. The Project Development Objective is to support the Government of Iraq's efforts to develop more effective, accountable and transparent public financial management. Key Outcome Indicators at approval were: - Component 1: Capital budget execution rate increases by at least 10 percentage points - Component 2 Outstanding balances in Spending Units decline by 30 percentage points, as a share of the budget - Component 3: At least 50% of contracts above threshold awarded competitively - Component 4: Ministry of Finance training institute providing training in all operational aspects of PFM project 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 3 16. The PDO was not changed during the Restructuring of January 25, 2013, but some indicators were changed and others were dropped. Annex 2 presents the details of the changes to the Results Framework and Key Indicators. See also the aforementioned table on Results Framework Analysis, as well as the discussion of components, for evaluation of the validity of the original and revised indicators. 1.4 Main Beneficiaries 17. The primary target group was resource managers and their superiors in the Ministry of Finance (MoF) and Ministry of Planning (MoP), as well as the line ministries. In addition, the Board of Supreme Audit (BSA) received significant support and capacity building through the Bank-executed portion of the Grant. Original Components (as approved) 3 18. Component 1. Strengthening budget formulation and implementation (US $3.95 million). The overall objective of Component 1 was to lay a stronger foundation for credible annual budgeting framed by a medium-term perspective and complemented by more rigorous selection and preparation of investment projects consistent with a national development (and sector) strategies) Activities included: i) Developing detailed sector strategies for two key ministries (Ministry of Health and Ministry of Education, but this is to be agreed at Negotiations); ii) Strengthening investment project preparation and appraisal; iii) Strengthening the capacity of the Economic Department of MoF; and, iv) Improving the process of annual budget formulation and budget classification and strengthening the public financial regulatory framework 19. Component 2. Strengthening Public Sector Procurement (US$ 3.2 million). Component 2 included: i) Supporting the public sector procurement system; ii) Developing sector specific standard bidding documents; iii) Establishing a procurement information and monitoring system; and, iv) Strengthening the capacity of key Iraqi implementing institutions, NGOs, and private sector in public procurement 20. Component 3. Strengthening Budget Execution and Implementation (US $2.33 million). Component 3 generally provided technical support to improve administrative and functional aspects of GoI’s budget execution operations. Activities included: i) Introducing an improved cash release system; ii) Developing a multiple 3 The component cost totals represent the best estimates for the cost of completing the ‘upstream’ consultant outputs within each component. However, they do not account for the numerous “downstream” activities which will arise from the consultant outputs. Prior to those outputs, it is not possible to accurately estimate the costs for these activities. For this reason, a large portion (20%) of the total Grant is Unallocated (see also Annex 1). In the Allocation table in Annex 4, and in the Grant Agreement, the bulk of the Unallocated has been included in the Consultant Services category, as this category represents the majority of the downstream activities; this reduces the Unallocated to less than 10% of the Grant. 4 treasury model; iii) Introducing a new commitment control system; iii) Improving financial reporting; iv) Assessing the internal financial control system, including the role of the Inspectors General; v) Creating new IT networks to improve intra-ministerial communications; and vi) Providing capacity development for line ministries to enhance their ability to implement investment projects 21. Component 4. Capacity Development and Project Management. (US $3.0 million). This component was designed to ensure the effective coordination of all of the training and capacity building efforts provided in the first three components. The component also foresaw a focused strengthening of the MoF’s internal training department. 22. Bank-executed Part of the Grant. At the request of the Recipient, a portion of the Grant was allocated for Bank-execution. In addition to the advisory services to be provided (as described above, and in paras. 47-49 below), the Bank-executed part of the Grant allowed the Project to support the efforts of the Federal Board of Supreme Audit (BSA) and other governmental entities. Specifically, support would be provided for advanced training of BSA staff in public finance, legal issues, accounting, and auditing. 1.6 Revised Components 23. At Restructuring, the original four components were retained, with some reductions in scope (see below), and an additional fifth component was added: Integrated Financial Management Information System (IFMIS). Specifically, Component 5 was costed at US$ 6 million and consisted of: a) Phase 1 – Prototype Development and Testing for the Supply and Implementation of Integrated Financial Management Information System to be conducted in the period January 2013 to August 31, 2013; and, b) Phase 2 – Pilot Implementation and Government wide Rollout based on the results of phase 1 comprising: i. Phase2A – Pilot Implementation – To commence on January 1, 2014 and conclude on December 31, 2014 ii. Phase 2B – Government wide Rollout – To commence January 1, 2015 and be completed by June 30, 2016. 24. Component 5 of the Project would support only Phase 1 of this plan. Following the evaluation of Phase 1, GoI was expected to roll-out the IFMIS on a Government wide basis (Phase 2). The justification for Component 5 was presented as follows: i) Provide capability to automate the budget preparation process to ensure complete and accurate budget formulation and presentation (to support Component 1); ii) Increase the ability of the Ministry of Finance to undertake central control and monitoring of expenditure and receipts in government Ministries, departments and spending units (to support Component 3); and, iii) Provide the Ministry of Finance with access to comprehensive and meaningful information on budgetary, financial and operational performance during the financial year (to support Components 1 and 3). 5 1.7 Other significant changes 25. Project closing date was extended to August 2013. In addition, funds were reallocated to accommodate an additional Component 5: Development of an IFMIS Prototype (see above). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 26. The Emergency Project Paper (EPP) description of “Lessons Learned” highlighted the following features: i) the importance of simple, well defined, and flexible project design to allow for adaptation to changing circumstances in a context of participation from the donor community; ii) the reality that Iraq presented additional specific challenges, because it was (and still is) a country in conflict, and the institutional weaknesses demand that project design not be overly ambitious; iii) limited possibility of Bank staff to visit Iraq, and few international consultants willing to work there; iv) in view of the above, higher than average costs for all projects, particularly to carry out adequate supervision; v) limited institutional capacity; and vi) difficulty of successfully implementing complex projects. In addition, with specific reference to fragile and post- conflict countries, the EPP stated that, “The project also takes into account lessons learned from PFM projects in fragile and post-conflict countries, including: i) ensuring robust understanding of country context and dynamics; ii) providing adequate funding and road map for capacity building upfront; and, iii) focusing on harmonization in the field and ensuring that limited country capacity is not diverted for harmonizing donors.” The EPP asserted that the design of the proposed project was to be kept simple, focusing on a few critical activities which were deemed achievable in a relatively short time-frame, and which will provide quick impacts throughout the PFM system. Factors that affected quality at entry (QAE) project implementation and outcomes are described in greater detail below (see Para 78-85). 2.2 Implementation 27. Project design provided that a Project Management Team (PMT) would be dedicated to this Project and would be supported by Component Technical Teams. However, during implementation, it evolved that key PMT members were unable to spend full-time on the Project. The PMT staff was obligated to continue with their duties at the MoF 4 . Further, the Component Technical Teams (CTTs) were never properly formulated. 28. The PMT unit appears to: 1) Have had insufficient capacity to engage in international bidding and procurement processes; and, 2) Lacked the pre-requisite 4 Nine of the eleven Project Management Team (PMT) staff, including the PMT, worked at the PMT unit on a part-time basis. 6 technical management to develop, manage, and follow-through on international tenders. Further, the payment approval procedure used was not appropriate. As a result, there were significant delays in PMU payments. All of the above factors contributed to the lack of effective project implementation. 5 . The ICR provides additional mitigating factors impacting the PMT unit below (Paragraph 38). Clearly, many of these factors (risks) should have been more thoroughly scrutinized and a sufficient back-up plan developed. 29. Low willingness of international consulting firms to continue working in Iraq. Of significant importance was the low willingness on the part of international consulting firms to work in Iraq. As a result, all international contracting took substantially longer than originally envisaged. In addition to the procurement delays, two important contracts under Component 1: Strengthening Budget Formulation was cancelled due to poor performance, and outputs for this component were not realized. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 30. The EPP did not provide a detailed monitoring and evaluation (M&E) plan. The original intermediate output indicators were significantly more substantive (e.g. with numerical targets) then the revised indicators from the 2011/2012 restructuring. For example, the original intermediate indicator for the Budget Call Circular provided for a hard target of a “new BCC with hard budget ceilings issued”. The revised intermediate indicator called for “new BCC in use? Yes or No”. The revised output measure is not as useful in measuring performance achievement because the government by law must develop and provide a new annual Budget Call Circular. 31. The monitoring and evaluation plan was meant to be developed early in Project implementation, but that did not happen, and as a result, systematic monitoring did not occur until much later in the Project. As the project accelerated its component outputs some improvements were evidenced in reviewing proposed outputs and outcomes. Nevertheless, systematic output monitoring has only taken place since early 2012, and to some extent allowed more proactive evaluation by the task team. 2.4 Safeguard and Fiduciary Compliance 32. The Financial Management (FM) and disbursement functions of the Project were managed by a Financial Management team consisting of a part time Financial Officer (FO), a full time Internal Auditor, and a part time Financial Consultant (FC). Overall, the team managed the project’s FM and disbursement functions in a generally satisfactory manner without any major complications. During the lifetime of the project, 5 The ICR recognizes that the several factors impacted the performance of the PMT project operations, including: 1) the lack of full-time staff; 2) initially low management and technical capacity of the PMT unit in engaging in managing and procuring for such complex projects. These factors contributed to the strain on PMT procurement processes. 7 the PMU unit encountered substantial turnover and several changes at the top management level including: the appointment of a new Project Management Team Director and Deputy Director. 33. The Interim Un-audited Financial Reports (IFRs) were prepared in accordance with International Public Sector Accounting Standards (IPSAS) and submitted in a timely manner. The Project’s accounts were subject to annual audit performed by an international audit firm acceptable to the Bank in accordance with acceptable Terms of Reference (TORs). All audit reports have been regularly submitted to the Bank without delays. The auditor issued unqualified “clean” opinions on all issued audited financial statements. The last audit report will cover the period from January 1, 2013 until closing date (August 30, 2013) plus the four month grace period ending December 31, 2013, due by February 28, 2014. 34. The Project suffered from delays in processing payments to beneficiaries due to a number of bottlenecks and redundant steps in the payment review procedures within the PMT and MOF. The World Bank Task Team and the PMT worked together to reinvigorate the payment process, to streamline procedures, and reduce a number of redundant steps, agreed during the February 2012 Supervision Mission. A new design for payment approval procedure was applied by the PMT, as of April 2012, and this did lead to some improvement in physical progress. 35. The Fiduciary Monitoring Agent (FMA) that was hired by the Bank for fiduciary purposes was effective. FMA staff visited the PMT two to three times per month on average to: (i) review the PMTs’ Interim Financial Reports (IFRs) and reconcile the PMTs records to the Bank records; (ii) monitor unclaimed expenditures; (iii) verify disbursement plan updates; and (iv) to provide on-the-job training in FM matters. The FMA included the PMT-prepared IFRs in their quarterly reports, as well as the adjusted IFRs (with their review comments and recommendations for the PMT IFRs). The FMA also conducted, on a sample basis, pre-screening and post- review of withdrawal applications for direct payments and reimbursements. Fiduciary compliance under the Project was generally good. There were no significant waivers from the Bank’s fiduciary policies and procedures during the implementation of the Project. The ICR would like to note that the fiduciary monitoring agent (FMA) played a noteworthy (valuable) role in this project. As such, one of the lessons learned is that the FMA (or a similar arrangement) is an essential feature of World Bank projects not only in Iraq, but likely in all post-conflict situations where normal Bank supervision is constrained. 36. The Project’s procurement process itself proved problematic. Every implementation status and results report through-out the life of the project emphasized the challenges in the PMTs technical and management capacity, and ability to support Bank procurement processes. From the outset, procurement processes and procedures were slow to evolve. The PMT appears to have lacked the required technical management and skills, to engage in relatively routine procurement processes and procedures. The relative difficulties in completing procurement awards can be attributed to a number of factors including, but not limited to: i) Changes in PMT management and 8 project staff; ii) the Part-time nature of PMT staff; and, iii) Overreliance on PMTs Procurement expert to: a) prepare the Terms of Reference(s) (ToRs) and Requests for Proposals (RFP); b) process the selection and management of contract negotiations with limited technical support. 37. During the life of the project, the procurement processes did gradually improve. Project documentation and archiving are considered to be satisfactory. Current delays in contract evaluation and awards have been largely attributed to technical issues rather than procurement specific issues per se. 38. No Safeguard policies were trigged for this Category Project. 2.5 Post-completion Operation/Next Phase 39. With the extension of Iraq Trust Fund (ITF) until December 2014 a new Technical Assistance Project on PFM Implementation Support (TA-P149073-TAS- TF016110) has been launched (under the ITF in November 2013). This technical assistance is expected to carry forward those projects that the Ministry of Finance considers priorities. The follow-on PFM project has four (4) components: (1) Design of Integrated Financial Investment Management Information System (IFMIS); (2) Conducting a Diagnostic Study on Public Investment Management (PIM); (3) Enhancing Ministry of Finance Website; and, (4) Building Capacity of the staff of the Ministry of Finance. The new Project’s cost is estimated to be USD2.48 million with a project deadline of October 30, 2014. 40. For the Bureau of Supreme Audit program: The independent evaluation report and the knowledge dissemination support the report’s conclusions and recommendations, and were taken into consideration while designing a follow up capacity development activity. An approved grant of USD 0.4 million from the Iraq TFCBII fund as part of a comprehensive FM program, is ready to be implemented. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 41. The World Bank’s project objectives and strategy were closely aligned to Iraq’s National Development Strategy. Bank support to Iraq over the period is based on supporting the delivery of some of the country’s most important national priorities. Bank support to Iraq emphasizes national programs to improve the lives of Iraqis across the country including in the areas of health, education and public financial management. 42. Relevance of Design: The relevance of project design is rated Modest. The results framework did not present a convincing causal chain between activities, outputs and outcomes with respect to the projects objectives. Projects that are complex, multi- faceted and that involve process and attitudinal changes from a multitude of government actors require much more scrutiny, analysis, and more informed project design from the outset. 9 3.2 Achievement of Project Development Objectives (Efficacy) 43. The Project’s initial design consisted of a combined total of twenty outcome and output indicators. Of the four PDO outcome indicators: a) one indicator was dropped entirely; b) one indicator was never measured during the life of the project so there is no link to progress; and) one indicator is largely an output indicator which is not S.M.A.R.T. compliant. Of the original 17 output indicators, four (4) indicators, or approximately twenty-five (25%) percent, were dropped from consideration. Further, two intermediate results output indicators appear to be so similar in nature, that it is difficult to distinguish the unique characteristic of the indicator or the output achieved. 44. The ICR rates achievement of the First Component as Unsatisfactory. A brief overview of the outcomes and outputs is provided in the following sections: 45. First Component: Strengthening budget formulation and implementation. Effective budget preparation processes with clear linkage to a multi-year strategic framework will assist the government in reducing budget variations, and improving the quality and efficiency of government expenditures. At the outset of this project PEFA (2008) describes the Iraqi budget call circular in the following manner: ‘The budget call circular is issued late and does not provide expenditure limits (ceilings).’ The first objective appears relevant. 46. Outcome Indicator Component 1: Capital budget execution rate increases by at least 10 percentage points. The capital budget execution rate was seventy-five (75%) in 2008. The proposed outcome indicator for this component was to increase the capital budget execution rate by ten percent (10%), to eighty-five (85%) percent. According to figures provided by Ministry of Planning (MOP) project execution rates reached seventy-eight (78.5%) percent in 2012. 6 The outcome indicator appears to be in the correct direction, but has not achieved its predicated outcome. A variety of factors play into this modest increase in execution rates: i) inadequate project design; ii) delay in cash releases to line ministries; and, iii) from a medium term budget perspective an inadequate linkage between medium term budget planning and near term budget preparation process. In the section below the ICR reviews the performance of Component 1’s output indicators. 47. The intermediate results indicator for this component consisted of five (5) output indicators. The actions and outputs in support of this component are discussed below: 6 The WB-Public Expenditure Review (PER) indicates capital spending efficiency rates for the Government of Iraq at/around 56% for 2012. The PER obtained their data from various MOP data sources. The ICR Review obtained development (investment) budget spending efficiency rates from the Ministry of Finance. Although, there is some difference in the measurement of capital spending efficiency rates, the overarching conclusion remains the same:- capital budget execution rates did not rise to meet the targeted rate of 85% as presented for this Project’s PDO indicator. 10 a) i. New budget call circular (BCC) designed and in use; and, ii. New budget call circular (BCC) in use. It is difficult to determine the actual output achieved as a result of this technical assistance intervention. GoI is required by Public Finance Act to produce two primary circulars (recurrent and investment) during a budget implementation period (See Results Framework Analysis, above). However, clear linkages between investment budgets and forward expenditure estimates are not readily observable. While some projects are selected based on sector plans, and some of these have recurrent cost implications, these estimates are not generally shared during operations budgeting. Moreover, the forward estimates that do exist are not developed from such costings. Expenditure ceilings are a major feature of the Budget Call Circular, but what is the relevance of expenditure ceilings when operational and investment spending rates are approximately 90% and 78.5%, respectively. b) Draft Sector Strategies: The Project proposed developing sector strategies for two key sectors for Ministry of Water Resources and Ministry of Education. The drafting of these two sector strategies was cancelled because these sector ministries asked other donors to provide assistance. The cancellation of the two technical assistance contracts adversely impacted the Bank’s assistance with respect to the developing a medium term framework for two key ministries. There are only three (3) sector strategies that are available: Energy, Industries, and Education. At present, none of these sector strategies has been adequately reflected (integrated) in the Budget. c) New Macro-Economic Fiscal Team Established in MOF – A Macro-Economic Fiscal Framework (MTFF) Team has been established with the Budget Department (and, not the Economic Department) since 2010/2011. The Iraqi Medium Term Fiscal framework uses the IMF financial programming modeling procedures and has developed a database of historical fiscal information. The MTFF supports estimation of fiscal forecasts and the annual development of top-down indicative budget ceilings. A rule-of-thumb measure for the usefulness of the MTFF, can it be used for assisting in other work? In fact, the MTFF has proved somewhat problematic in that it could not be used for some ESW undertaken in 2013 (e.g. Construction of the Boost data tool in the Water Sector). The MTFF is still largely inundated with poor or inappropriate data and the available information is considered to be generally unreliable. Moreover, the current MTFF excel file cannot be updated by Iraqi nationals-it has to be updated by an external consultant who comes once a year for such a purpose. The development of a Medium Term Expenditure/Budget Framework (MTEF/MTBF) has not been accomplished. At present, the Ministry of Planning has developed a 5-year Project Investment Plan (PIP). The PIP is combined with a 1- year forward operational budget to produce an annual budget. The MoF (operational budget) and MoP (development budget) work largely independently. It is difficult to consolidate budget data from both the operating and capital budget. Donor projects are not generally captured in the Iraqi budget. Donor grant-funded projects are captured in a separate database facility. Although policy discussions have been 11 introduced into the budget process, there is no effective bottom-up Medium Term Budget Framework. d) New Procedures in Place for Project Preparation and Appraisal – A review of various ISRs (2009, 2012) indicates that there was effectively no improvement in the implementation of new projects. ISR (9/2013) claims a 10% improvement in capital budget execution performance as measured by MOP and other relevant entities. However, this did not take place. FY2012 capital execution rates as stipulated by MOP were 78.45%, barely 3.45 percent above the original baseline estimate established in 2008/09. Project execution rates estimated up and until October 2013 are estimated at 40.3%. While new procedures and guidelines were presented to MOP, much of the on-the-ground technical assistance work and project preparation guidelines were established by USAID (which used the Project Management Institute Training Handbooks and Certification process) and Project Evaluation check-list. 48. Based on the above assessment, the ICR rates achievement of the Second Component as Moderately Satisfactory. A brief overview of the outcomes and outputs is provided in the following section: 49. Second Component: Strengthening Public Sector Procurement (Improvements to the System of Public Procurement). Effective public procurement practices are the cornerstone of achieving any success in public financial management expenditure efficiency, service delivery improvement and accountability. The intermediate results indicator for this component consisted of four (4) indicators. 50. Outcome Indicator Component 2: At least fifty (50%) percent of contracts above threshold awarded competitively. This PDO outcome indicator was dropped due to inadequate baseline information, and no replacement indicator developed. Hence, even if some of the intermediate outcome/output indicators were achieved (see below), it is difficult to determine the level to which public sector procurement was strengthened. In the section, below we review the performance, actions and outputs in support of Component 2’s objectives. a) Standard Bidding Documents (Finalized). The project completed seven (7) sector specific standard bidding documents (SBD) finalized in July 2013. These sector specific bidding documents appear on the MoP website(s). However, in order to make SBDs mandatory, these documents need to be tested or piloted. This process involved training the procurement specialists in understanding the SBDs, and then applying these SBDs to a procurement process. b) National Procurement Manual (Finalized) The National Procurement Manual was prepared in line with the current National Procurement Law and submitted to GoI in Final Draft form awaiting their consent and final approval. The manual remains prepared but has not been finalized nor distributed. A national procurement strategy has been developed, and capacity building efforts have been adopted by Ministry of Planning. 12 c) Procurement Bulletin Board. The existence and operation of a procurement notices via a single portal was to be the cornerstone achievement of this component. Regulations published in 2008/09 highlighted the procurement process with a clear time frame for resolution. Under the PFM project only the Terms of Reference (ToR) and road map for implementation were prepared in cooperation with USAID. However, the website itself has not been implemented, partly due to ongoing design issues. Ministry of Planning has allocated a budget to implement this procurement bulletin board (as part of its 2013/14 Governance Action Plan). The procurement bulletin board itself was not completed. d) Training Needs Assessment. A national training strategy was developed and capacity building efforts in procurement established. The World Bank (and other Donor Agencies including USAID Tarabot) provided training on procurement policies and procedures. A number of training programs were provided for in the draft procurement manual, training on Standard Bidding Documents, and training of trainers. 51. Third Component: Strengthening budget execution and implementation. This component primarily is designed to improve the efficiency of government of expenditures and provides important information on the credibility of Ministries, Department and Agency (MDA) budgets. Credible MDA budgets facilitate realistic cash flow forecasting and reporting, and mitigate the frequency of in-year requests (virements) for adjustments to budget allocations. The intermediate results indicator for this component initially consisted of seven (7) indicators. Three (3) indicators (33% of the total) were dropped and one indicator was not achieved. 52. The ICR rates achievement of the third component as Moderately Satisfactory. A brief review of Component 3 is provided in the following sections: 53. Outcome Indicator Component 3: Outstanding balances in spending units decline by thirty (30%) percentage points as a share of the budget. The original PDO provided that the outstanding balances in spending units as a share of budget in 2008 was thirty-eight percent (38%). Such large cash balances as a percent of budget are generally attributed to poor budget formulation, development projects that have been poorly designed and are not at the stage where procurement would be effective, and in the case of Iraq - prefunding of spending unit accounts. Further, large cash balances directly reflect the inadequate linkage of procurement plans to ministerial cash management plans. 54. Technical assistance was projected to translate to a 8 percent reduction in cash balances as a percent of budget, or at the end of the project cash balances be less than thirty (30%) percent of total budget. Three Project ISRs provided that the cash comparisons of results with the baseline will be done, but no comparison with the baseline was ever developed. 13 55. The Project’s ISR-3 and 4 indicated that there was a reclassification of accounts in 2009, due to which the Cash Balances at 31 December 2009 were reduced to 3.843 trillion (5.5% of the Budget). The cash balances stood at 5.937 trillion ID on June 2011 (6.1% of the budget) as per the new classification. Thus, there is a problem in comparing the progress with the baselines as the definition of indicator has changed. The way forward with respect to this indicator was to estimate the present cash balances as per earlier classification by including provincial accounts, accounts of government trading enterprises, and purchases of vehicles. However, this was never done and cannot be done during this ICR. While the need to change the indicators may be identified during the Mid-term review, indicators are changed through formal restructuring approved by the Board or Management (as applicable). 56. A quick review of the current status of a number of ministerial cash balances, indicate that cash balances of line ministries are IRD 3.9Trillion (September 2013). Without reviewing the 640 plus spending unit bank accounts at the Central Bank of Iraq, it is difficult to determine whether Indicator 2 has been achieved. Further, it is not possible to attribute this improvement directly to the TA provided under the project. 57. A review of the output indicators for Component 3 is as follows: a) First round of training on new Cash Release System (CRS) completed with several ministries. The PFM project was able to introduce rules and guidelines for introducing a commitment control system and improving financial reporting. The Project Management Unit insisted on implementing the Commitment Control/Cash Release system before providing any Capacity Building Exercises. A capacity building workshop was held where 87 persons attended. The Cash Release System was not rolled-out. b) Detailed Rules and Regulations for New Commitment Control System Developed. The PFM project introduced rules and guidelines for cash management and commitment control system and improving financial reporting. PMT insisted on implementing the Commitment Control/Cash Release system before providing any Capacity Building Exercises. Eighty – seven (87) persons did complete basic training on cash release systems. Training for other line ministries has not been extended. However, the ICR review questions the validity of this indicator which provides that the new system of commitment controls must demonstrate a 30% improvement in the coherence between cash releases and actual needs. What is it that this indicator is actually measuring? c) Assessment of Financial Control arrangements completed. The Government of Iraq had contracted Ernst and Young (E&Y) to perform a status analysis of the current internal financial control arrangements in four ministries and a roadmap describing the way forward in implementing reforms to the internal financial control function in line ministries through the adoption of best practices. The study identified the need for developing a holistic public internal control strategy across the entire 14 public administration to establish complimentary roles for the various control-related institutions and activities and reflecting the objectives of this strategy in primary and secondary legislation. The study reported various gaps between the current status and international standards and good practices in the legal framework, governance, staff, infrastructure and operations. Building on the successful outcomes of this study and the reported short-Mid-Long term reform recommendations, the World Bank is currently providing Technical Assistance (TA) to the Ministry of Finance in implementing selective short term reforms that were reported on the study. This TA is financed by Iraq TF under the operation named "Improving Governance through strengthened FM program. Owning to the limited time-frame of this TA (12 months), and the significant and deep changes in the financial internal control structure as reported in the study, the TA is helping the GoI in the short-term reforms areas that are believed to be completed within the TA period. d) Website being regularly updated. MoF hired Abacus Consulting and Ask Tech Links to upgrade its website with following objectives: i) Introduce a modern and sound Content Management System (CMS) & web portal using SharePoint 2010 (Internet & Intranet) for MOF staff and general public; ii) Increase the operational efficiency and effectiveness of Ministry of Finance staff through automation of their files, directories and documents based upon their roles & privileges with the enforcement of the approval mechanism; iii) Ensure the availability of information with on as and when required basis through the implementation of Internet/ Intranet portal; and, iv) Facilitate MOF staff to manage support requests /issues with a single point on control through centralized Help desk application feature. During the project, amongst other things, the company delivered the following scope of work: 1) Designed and developed Intranet secure website covering Document & Content Management and centralized Help Desk with notifications; 2) Developed Internet Portal covering News Bulletin, Calendar Events, News Bar, Currencies Exchange Rates, News Archive, Search, Updateable content pages, Photo Gallery, Discussion Forum and Help Desk (customer care) features; and, 4) Provided dedicated hosting services for 3 years for internet website. The new website includes functionalities like secure Intranet, Webmail, help desk functionalities, secure shared directories, dynamic contents management, etc. Ministry of Finance has asked for additional refinements. e) The following three activities and outputs were dropped from this component: 1) New Financial /Fiscal Control System; 2) Training Materials for Financial-Fiscal Control System; and, IT assessment Completed and functional requirements for upgraded system. 58. Based on the following assessment, the ICR rates achievement of the Fourth Component as Moderately Satisfactory. The section below provides a brief discussion and review of some of the key component outputs. 15 59. Fourth Component: Capacity Development and Project Management. The activities and outputs under this objective were designed to develop the human resource capacity of predominantly the Ministry of Finance and core line-ministries. 60. Outcome Indicator Component 4: Ministry of Finance training institute providing training on all operational aspects of PFM project. This “outcome indicator” is inappropriate as it is too narrowly defined: even if MoF staff were trained on operational aspects of this Project, it would not translate to progress towards achievement of the PDO. A more appropriate indicator would have been, e.g. ‘Ministry of Finance training institute providing training on all operational aspects of public financial management”. a) The key objective was the establishment of a fully functional training program at the Ministry of Finance Accounting Training Center (MoFTAC). This has not been fully achieved. A number of trainer-of-trainer programs have been held. However, a fully functional Finance and Accounting Training program has not been developed and implemented. b) A review of the current progress of the Bank’s technical assistance inputs, indicate that a Training Needs assessment was completed, and a few training of trainer sessions were provided. A Training Needs assessment was completed, a few training of trainer sessions were provided. There is inadequate depth of training ability in any of the key components of public financial management. The detailed needs assessment provided to MOFTAC by Coffey LLC has been adopted by MOFTAC. MOFTAC has put forward a 5-year cumulative budget estimated at USD13.5 million with a detailed training plan. Subcomponent on Board of Supreme Audit (BSA) (Bank-Executed Technical Assistance) 61. The technical assistance for the BSA capacity development program was jointly designed with the BSA and was undertaken through a companion trust fund executed by the Bank. As noted above, this Bank-executed part of the Grant was designed to allow the Project to support the efforts of the BSA and other governmental entities to reinforce their operational capacities. Specifically for the BSA, support was provided for advanced training of BSA staff in public finance, legal issues, accounting, and auditing, etc. The ICR rates achievement of the Technical Assistance as applied to the Bureau of Supreme Audit (BSA) as Satisfactory. The following section provides a brief review of the program as it pertains to the BSA (for more detailed discussion see Annex 2). 62. The BSA technical assistance modality was designed to cover the performance of focused peer reviews based on pilot audits in cooperation with the knowledge-providing peer SAI. The program completed this task in three primary areas: 1) risk-based audit; 2) performance audit, and 3) procurement audit, in cooperation with the UK National Audit Office (UK NAO), Netherlands Court of Audit (NCA), and 16 French Court of Accounts, respectively. The technical assistance provided training to more than 15 master trainers, 380 staff, on risk-based audit, specialty audits, performance audit, and procurement audit, investing around 12,700 training hours. The effectiveness of this program was put into action when a pilot audit of one of the Bank-supported projects in Iraq was carried out by the BSA in compliance with international auditing standards and following training on Bank project implementation and fiduciary guidelines and knowledge exchange with a SAI which audits Bank-supported projects (the NIK). 63. The process of revising the risk-based audit manual was initiated through cooperation with both the UK NAO and the Polish State Audit Office (NIK). The development of the procurement audit manual was initiated through cooperation with the NIK. The forensic audit manual was initiated through cooperation with the French Court of Accounts. The program provided basic knowledge and exposed the BSA to experiences in specialized subjects such as oil audit, forensic audit, arbitration, contract management, governance, auditing of World Bank-financed projects, and Monitoring and Evaluation. The BSA technical assistance program had addressed the organizational capacity including strategic planning, annual planning, and exposed the requirement of a change management process. 64. Frequent evaluation surveys were run after each workshop to get the feedback of the participants and guide further planning. A mid-term high management level event joined by key global SAIs (NCA, UK NAO, French Court of Accounts, NIK, Canada Auditor General Office, and Malaysia Auditor General Office), CoMSEC, INTOSAI-IDI, and World Bank CFO, Country Director, and Sector Manager, was organized to share knowledge on global experiences and providing advice to the BSA. The program, as well, provided a platform for dialogue and joined knowledge sharing among the pillars of public governance in Iraq, including the BSA, Commission of Integrity, the Inspector General Offices, MOF, and internal controllers at line ministries on cross-cutting topics such as control, audit, and inspection. One of the program workshops provided a joint input to the Iraq Governance Action Plan. A stock- taking was facilitated of the knowledge-dissemination process carried by the BSA trained master trainers and conducted by the NIK, generating a set of recommendations. During the program the BSA trained master trainers had disseminated training on risk-based audit, performance audit, and procurement audit to around 150 investing around 2,500 training hours. A pilot audit of one of the Bank-supported projects in Iraq was carried out by the BSA in compliance with international auditing standards and following training on Bank project implementation and fiduciary guidelines and knowledge exchange with a SAI which audits Bank-supported projects (the NIK). 65. An independent evaluation of the technical assistance program provided to the BSA was performed at the end of the project, by an independent evaluator, and highlighted the successful achievement of the key project objectives. The program was evaluated as having produced a perceptive and thorough analysis of the BSA’s capacity development and training needs based on the best advice available – the 17 guidance on SAI capacity building approved by INTOSAI, having delivered a comprehensive Train the Trainers’ program for the BSA that introduced the organization to modern, up to date audit methodologies relevant to its development needs, and having the BSA taking active steps to implement these new audit approaches. 66. Fifth Objective: Development and Testing of Integrated Financial Management System (IFMIS) prototype. On the basis of the approved indicators (and, not on new indicators for a follow-up project), and based on the above assessment, the ICR rates achievement of the new Fifth Component as Unsatisfactory. The following section provides a brief overview of the components outputs. 67. Component 5 was started relatively late in the implementation cycle and is a new component. The intermediate results indicators include: 1) Prototype for IFMIS developed; and 2) Prototype for IFMIS tested. This was a Level 2 restructuring approved by the Country Director However, the ISRs do not reference these (approved) indicators, but rather provide: i) The development of a System Requirement Study (SRS) to be completed; and ii) The Government’s preparation of a Request for Proposal (RF) document that can be released for acquisition and implementation and implementation of the IFMIS. 68. Approximately USD 6.0 million of the original USD 18.0 million (or, 33% of the original facility) was left unspent, with approximately 1 year left until project expiration. In a bid to fully utilize these unspent resources (before project expiration), an attempt was made to reinvigorate the project by refocusing these funds on a limited, but ambitious, Integrated Financial Management Information System (IFMIS) prototype project. The underlying premise of the project was to develop a prototype financial management system using the Freebalance Inc. Accountability Suite V7.0. Freebalance is the management system that currently underpins the Iraqi Ministry of Finance. The Iraqi financial management system was largely not upgraded or expanded after 2006 when the prime contractor withdrew from Iraq. The award of a contract to Freebalance for the development and testing of the prototype was negotiated, but the World Bank temporarily suspended Freebalance, following which the contract could not be signed. Many project team members however, point to the suspension of Freebalance as the ‘death knell’ of what could have been an otherwise successful attempt to introduce an updated prototype management information system to Iraq. 69. There were numerous risks that the Project Team did not appear to fully appreciate when adding this new component under the project. Considering the original project closing date June 30, 2013, the remaining time left under the project was less than six (6) months at the time that the new component was added. If there were any even slight delays, at any stage of implementation, there was a substantial risk that the task would not be completed within project time. As a result, the Project team requested and was granted a two month extension to end-August 2013. The question remains whether this eight (8) month period still would have been sufficient. 18 70. The issue of project procurement also tends to be of overarching concern. Every ISR review listed PMT procurement as a major impediment to project implementation. As a result, delays in procurement of hardware and the software would not have been an unwarranted concern. Discussions with PMT project team members indicate that there was a general reluctance to engage in the procurement of Freebalance for a variety of reasons. The most important consideration brought to the forefront by the PMT management and operating team is that they did not fully understand how the procurement of Freebalance (financial management software) was to unfold. There was substantial apprehension among PMT members in understanding and negotiating the software licensing arrangements. Even if the Freebalance procurement would have unfolded, in all likelihood, there would have been a number of bottlenecks along the way. 3.3 Efficiency 71. The EPP did not contain any cost effectiveness analysis, nor were any other economic or financial analyses performed during Appraisal (e.g. NPV calculation, economic or financial rates of return, etc.). In addition, there is limited useful information available to compare this project to similar projects in other countries. Another complicating factor is the large unallocated portion at Appraisal, which accounted for 20% of the Project total (see Footnote 4 above). Therefore it is difficult to rigorously assess the efficiency of the project according to the established ICR criteria. Nevertheless, Efficiency is rated Moderately Unsatisfactory based on: i) the non- utilization of most of the Unallocated portion, due first to the failure of any of the envisaged downstream activities to materialize, and then the failure of the IFMIS component; ii)the termination of several contracts due to poor performance, and hence reduced outputs compared to the plans 7; iii), poor payment efficiency (delays in releasing payments); and iv) higher than initially expected costs due to the cost of providing protection and transport to the foreign consultants. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory (see above) 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Not Applicable (b) Institutional Change / Strengthening 72. The independent evaluation of the BSA technical assistance program provided instrumental guidance for the BSA and donors while proceeding with a 7 Points i) and ii) are largely responsible for the low final disbursement position – only US% million 9.78 disbursed of US$ million 16.0 allocated (61%) 19 sustainable institutional building process. More precisely, the evaluation advised the BSA to invest in further structuring its professional training program, strengthening its methodological unit and quality assurance arrangements, completing the revision of its audit manuals, and in further developing its strategic capacity to prioritize its work and activities in line with its strategy. The independent evaluation further advised Donors of the need to define some precise medium term objectives for the impact of their support on the BSA’s professional capacity and capability to ensure that the BSA continues to raise the quality of its audit work in compliance with international auditing standards as defined by INTOSAI. 73. The ICR finds that with respect to Procurement there is some positive institutional change. Although in a nascent phase, the pilot ministries are beginning to use sectoral SBDs. However, it is clear that there is still substantial work that needs to be undertaken to ingrain an effective public procurement process into the mainstream. (c) Other Unintended Outcomes and Impacts (positive or negative) Not Applicable 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 74. ICR did not undertake a beneficiary survey. A project wrap-up roundtable was held with key stakeholders. A brief write-up is presented in Annex 6. 4. Assessment of Risk to Development Outcome 75. Rating: High. The achievement of the development objective is Moderately Unsatisfactory reflecting failure of several key consultancies and project components to deliver their intended outputs. In view of this alone, the risk to the development outcome is High. The ICR team reviewed the provision of the technical assistance as applied to the primary targeted beneficiary ministry (MoF):- there is insufficient evidence to conclude that the capacity building activities of this Project significantly increased the MoF’s capacity. For example, as discussed above, the outcomes related to budget formulation and execution were not completely realized. The risk to the development outcome remains High. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Unsatisfactory. 76. The Bank preparation phase consisted of intensive dialogue with the government (primarily MoF and BSA) to determine where a Bank project could most effectively intervene to support the government’s PFM Action Plan. The specific project activities developed were considered to be appropriate and timely. Importantly, the Bank worked to ensure that project activities would not duplicate the work of other donors and organizations (especially the IMF, USAID, and DfID). 20 77. There were a number of key flaws in design, which can be seen as barriers to higher Quality at Entry 8: i) insufficient understanding of the political economy of PFM reform; ii) complexity of technical interventions (as opposed to the ‘simple’ design described in the EPP; iii) inadequate understanding of and planning for the difficulties the PMT would confront, particularly regarding capacity constraints and team composition; iv) the large unallocated portion of the project due to over-ambitious expectations for downstream performance, and inadequate back-up plan preparation in the event that upstream components under-delivered; v) a challenging understanding as to how to effectively coordinate Project activities with the KRG and, vi) there was no Monitoring and Evaluation (M&E) Framework established prior to (or during) implementation. 9 78. Political Economy of PFM Reform. During project preparation and entry, the PFM Action Plan had been agreed between the donors and the GoI and the MoF appeared to provide a reform champion at a high level. These factors combined to create the impression that there was strong GoI demand for comprehensive PFM reform. While this may have been accurate for some elements of the PFM sector, it became clear during implementation that any such demand was not in fact widespread. It is less clear whether the Bank team could have foreseen this lack of country counterpart buy-in considering the speed of preparation. A more comprehensive political economy analysis may have assisted the project preparation team in refining and narrowing the scope of the Project. 79. Complexity versus Simplicity. The project design attempted to engage in relatively simple technical solutions. Nevertheless, these solutions required significant input from international consultants. This process turned out to be relatively complex. Further, the total package of interventions was quite ambitious considering the fragile conflict environment context and the capacity of the PMT. 80. PMT capacity. The issues with PMT capacity have been raised above (Section 2.1, Other Factors), and are further elaborated in the Lessons Learned. 81. Large Unallocated Portion. One of the key problems throughout most of implementation was the very low disbursement rate compared to the total Project allocation. Project design foresaw that upstream activities would be completed in the early phases, and these would lead to demand for additional downstream activities. However, Project design did not develop any adequate backup plan for the unallocated funds in the event of slow progress, or failure, of upstream activities. The unallocated funds created an overhang which skewed disbursement ratios. 82. In retrospect, it would have been more appropriate to focus on a narrower set of clearly defined outputs and have more specific (targeted) cost estimates. Once 8 While the ICR finds flaws in QAE, these do offer important lessons for future operations – please refer to the Lessons Learned section for further detail. 9 In addition, the ICR questions whether the use of Emergency procedures was appropriate for this operation; that discussion is elaborated in the Lessons Learned section. 21 those materialized, the GoI could have returned to the Bank for additional funding assistance. While the disbursement ratio issue is primarily internal to the Bank (and is not insignificant), the client-side presents the larger problem. The issues with the IFMIS gambit are presented in other sections. For the purposes of the QAE discussion, the large unallocated portion remaining towards the end of the project without an adequate back-up plan created difficult program choice incentives. 83. Relationship with KRG. Representatives of the KRG government were present in all phases of preparation, including Project Negotiations. As a result, Project design planned for a robust framework for engagement between Baghdad and KRG authorities. Early in implementation, an MOU was signed to formalize these plans. Unfortunately, the political dynamic was significantly more complicated during implementation than expected. As a result, the KRG did not participate as expected in the Project (though they did pursue PFM reforms via other means). 84. Lack of M&E Plan. The lack of a Monitoring and Evaluation Framework (M&E) prior to implementation is a major shortcoming of Project preparation, and clearly has led to difficulty on the part of all parties to assess Project progress and take proactive measures as needed. In addition, it has made it difficult to assess Project achievements at the ICR stage. 85. Overall, while the relevance of the objectives and design are rated Modest, and the PDO is relevant to the Bank’s strategy in Iraq, the above shortcomings in design lead to an Unsatisfactory Rating for Quality at Entry. (b) Quality of Supervision Rating: Moderately Unsatisfactory except Fiduciary which was Satisfactory 10 86. The early phase (2009-2011) of supervision was characterized by an intensive level of interaction with all main actors in the PFM sector. The second Project TTL was based in Baghdad, and chaired the Public Finance Working Group. The PFM Working Group comprised all main donors active in PFM and key members of the GoI. This group met monthly and worked to ensure that the various interventions were mutually reinforcing and to avoid duplication of effort. One of the main successes of this early phase was the convening of a high-level Budget Strategy workshop in May 2010 (attended by the major donors and over 40 GoI representatives). As a result of this workshop, a budget strategy for 2011-2013 and a MTFF for 2011-2013 was produced. These outputs, particularly the budget strategy were expected to pave the way for greater progress towards meeting the PFMP development objectives. 10 (See Section 2.4 Safeguard and Fiduciary Compliance, paras 37-44 above. Safeguard compliance is not applicable, as this was Category C Project). 22 87. By February 2011 ISR it was clear that whatever progress had been made during the initial implementation phase of the Project was in jeopardy. Implementation Progress had been downgraded to Moderately Unsatisfactory, with the assumption that an upgrade would be possible if technical assistance contracts were signed in a timely manner time. Delays in contracting, the lower-than-expected price of the winning bids combined with the increasing level of unallocated funds were starting to appear as major impediments to the successful completion of the Project. The Unallocated funds prompted an early need to consider reallocation of some of the proceeds. The MoF was already suggesting activities for the unallocated funds. Despite these abovementioned issues, the Bank team noted in ISR-3 “good progress on efforts to procure the consultants for the 5 large packages, covering all project components and 50% of project funds, and a sixth tender for the development of the MoF website…”. 88. Mid-Term Review (MTR). During the mid-term review (MTR) in December 2011, progress remained slow. The MTR ISR noted, “There was inadequate supervision and implementation support from the Bank side. There were delays in giving ‘no objections’ and the PMT was not given appropriate guidance at appropriate time leading to low disbursements, low commitments, large proportion of unplanned funds, and absence of an adequate M&E system”. 11 Notwithstanding, approximately 43% of project funds were committed and additional 17% were in the process of being contracted. But, there was still 39% unallocated funds. The MTR also recommended changes to some key indicators, particularly the PDO#3 on cash balances. 89. In an effort to utilize the Unallocated funds and maximize the impact of the Project, the Project Team embarked on establishing an Integrated Financial Management System (IFMIS) prototype. The MTR recommended restructuring the Project. The issues with the IFMIS prototype have been covered in other sections. For the purpose of evaluating Bank supervision performance, the main issue is the timing of restructuring (see below). 90. Day-to-day performance of the project did increase towards the later stages of Implementation. Several factors contributed to the increase in the project’s day-to- day performance during the final two years of the project, with the most important being a significant increase in spending efficiency (as measured by disbursement levels) compared to the project’s initial two years. Project performance improved largely owing to the Project team’s ability to work with MoF to streamline the payment system and establish and utilize the Bank’s e-disbursement platform. In addition, the Bank team was able to persuade the MoF to hire full-time coordinators for two components. Nevertheless, this later stage of supervision was also characterized by failure of two key contracts. The ICR notes that this is particularly due to the inability to move forward with the IFMIS, due to the suspension of the contractor. 11 This ISR also noted the issues with PMT staffing and MoF leadership, which is highlighted in the Borrower Performance section below. 23 91. At Project Closing, US$ 6.22 million (38%) was cancelled from the original US$ 16 million. The ICR notes that the Bank teams worked diligently throughout Supervision. While there are many factors involved in the poor performance of the Project a few things stand out on the Bank supervision side including: a) Quality at Entry. A number of QAE issues created difficulties from the beginning of supervision. However, the WB supervision effort was not sufficiently proactive in the early stages in terms of mitigating them, particularly regarding the monitoring system. b) Lack of M&E Framework. An early effort was made to engage MoF on establishing the M&E system, but this was not systematically followed up. As a result – and despite mentioning the problem in all ISRs - no formal M&E system was ever implemented. c) Fragile Conflict State Issues (i.e. Bombings at Ministry of Finance). During the project implementation period, the Ministry of Finance was bombed several times. This directly affected the efficient operations of the PMT, as they were forced to disperse among multiple office locations, making face-to-face collaboration much more difficult. Further, the Minister of Finance himself was replaced (twice). d) Provision of timely support to the PMT, especially on procurement. The early phase of supervision did not adequately provide the support that the PMT needed in carrying out complicated international bidding processes. This was rectified to some extent by mid-Project and procurement actions. e) Failure to increase staff availability. The lack of full-time staff for the PMT, which de facto meant a lack of full-time concentration on the Project, was a problem throughout implementation. The supervision teams made repeated efforts to encourage allowing PMT staff to have more time to work on the Project. These efforts were by-and-large were not successful (with the exception of the two full-time staff noted above, and this in any case came late in implementation). The ICR would like to stipulate in this section that the lack of a full-time dedicated team focused on implementing this complex project, as well as PMT staff turnover, was a major impediment to successful project implementation. f) Delays in Restructuring. From the time of the first agreed discussions on restructuring (December 2011), it took over one year for the restructuring to be made effective (January 2013). g) Delays in Modification of the Project Results Framework. The ICR considers the lack of early modification of the Results Framework as a major shortcoming in Supervision. From at least the MTR, it was clear that a number of the indicators were not appropriate, and needed revision. The MTR would have been the optimal time to revisit these indicators and propose changes. Such a discussion with MoF also might have acted as a catalyst towards establishing a primary M&E system. 24 h) Of particular importance would have been to revise the Outcome Indicators for Components 2 and 3. Component 2 indicator was later dropped, and no replacement was created, leaving it very difficult to assess this Component’s contribution to the PDO. The cash balance indicator for Component 3 was known to be flawed, but it was retained throughout implementation. In fact, this indicator’s value was further reduced when it was changed to a simple binary “Y/N” during restructuring. Hence this indicator also is of little use in evaluating the component’s contribution to PDO achievement. The changes to a few indicators at restructuring appear to reflect an effort to retrofit the indicators to what might be expected at Project close. i) Was Restructuring appropriate? Or would Cancellation have been a better option? The Restructuring Paper (RP) (Report No. 74235-IQ) provided a number of rationales for the restructuring. From the perspective of the achievement of the PDO, the RP stated “the Project’s impact would be significantly enhanced if an IFMIS is also implemented. Component 1 on Budget Preparation and Component 3 on Budget Execution Controls can deliver better results if implemented together with IFMIS.” In addition, the RP emphasized that the government was ready to add US$ 10 million to implement the Project, as well as emphasizing the history of IFMIS discussions, and the fact that IFMIS was part of the IMF Stand-by Arrangement. From a timing perspective, the RP sought the short extension (3 months) of the Closing Date. 92. An addition to the Bank-executed portion was requested, in order to capitalize on the success of this component, particularly with reference to additional capacity building for the BSA. The ICR finds that only the additional financing for the BSA appears appropriate. This component had clearly delivered and was exceeding expectations, and had high demand and buy-in from the government. 93. Extension of the closing date. This makes sense only for the BSA. The performance up to that time on the other components does not suggest that an additional three months would have made a significant difference. 94. Would cancellation have been a better option? The initial impetus in late 2011 to develop an IFMIS prototype was clearly motivated by the good faith effort of the task team to leverage project impacts to the client while also maximizing full use of the undisbursed Grant funds. Assuming the IFMIS prototype could have been implemented, the Restructuring would likely have contributed to the PDO achievement. Further, successful implementation of the IFMIS prototype would have used most of the remaining Project funds. 95. The likelihood of success of the IFMIS prototype was low by the time the Restructuring went forward in early 2013. The ICR suggests that in retrospect, it would have been more appropriate to move forward with: a) Elements of the BSA component, and something smaller and more achievable in the timeframe, e.g. the SRS, “Plan B” noted in ISR-5; or, b) Elements of the BSA component and simply cancel the remaining uncommitted balance immediately. 25 96. An early restructuring is a clear proactive measure. Cancellation would also have been a clear proactive measure. (c) Justification of Rating for Overall Bank Performance Rating: Unsatisfactory. 97. According to ICR Guidelines, Overall Bank Performance must be rated as “Unsatisfactory if Bank performance was rated Unsatisfactory on both dimensions, OR was rated Unsatisfactory on one dimension and Moderately Unsatisfactory on the other dimension”. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory. 98. This section of the ICR makes a distinction between the day-to-day performance of the Project Management Team, institutional aspects within the MoF, and within Iraq which may have contributed to sub-optimal performance. The ICR notes that the members of the PMT worked diligently throughout implementation, often under very difficult circumstances. Although numerous issues arose, the majority of these are related to the overall operating environment and some design issues (described above). 99. Prior to the project, MoF actively engaged with the international community on the formulation of the Public Finance Management Action Plan. This plan included an ambitious agenda addressing many key issues in the PFM sector. The GoI agreed with the international community (primarily the IMF, World Bank, and USAID) on a division of labor wherein the various elements of the plan would be undertaken by the organizations according to their strengths. The allocation and harmonization of dedicated technical assistance efforts by Donors appeared to seamlessly mitigate many of the risks associated with such a complex reform effort. The international donor community engaged in regular (monthly) PFM Working Group coordination meetings in order to ensure coordination of activities. 100. Diminishing momentum and insufficient understanding regarding the complexity of undertaking such a complex PFM reform. During the implementation of the Project, momentum for the overall PFM reform agenda waned. There are many reasons for this including but not limited to: i) Ongoing political and security issues; and, ii) The substantial downsizing of the international community’s efforts after 2012-which appears to have allowed the focus of reform to fade. This can particularly be seen in the lack of effort to establish a Project Steering Committee, as had been agreed during the MTR. Moreover, it now seems clear that the complexity – and crucially the long-term nature of the reform efforts – was not well-explained by the donors, nor well-understood by GoI authorities. 101. A Challenging Approval Process. The issue(s) with approvals from the Bank have been documented. However, the Ministry of Finance’s approval process can 26 also be characterized by an exceedingly top-down structure, in which it is very difficult for managers of line units to make day-to-day technical decisions without approval from the highest level of the Ministry. This was particularly apparent for the various undertakings of the PMT and is a major factor in project delays (e.g., approval of TORs, contracts, payments to consultants, outputs, etc.). This problem was compounded by an extraordinary level of review of outputs by beneficiary units. In brief, it is the role of beneficiary units to provide comments on consultant deliverables, and then for the PMT to issue approvals. Instead, the PMT was waiting for the units to approve deliverables, rather than taking decisions themselves. This led to substantial delays in payments to consultants, and in some cases, led to various consultants cancelling further work. 102. Coordination with KRG. Project design incorporated benefits for the KRG, in line with the established practice for all projects in Iraq. Besides the benefits themselves, it was intended that the KRG would have a role in coordinating the project activities. An MOU to this end was signed in June 2010, but little coordination ever took place. It is not possible to quantify the impact of this lack of coordination, but it does reflect a failure in achieving overall project objectives. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory (As the Implementing Agency the PMT was subordinated directly to the Ministry of Finance). 103. Part-time staffing of PMT. Throughout implementation, it proved difficult to constitute a PMT of full-time staff members. The part-time staffing of PMT’s operations resonates with this ICR as perhaps one of the most important factors in the slow project implementation progress. The part-time staffing issue was raised in all ISRs, and as late as the Supervision ISR of March 2013 (6 months prior to Project closing). Experience shows clearly that without full-time attention, it is much more difficult for projects to attain planned results, and it is likely that implementation progress would have been improved if the PMT members could have spent more time directly managing the project. 104. The full-time vs. part-time issue also reflects flaws in both the design phase, and the Bank’s supervision effort. In the first instance, the design phase should have taken a more rigorous evaluation of the real possibility that the PMT members would be unable to work full-time. At design and implementation stage, key members of the PMT were fulfilling important roles within the MoF. In spite of government assurances, it now appears unrealistic to have expected that these key MoF staff would be able to take significant time away from their regular duties in order to implement this relatively small project. The MoF did eventually respond to the requests of the Bank and the PMT by hiring one full-time procurement officer. The problems associated with part- time staffing levels may reflect a degree of unwillingness on the part of MoF to change Project arrangements, but it also must be seen as an unsuccessful aspect of Bank supervision. As an alternative, it might have been possible to reallocate some project funds to hire additional specialized staff for the day-to-day operations, while still allowing the PMT leader to have a part-time oversight and approval role. 27 105. Dispersal of PMT staff. While the part-time issue was significant, it must be noted that the PMT’s functionality was also impaired because the staff were dispersed among different locations throughout implementation. The original dispersal was the direct result of multiple bombings of the Ministry of Finance in 2010 and 2011, which necessitated staff being spread amongst numerous buildings. Being in different locations made holding regular meetings difficult:- i) Transportation was not easy; and, ii) Staff were understandably concerned for their safety. 106. On the government side, higher-level leadership in the MoF was either unable or unwilling to modify work arrangements and/or add additional staff to allow a full-time effort. Further, the dedicated Component Technical Teams did not ever fully materialize. These teams were intended to provide technical support to the PMT, especially in writing TORs and evaluating bids, thereby facilitating more efficient work. Instead, the PMT received only ad hoc advisory services. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 6. Lessons Learned 107. Public Financial Management reforms generally fall into two types: i) efforts to improve budget transparency and classification; and, ii) those that improve the scope and comprehensiveness of the budget. Under the first area of reform, an existing body of accepted practice exists to reform economic classification and there are incentives to align accounts along standard international practice to facilitate the production of comparable fiscal and economic data. While there may be some resistance to such reforms stemming from basic bureaucratic inertia, no fundamental interests are challenged or mandates threatened. Once implementation is complete, the reform itself becomes part of the fabric of the system and endures. 108. On the other side of the spectrum are a host of reforms that are neither particularly easy to implement nor which bring the promise of substantial fiscal gains, at least in the short-term. Some public financial reforms to improve the scope and comprehensiveness of the budget are not technically difficult to implement. However, unless significant political capital is invested in overcoming bureaucratic resistance, these reforms are unlikely to move forward. 109. At the strategic level, the Government must have all embracing buy-in to the reform process and understand its implications. At the heart of any PFM reform process is the desire to achieve a transparent, accountable government processes that provide the mechanisms for improved service delivery to the citizenry. If there is inadequate ownership of the reforms, then any ambitious public financial reform program will have only negligible chances for success. 110. Was processing the Project under OP/BP 8.00 - Rapid Responses to Crises and Emergencies (hereinafter ‘emergency procedures’) appropriate? The 28 EPP made the case for use of emergency procedures on the ground(s) that: i) these procedures had been used in all Iraq projects as of the date of this project; ii) emergency procedures had been the default option under the Interim Strategy Note; and iii) the project would address 4 of the key priorities of the OP. The EPP additionally made the claim that “While PFM per se is not a sector typically associated with emergencies, it is critical to sustainable longer-term reconstruction in Iraq. Several studies, by the Bank, the IMF, USAID, DFID, and others, indicate that deficiencies in financial management are leading to poor budget execution and difficulties in accelerating the pace of reconstruction.” Additionally, the EPP was justified on the basis that emergency procedures were needed in order that the project could be completed by the closing date of the Iraq Trust Fund, and that the fund would be quick disbursing. 111. While it is accurate that a well-functioning PFM system could ceteris paribus greatly enhance the sustainability of longer term reconstruction, this should not immediately extend to the notion that the World Bank’s PFM intervention should proceed on an emergency basis. The success of Iraq’s reconstruction efforts rested on a multitude of factors, which required more deliberative thinking, and clarity of vision with respect to execution and proposed outputs. Without robust political economic and project design analysis it would be extremely difficult to implement a complex project of public financial management reforms. 112. Engaging in a more thoughtful PFM project due diligence process, combined with fewer but more targeted PFM reforms, would not have in any manner adversely changed the dynamics of PFM reform in Iraq, and would not have materially affected the sustainability of reconstruction efforts. In addition, project justification constructed on the basis of status quo processes and ISN conformity, appears at ICR stage to be opportunistic versus being based on rigorous determination of the costs and benefits of moving more slowly. 113. The downside of following emergency procedures in fragile conflict environments is the compression (reduction) of normal World Bank internal review processes. Such compression does save time, but it also reduces the ability of the Bank to adequately reflect on the Project, moving much of the review from ex ante to ex post. Although additional clarifications for project implementation provided that project management would incorporate lessons learned and keep objectives and goals limited and the project design was considered to be simple, in reality, the Iraq PFM project was multi-faceted and complex. Use of regular preparation procedures would have allowed a more thorough review process during design, and might have suggested changes to improve the project’s chances of success. 114. Projects that are complex, multi-faceted and that involve process and attitudinal changes from a multitude of government actors require much more scrutiny, analysis, and more informed project design from the outset. There are quite simply too many moving parts in such an ambitious public financial management program. Further, in order to enhance the flexibility of this project the initial project design left a substantial portion of funds unallocated. However, once the various 29 downstream tasks did not materialize there was neither a sufficient back-up plan to use the unallocated funds, or government buy-in to move forward. 115. There are a number of questions and concerns that the World Bank should reflect upon regarding this last ditch IFMIS Iraqi prototype financial management system effort. The initial vision for the introduction of Component 5:- “IFMIS Prototype” was one where the WB would engage in a “modest” risk – “high reward” outcome (if it succeeded). The development of a ‘prototype’ effectively requires an agreement on the functional requirements of the system, and the customization of that software as a prototype. This in effect would limit this component’s outputs to very discrete, measurable factors. Presented in this manner, the decision making process within the World Bank and the project management team itself is readily understood. However, the larger question is whether risks are the same in a fragile conflict environment as they are in a middle income (non-conflict) country. As a number of World Bank reviews indicate (Dorintsky et al: 2003, 2011) there is a very limited chance that IFMIS efforts will succeed, in general, on time and on budget. Dorintsky (2011) in a comprehensive study of IFMIS projects around the world concluded that: 1) only 43% of IFMIS related projects are delivered as specified; 2) only 50% of IFMIS projects are delivered on budget; and 3) only 21% of IFMIS projects are delivered on time. Although the proposed Iraq PFM IFMIS project is only a prototype, the likelihood of developing it on time, on budget, as per project specifications in the time frame allotted prior to the project closing date appear low, even under the best of circumstances. 116. The rationale presented for establishing this IFMIS prototype was to ‘realize the project objectives and magnify the impact of other components.’ It is not clear how this could possibly be achieved, considering that so many of the Public Financial Management performance measures in (primarily) Component 1, and to a lesser extent in Components 2 and 3, were largely unfulfilled and rated as moderately unsatisfactory. Further, so many of the fundamental elements that are required to be in place for a project of this magnitude - especially and singularly importantly ownership by the Government of Iraq - appear not to have been firmly in place. 117. The critical success factors for any IFMIS project (large or small) are quite clear: a full-time project coordinator is required; there needs to be strong political will and a champion at the political level; and the project needs to provide an abundance of training. Many government entities will reach out for an Integrated Financial Management System solution believing that having the system infrastructure alone will create the mechanisms for an appropriate public financial management environment. However, an IFMIS is only a tool and needs to be supported by appropriate reforms on a multitude of levels. Many of the basic reforms that would support a restructuring of the project to support this prototype development (and, subsequent roll- out if successful) are simply not in place in Iraq. 118. The independent evaluation report on WB’s BSA intervention provided keen insight on lessons learned and reasons for success stipulating that “the structure and processes that the World Bank put in place for planning and delivering the 30 project had been instrumental in ensuring that the project successfully met its objectives”. Three key factors contributed to the success of the program, including: (1) the strong, consistent strategic direction by the program manager ensuring that all project activities were targeted on meeting the overall aims and objectives that had been agreed for it, in addition to ensuring close, strong working relationship with the BSA; (2) the international consultant(s) with broad experience and strong links with the international links of SAIs being able to identify the institutions best placed to provide the type of professional audit training, advice and guidance that the BSA needed; and, (3) the local consultant who is in frequent touch with the BSA, facilitating the communication, gaining the trust of the BSA, and reasonably compensating for the limited access of international experts and Bank staff to Baghdad and the BSA. 119. A key overarching lesson (or question) that the WB can take from this project, and the ICR review, is whether the Bank has sufficiently flexible procedures to adapt project design when the initial project assumptions (para 12) are too enthusiastic, and/or the project design proves to be too complex? The Bank and project management teams did not foresee –that in retrospect- the reform environment would dramatically (and, quickly) deteriorate. Perhaps, the first lesson would be to always go into fragile conflict environments with a simpler, more targeted, approach. The review also points out the crucial importance for there to be clear and measurable indicators, and an agreed monitoring and evaluation plan prior to Project start. At present, task teams cannot change Results Frameworks (and hence indicators) without formally restructuring a Project. It is worth evaluating whether Bank procedures can be improved such that teams can modify especially Results Frameworks more easily. 120. Further, broad public financial management reforms as represented by this project do not easily dovetail with emergency lending procedures. As presented above, use of regular procedures would have allowed a more constructive internal dialogue and improved project design. In particular, it now seems unlikely that such a large amount of money would have been allowed to remain unallocated, and later threaten the project and lead to demand-side response. Ultimately, the Bank needs more adaptable and flexible procedures for countries that fall under the fragile conflict environment umbrella. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies 121. Discussions with the Project Management Unit and Government counterparts indicated that there were several issues that adversely impacted the evolutionary flow of the Project, including: i) Weaknesses by the various beneficiary entities in contract management; and, ii) Poor terms of reference development. As a result, the PMT believes that many of the consultancies were poorly designed, implemented and executed. The PMT ICR review agreed that that the change in the PMT leader, procurement manager, and accounting manager weakened the effectiveness of PMT project delivery. In effect, the reliance of an external funded procurement manager, for a period of time, lead to delays in procurement and payment release. Recommendations from the PMT include the following: 1) the schedule of more 31 frequent consultation meetings with the World Bank (no less than 3 annually); 2) additional assistance in the selection of eligible firms when it comes to the selection of consultancies in international procurements; and, 3) more assistance from the World Bank with respect to understanding and improving contract negotiation skills. (b) Cofinanciers/Donors USAID has been supporting Improving Iraqi Procurement Practices, Improving Program Management, and assisting the Iraqi Government in streamline its onerous regulations to a more business friendly regulatory environment. (c) Other partners and stakeholders 122. Not Applicable 32 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) 12 Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) A) Recipient-Executed TF Component 1. Strengthening Budget Formulation and 3.95 0.84 21.3 Implementation Component 2. Strengthening 3.2 1.38 43.1 Public Sector Procurement Component 3. Strengthening Budget Execution and 2.33 3.41 148.0 Implementation Component 4. Capacity Development and Project 3.00 2.49 83 Management Project Management and 0.70 0.39 56 Auditing Sub-total Allocated 13.18 Unallocated 2.82 B) IFMIS – New Component [6.00] 13 1.27 21.1 added at Restructuring Subtotal Recipient Executed 16.00 9.78 62.5% C) Bank-Executed Companion 2.00 2.538 127% TF Total Project Costs 18.00 12 Final component expenditures to be confirmed by PMT prior to ICR finalization 13 Reallocated at Project Restructuring from original financing – not Additional Financing. 33 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Iraq Reconstruction Trust Fund – 16.00 9.785 62.5% Recipient-Executed Iraq Reconstruction Trust Fund – Bank-Executed a) Original 2.00 2.53 127% b) Restructuring 2.62 2.53 97% 34 Annex 2. Outputs by Component Component Proposed Contract Donor Support Objective Project/ Component Achievement Comment/Other 1 Component Contract Outcomes (as issues Activities Output` Per PAD) Component • Developed PFM.C.1QCBS.O1- U.S. - Build credible 1. New Budget Consultant Cancelled the 1: Sector a) Strengthening Treasury/USAID annual budgeting Circular contract preparation part of Strengthenin Strategies for budget preparation Medium Term framed by Designed terminated (due sector strategy for g budget two key by i) capacity Fiscal Framework/ medium-term to poor MOWR and MOE formulation ministries: a) strengthening in Budget perspective within 2. Draft Sector performance?, because ministry and Education; b) economic Manual/Budget public investment Strategy for not meeting had contracted with Implementat Ministry of forecasting; ii) Classification planning process one/two key client another donor. ion Water improve budget Sectors expectations Resources preparation process; DFID etc.?) • Strengthen iii) Improve quality MTFF/Budget 3. Draft project and adherence to Strategy (Budget Medium preparation budget call circular; Circular) Budget Term and appraisal iv) consultant e-Gen Priorities Forecasting • Strengthen the Consultants Develop detailed Framework capacity of sector strategies Developed the Economic for two pilot Department of ministries: MOF Ministry of Water • Improve the PFM.C.1QCBS.02- Resource and Amendments – to Process of Strengthen Budget Ministry of extend the annual budget preparation process Education; Consultant implementation formulation by 1) Developing Strengthen contract period from 12 and budget Sector Strategies; 2) Investment (terminated) due months to 24 classification Improving Project to poor months? • Improve methodology of Preparation and performance? content of and Investment Project appraisal and adherence to Preparation, Implementation budget call Evaluation and Guidelines; and circular Capacity Building Develop training curriculum 35 Component Proposed Contract Donor Support Objective Project / Outcomes Comment/Other issues 2 Component Contract Output (as Per Original Activities PAD) Component • Strengthen the PFM.C.II.QCBS.01: Procurement • Develop a • Strengthen Amendments: 2: proposed Developing assistance and standard the • Change commencement Strengthenin procurement Standard Bidding capacity building request for Proposed date to January 2013 g Public institutional Documents (SBD), with beneficiary proposal institutional • Extend Implementation Sector system Master ministries. (RFP); system period to 21 months (from Procurement • Prepare Implementation USAID/Tarabot • Develop • SBD 18) (Improvemen Standard Manual, and project standard created for • Cancel Phase VI: ts to the Bidding Training. Consultant training/support on bidding key sectors Description of Service systems of documents for International Procurement documents • Master • Cancel last payment Euro Public key sectors Training Center for • Develop SBD Implementat 39,600 Procurement) • Master International Labour For MOH (for ion manual • Cancel Appendix B Implementatio Organization (ILO) procurement developed Reporting requirements n manual of • Procuremen • Extend Expiration date to developed pharmaceutica t bulletin August 31, 2013 instead • Procurement l products) board of August 20, 2013 bulletin board • Develop SBD developed • Removed/Cancelled developed for Ministry • Build planned TA to • Build the of Education capacity of Regulator authority RA capacity of the • Develop SBD the public on complaints Public for Ministry procurement mechanism and Procurement of Electricity regulatory blacklisting; Regulatory • Develop Authority • Removed study tour of Authority and prequalificati and key RA key on and implementin Implementing evaluation g agencies Amendments: agencies, etc documents • Modified terms of • Develop reference so that training consultant use curriculum “Controlled Sample” (including rather than methodology “Representative Sample” for SBDs) • Included five (5) 36 Component Proposed Contract Donor Support Objective Project / Outcomes Comment/Other issues 2 Component Contract Output (as Per Original Activities PAD) • Develop Additional Trainees national implementatio n manual Amendments: Contract • Develop cancelled. Reason provided: training insufficient time/funds? curriculum • Public PFM.C.II.QCBS.02 procurement Amendments: Contract Consulting services Training Cancelled for Procurement Strategy Training needs Arab • Develop Administrative Training Development curriculum Authority and complete (ARADO) pilot local procurement training PFM.C.IIlQCB.O3 Design and create web-based portal for publication of bidding opportunities and documents. PFM.C.II.IC.O1 TA for RA for complaints and black listing 37 Component Proposed Contract Donor Support Objective Project / Contract Outcomes Comment/Other 3 Component Output (as Per Original issues Activities PAD) Component • Introduce PFM.C.III.QCBS.0 • USAID/Treas • Define and agree • Rules and Removed/Cancelled/ 3: improved cash 1: Strengthening ury/State: on conceptual procedures for Postponed: Assessing Strengthenin release system treasury operations, Commitment framework for cash release multiple treasury g Budget • Assess financial control, Control, Cash commitment system system (because MOF Execution multiple accounting and Release control developed and planned to establish and treasury reporting: System, • Provide detailed agreed; TSA) and Implement Implementat system in Telecom/Telematiq Reducing rules / regulations • Diagnostic IFMIS; MOF has ion KRG ue Inc. (Prime Outstanding for conceptual Study on previously established • Introduce new contractor); Balances; framework multiple internet network (so, commitment International Accounting • Capacity treasury model this component no control system Business and Framework, building/Training completed longer required) • Improve Technical Monthly and • Pilot the system in • New rules and financial Consultants (IBTC) Annual two ministries regulations for Amendments: Terms reporting (sub); Iraqi Reporting, (education and commitment and conditions of • Assess the Scientific Company IFMIS, MOF water resources) control system payments amended to internal for Information Website developed and reallocate the advance financial Technology (al • UN: agreed payment amount, control system Wafeer Group for Subnational • Assessment of according to option on • Upgrade Technology and Budget roles/mandates January 2012; Extended content and Education). Execution, of Inspector the implementation functionality KRG, Anbar, Generals period to 26 months of MOF Basra completed instead of 24 months. website • DFID: IT Allows for sufficient time for completion of • Create internal Accounting Requirements framework and for MOF remaining tasks. MOF network Capacity reporting Internal development • Others: Network of line Budget Defined/Techn ministries to Execution ical enhance their assistance with Specifications ability to beneficiary developed implement ministries and investment local Amendments: None government 38 Component Proposed Contract Donor Support Objective Project / Contract Outcomes Comment/Other 3 Component Output (as Per Original issues Activities PAD) projects PFM.C.- • Description of Amendment: Contract III.QCBS.02: Standards, amended as follows: Assessment of regulations, and 1) Two Proposed staff Internal Financial procedures changes (project Controls, Including followed by manager, develop roles of Internal internal auditors, 3) Auditors and • Description of any 2) Added new task to Inspector Generals specific consultant which is (IG) Ernest and weaknesses in the supplying Young Consultant internal audit installation of key function and hardware to Create overlap areas, and Maintain MOF • Recommendations Website to resolve 3) Add new term to weaknesses in the the payments internal financial conditions to allow control function partial payments (including accountants, internal auditors and IGs) • A report charting the way forward for modern internal financial control function. Including a roadmap and appropriate organization structure. 39 Component Proposed Contract Donor Support Objective Project / Contract Outcomes Comment/Other 3 Component Output (as Per Original issues Activities PAD) PFM.C.III.QCBS.0 • Support bi-lingual 4: Create and contents; SEE maintain Ministry of FULL Finance Website CONTRACT LIST ASK Group of Companies with • Design/implement strategic Partner – MOF ABACUS consulting Logo/electronic Technology PFT format PFM.TR.III.CQS.O Prepare well- • Two components Contract Cancelled 1: Training trained staff (intranet and Services for IT to manage Public) Management and and Operational implement • Complete training Training: New projects of 50 trainees for Horizons Computer IT staff in MOF Learning Services and Information Technology Directorate PFM.TR.III.CQS.3 Contract Cancelled Enhance treasury operations in the provinces PFM.C.III.IND.O1 Contract Cancelled 40 Component Proposed Contract Donor Support Objective Project / Contract Outcomes as Comments/Other 4 Component Output per Original Issues Activities PAD Component • Ensure PFM.C.IV.QCBS.O USAID/Treasury; Contract Outputs: • Assessment of Adding IFMIS as the 4: Capacity coordination 1: Capacity Curriculum Design Develop/implement a training needs Fifth (5) Component Development of training and development of 12 month training completed of the project using and Project capacity MOF Training program of events; • Assessment of uncommitted funds Management building Institute (Coffey Other: Training Develop five year the capacity of through restructuring Team provided in International for MOF and training program the existing of GOI request letter the first three Development subnational level including suggestions Training dated October 30, components Middle East Pty officials including for curricula, training Institute in 2011. • Strengthening Ltd) through Finance programs and material. Baghdad of MoFs Institute Completed Amendments: internal ‘in- • Time-bound Contract extended to 26 house’ training comprehensive months and 20 days Finance training plan instead of 24 months international with strategies (due to suspension of and local PFM.C.IV.QCBS.O Prepare and and modes for implementation of the advisory 1: Training services train staff as Prepare well trained training contract?) services for Evaluation of to the staff on modern internal defined internal control requirements control procedure • Curriculum systems (Price of internal practices. Have prepared an d Waterhouse control/comm understanding of initiated Coopers) itment control internal controls Prepare and system. PFM.C.IV.QCBS.O train staff as 1: Training services to the for requirements Develop understanding Communications of modern and skills in the field of and Petroleum procedures in modern auditing Industries for auditing taxation practices. General taxation Commission of (Petroleum Taxes (Price and Waterhouse communicati Coopers) ons industry) 41 Component Proposed Contract Donor Support Objective Project / Contract Outcomes Comment/Other 5 Component Output (as Per Original issues Activities PAD) Component • IFMIS • USAID / • Provide capability • Prototype for • Project closing date 5: Component BearingPoint to automate the IFMIS extended to August Integrated Added previously budget preparation developed 2013 Financial (USD$6,000,0 supported this (support • Prototype for • Task of this Management 00) project component 1). IFMIS tested component was to Information • Prototype • 11 technical • Increase the ability build on a system System Development orders given to of MOF to that was (b) and Testing Freebalance undertake central implemented under Reallocation for the Supply for maintaining control and the US Department of Grant and the ongoing monitoring of of treasury Proceeds; Implementatio operation of expenditure (former) financing. and, c) n of IFMIS to the system receipts in • MOF would like to Extend the be conducted government use Free balance closing date in the January Ministries, accountability of recipients 2013 – August departments and Suite V7.0 31, 2013 spending units. • SRS for period. • Provide MOF with implementing • Pilot access to IFMIS would also implementatio comprehensive and be conducted as an n and meaningful independent government information on assignment wide rollout budgetary, • Cost of developing based on financial, and US$4.9 million phase I results operational • ICT Hardware • Commencing performance during US700K plus January 1, the financial year. disaster recovery 2014 – site US$300 December 31, 2014 and Government wide roll-out to be completed by June 30, 2016 42 Component – Proposed component Contracts Donor Output Outcome Achievement Indicators BSA objectives and activities Support I-Iraq PFM project – Bank-executed grant (Allocated : USD 1,000,000, additional amounts granted when phases completed – Disbursed: USD1,300,000 + approx. USD 400,000 in kind expenses by BSA) Component 1: • Managing & coordinating World Bank Program designed, Well-designed Component and Design of program. Concept program FM staff implemented, and evaluated contracts fully Identification of qualified planning, • Assisting in planning & evaluated, and program completed consultants. coordination designing program, International completed addressing Signature of contracts. monitoring and communication with STC contract priority assessed Execution of contracts as per evaluation SAIs/experts, and periodic Qualified capacity and agreed TORs. reporting consultants training needs Reports submitted. • Assisting in data collection Local STC identified, & communication with contract contracts timely BSA signed, and • Strengthening the BSA effectively M&E capacity completed in • Independent evaluation of M&E STCs accordance with the program contracts agreed TORs • Any other related activity that is not covered the technical components Independent Evaluator contract Component 2: Assess the BSA’s present Assessment made Interim Component and Agreement on the Assessment of capacity and training needs in See above and assessment outcome: contracts fully conclusions of the the present order to establish a train the report completed Definition of the completed assessment report (including capacities and trainers’ program for training recommendations for action) training needs strengthening its capacity activities needed as the recognition by the for developing BSA of the gaps to fill, the Assessment process the professional improvements to achieve and according to best practices by capacity of the the processes to launch for IDI BSA achieving these goals. including: 1. Self-assessment by BSA Final outcome: Decisions taken by the BSA 43 Component – Proposed component Contracts Donor Output Outcome Achievement Indicators BSA objectives and activities Support staff supported by Local Assess capacity in order to: consultant at the end of the 2. Select the best master 2. Taking stock of BSA staff’s project with a trainers answers to questionnaires, view to evaluate 3. Disseminate the lessons 3. Performing an assessment capacity of the training events workshop, development 4. Initiate the change Completing the assessment next steps management process, during the following phase of that shall accompany the the project (training events) capacity development in the framework of the strategic plan 3.1 Financial audit United Pilot audits Interim Completed No. of master trainers trained A training cycle of three Kingdom (UK performed outcome: cycles, delivered during the workshops: 15 Component 3: workshops, completed by NAO) contract according to the Better workshops, Training the remote training between the Performan new chapters of professional trained master No. of audit team members trainers’ workshops, ce audit the manual knowledge trainers, trained during the program performed with the NAO of the manual, disseminated executed pilot workshops: 380 United Kingdom training, Dissemination among the BSA audits, and fully Netherlands and peer within the BSA of staff executed No. of auditors trained by the 3.2 Performance audit Court of Audit review by the lessons of the contract master trainers: 83 A cycle of two workshops and a (NCA) contract NCA workshops cycles Medium term study tour in Den Haag (the financed outcome: In addition to No. of pilot audits performed Netherlands) performed with by Improve quality what was by BSA: 9 the Netherlands Court of Audit Netherlan of audit originally ILO contract ds planned: No. of workshops: 28 3.3 Procurement control and Ministry Collaboration -selected audit audit French Court of of Affairs between the teams were also No. of specialized peer One workshop with the ILO Accounts various parties trained with the review: 1 procurement audit, training Center and one peer experts involved in the master trainers. 1 performance audit review with auditors from contracts public - audit manuals French Court of Accounts governance being revised by In the medium-term and BSA based on when a full field survey could 1.4 Oil audit adopted be conducted: 44 Component – Proposed component Contracts Donor Output Outcome Achievement Indicators BSA objectives and activities Support Two workshops, 1 with oil Private experts methodology - % of auditees who are industry internal control and contracts satisfied with the audit experts, 2 with auditors of improvements in the quality the SAI of Brazil of the audit -% of total recommendations 3.5 Forensic audit Iraq FMA implemented by the auditees Workshop held by the Bank INT contract -For forensic audit: no of and a financial prosecutor of the cases submitted to the French Court of Accounts with Commission of Integrity and the participation of Judiciary representatives from CoMSEC, Polish Supreme the State Council, the Judiciary, Audit Office Commission of Integrity and the (NIK) Inspector General function. Forensic audit study visit to France Bank’s projects audit Workshop with the fiduciary monitoring agent of the Bank in Iraq Training session on site with the Wroclaw office of the Supreme Audit Office of Poland, in charge of auditing one Bank’s project 3.7 Develop Capacity in change management process Higher management workshop Study tours in Warsaw and Den Haag Workshop on strategic planning, monitoring and evaluation practice and change management 45 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Not Applicable 46 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending/Grant Preparation Monali Chowdhurie-Aziz Sr. Public Sector Specialist MNSPR Task Team Leader\ Nazaneen Ismail Ali Senior Procurement Specialist MNAPC David Biggs Consultant PRMPS David A. Bontempo Operations Analyst MNSPS Ruxandra Burdescu Resident Representative LCCUY Mona El-Chami Sr Financial Management Specialist MNAFM Mira Merhi Consultant PRMPS Muna Abeid Salim Senior Program Assistant MNSPR Supervision/ICR Yahia Khairi Said Sr. Public Sector Specialist MNSPR Task Team Leader Nazaneen Ismail Ali Senior Procurement Specialist MNAPC Sepehr Fotovat Senior Procurement Specialist MNAPC David A. Bontempo Operations Analyst MNSPS Mona El-Chami Sr Financial Management Specialist MNAFM (b) Staff Time and Cost Staff Time and Cost (Bank Budget Trust Fund Only) Stage of Project USD Thousands USD Thousands Cycle (including travel (including travel No. of staff weeks and consultant and consultant costs) costs) Lending FY08 and FY09 30 265,989 194,153 Total: 30 265,989 194,153 Supervision/ICR FY10 201,085 FY11 188,300 FY12 341,905 FY13 1.4 7,153 97,188 FY14 0.55 2,766 77,724 Total: 1.95 9,919 906,202 47 Annex 5. Beneficiary Survey Results Not Applicable 48 Annex 6. Stakeholder Workshop Report and Results (if any) Not applicable 49 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR 1. Discussions with the Project Management Unit and Government counterparts indicated that there were several issues that adversely impacted the evolutionary flow of the Project, including: i) Weak comprehension by the various beneficiary entities on contract management; and, ii) Poor terms of reference development. As a result, the PMT believes that many of the consultancies were poorly designed, implemented and executed. 2. The discussion also pointed to some weaknesses in the PMT. The PMT – ICT report agreed that that the change in the PMT leader, procurement manager, and accounting manager weakened the effectives of PMT project delivery. In effect, the reliance of an external funded procurement manager, for a period of time, led to delays in procurement and payment release. 3. Recommendations from the PMT include: 1) the schedule of more frequent consultation meetings with the World Bank (no less than 3 annually); 2) additional assistance in the selection of eligible firms when it comes to the selection of consultancies in international procurements; and, 3) more assistance from the World Bank with respect to understanding and improving contract negotiation skills. 50 Annex 8: Iraq PFM Action Plan Republic of Iraq:- Public Finance Management Project “Iraq Public Finance Management Action Plan » No. Issue 2009 2010 2011 2012 Type of assistance Capital Budgeting Budget Preparation 1 Review of budget strategy and exposure to international X International experts will review Iraqi practice practice and provide advice. Iraqi officials will be exposed to international norms by means of workshops and/or a study tour. 2 Exposure to international practice on development of MTFF X Iraqi officials will be exposed to international norms by means of workshops and/or a study tour. International experts will advise Iraqi budget officials on a methodology and process tailored to Iraqi circumstances. Relevant training courses will be introduced as part of a comprehensive capacity development strategy covering all aspects of budget reform. 3 Budget ceilings for line ministries X Advice will be provided on the process, content and communication strategy for the introduction of budget ceilings for current and capital expenditure. Training of budget officials will take place in central ministries, line ministries, governorates and districts. 4 Review of Budget Call Circular in the light of international X International experts will advise the GoI on best practice and suggest improvements improving the content of the BCC to bring it more in line with modern international practice. 5 Introduce Budget Manual (including guidance and X International experts will review all existing instructions) budget regulations and procedures and facilitate the development of a comprehensive Budget Users Manual, allowing for the incorporation of revised procedures over time. 6 Pilot medium term financial planning in MOE and Ministry of X International advisers will work in the Municipalities and Public Works Ministries of Finance, Planning, Education and Municipalities to guide the process of 51 No. Issue 2009 2010 2011 2012 Type of assistance introducing medium term financial planning on a pilot basis. Workshops will be used to develop awareness, knowledge and key skills 7 Move to implement functional classification X Advice will be provided by international experts on the development of a functional budget classification in parallel to functionality developments in the IFMIS Budget Execution, Cash Management and Reporting 8 Effective monetary plan defining actual department needs X Accounting Department will inform the according to schedules prepared by the ministries and concerned bodies in order to prepare and departments. follow up with the schedules to provide the cash needed . 9 Make use of provincial treasuries as they represent MOF, and X This requires habilitating treasury buildings by include multiple treasuries within the same province, if providing them with furniture and work necessary, according to the number of affiliated departments requirements such as computers; and as applied in many countries and in Kurdistan region (e.g. developing the employees in this respect in Erbil, Dohuk: 5 treasuries). The aim is to facilitate works, ease order to have efficient staff . burden over MOF, and control cash in hand by minimizing number of units financed by MOF . 10 Introduce commitment control system. X 11 Consider amending Financial Administration Law. X A committee has been formed at Prime Ministry to follow up with this issue . 12 Issue unified accounting instructions to apply the accounting X A committee formed for this purpose . system and provide accounting treatments and records of accounting documentation . 13 Address and follow up with the issue of advances and X This requires forming a committee tasked with outstandings appearing on Accounting Department records preparing the schedules of outstandings and and concerned departments . working on processing them either by settlement or writing-off. 14 Use internet system to post all data and laws issued by MOF, X MOF website has been created. as well as all MOF activities . 15 Capacity building: design specialized courses to improve X Prepare advanced and intensive training activities of Budget Department, Accounting Department, as programs in the fields of budget and well as other supporting departments . accounting; and improve managers and employees . 52 No. Issue 2009 2010 2011 2012 Type of assistance 16 Create an internal network at MOF, linking Budget, X The network aims to develop MOF activities accounting, communication, and public debt functions in and support it in terms of providing its order to facilitate and exchange information . requirements such as computers from WB . 17 Improve and expand application of IFMIS. X Hold for time being. The work will resume according the implementing contractor, and a committee has been formed in this respect . Public Procurement 18 Essential ratification of the Public Procurement Law to X follow the standard protocol for presenting new legislation initially to the Council of Ministers and Shorra Council at the Ministry of Justice, and subject to the Council’s vetting of the new law, to the Council of Representatives for alignment with other national laws 19 a decision on enactment of the implementing regulations X This requires a conference to finalize the would fill in the details for the procedures authorized by the implementing regulation in both Arabic enacted Law of Public Contracts; The regulations would be and English version to include the inter- subject only to the approval of the Council of Ministers ministerial committee members and the Bank’s Procurement Team 20 a decision on enactment the standard bidding documents to X The Bank would need the Bank sending be issued to all procuring entities the final version of SBD’s to the government of Iraq and then the government of Iraq takes the necessary action to enact these SBD. 53 Annex 9. Comments of Cofinanciers and Other Partners/Stakeholders 1. A number of donors have been providing support to PFM reform and capacity development over the last four-to-five years. These include International Monetary Fund (IMF), the United Nations (UN), the United States (USAID), the United Kingdom (DFID), and the European Commission (EU). The US Government has worked on a range of PFM issues through the U.S. Treasury, U.S. Agency for International Development (USAID) and State Department, while the UK Government has been actively involved in economic management through its Department for International Development (DFID). The UNDP has managed a program of support to the Board of Supreme Audit as well as providing support to anti-corruption initiatives. The IMF has been providing technical assistance in support of its Stand-by-Facility. 54 Annex 10. List of Supporting Documents 1. Republic of Iraq (Ministry of Finance) (2013) Project Implementation Completion Reports (ICR) of the Emergency Public Finance Management Project (EPFMP) World Bank Grant no. TF94552 2. Derner, Cern, Joanna Watkins, and Willian Dorinstksy (2011) Financial Management Information Systems – 25 Years of World Bank Experience About What Works and Doesn’t Work. World Bank: Washington, D.C. 3. Etiman Management Consultancy Services, Inc. (November 2013) ITF Project: Financial Quarterly Report July 01- October 31, 2013 (Final Report) Emergency Public Finance Management Reform Project (EPFMRP). Ministry of Finance (Mo) of the Republic of Iraq 4. Etiman Management Consultancy Services, Inc. (July 2013) ITF Project: Procurement Quarterly Progress Report April 1st – June 30th, 2013 Emergency Public Finance Management Reform Project (EPFMRP). Ministry of Finance (Mo) of the Republic of Iraq 5. Grogan, Frank (August 2013) Report on the Evaluation of the World Bank Project to Support the Capacity Development of the Board of Supreme Audit of Iraq, World Bank: Iraq - Public Finance Management Project- Iraq Board of Supreme Audit Component 6. World Bank (2013) Restructuring Paper on a Proposed Project Restructuring of Iraq: Public Finance Management Reform Project (P110862) (Grant TF094552) Report No. 474325-HQ 7. World Bank (January 2013) Aide Memoire January 24, 2013 8. World Bank (October 2013) Project Summary Sheet 9. World Bank (2013, 2011, 2012, 2010) Implementation Status and Results (Seq. No. 1, 2, 3,4,5) 55 10. World Bank (2009) Emergency Project for a Proposed Grant in the Amount of US$18 Million from the World Bank Iraq Trust Fund to the Republic of Iraq for the Public Finance Management Project, Report No. 46861-HQ 11. World Bank (2008) Iraq Public Expenditure and Financial Accountability (PEFA) Assessment Public Financial Management Report Washington DC 56