Document of The World Bank Report No: ICR3057 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-4072 GUI) ON A CREDIT IN THE AMOUNT OF SDR 16.6 MILLION (US$27.3 MILLION EQUIVALENT) TO THE REPUBLIC OF GUINEA FOR A HEALTH SECTOR SUPPORT PROJECT JUNE 27, 2014 Human Development Sector West and Central Africa Health, Nutrition and Population (AFTHW) Country Department AFCW3 Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective 31/12/13) Currency Unit = Guinean Franc (GNF) September 2006: US$1.00 = GNF 5463 June 2014: US$1.00 = GNF 6830 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immuno-Deficiency Syndrome ANC Antenatal care APL Adjustable Program Loan ARV Anti-Retroviral Therapy CAS Country Assistance Strategy CBO Community-Based Organization CHC Comprehensive Health Center CPN Ante Natal Consultation CPS Country Partnership Strategy CRD Rural Development Community CS Health Center (Centre de Santé) C-Sections Caesarian Sections DAAF Directorate of Administration and Finance (Direction des Affaires Administratives et Financières) DALY Disability Life Years DCA Development Credit Agreement DHS Demographic and Health Survey (Enquête Démographique et de Santé) DNPH Directorate of Public Hygiène (Direction Nationale de l’Hygiène Publique) DO Development Objective DPS Prefectoral-level Health Department (Direction Préfectorale de la Santé) DPTC3+M Diphtheria, Pertussis, Tetanus third dose, plus Measles DRS Regional-level Health Department (Direction Régionale de la Santé) EA Environmental Assessment EPI Extended Program of Vaccination ESW Economic and Sector Work FM Financial Management FMA Fiduciary Management Agency (Agence de gestion financière) FMR Financial Management Report (Rapport de suivi financier) GDP Gross Domestic Product GNF Guinean Franc GTZ (GIZ) Gesellschaft für Technische Zusammenarbeit (German Agency for Technical Cooperation) HD District Hospital (Hôpital de district) HIPC Heavily Indebted Poor Countries HIV Human Immunodeficiency Virus HMIS Health Management Information System (Système national de gestion de l’information sanitaire) ICR Implementation Completion Report IDA International Development Association IMCI Integrated Management of Childhood Illness IP Implementation Progress ISR Implementation Status Report ITP Intermittent Prevention Treatment for malaria KPI Key Performance Indicators LLIN Long-acting insecticide treated bed nets MAP Multi-sectoral AIDS Program MCH Mother and Child Health MDG Millennium Development Goals M&E Monitoring and Evaluation MIS Management Information System (Gestion d’information sanitaire) MOF Ministry of Finance (Ministère des Finances) MOH Ministry of Health (Ministère de la Santé et de l’Hygiène Publique) MICS Multiple Clusters Indicator Survey MTCT Mother to Child Transmission (of HIV) MTR Mid-Term Review MURIGA Community risk funds (Mutuelles des Risques liés à la Grossesse et à l’Accouchement) MWM Medical Waste Management NGO Non-Governmental Organization OOP Out-of-Pocket expenditures PACV Community Support Program (Bank-financed project) PAD Project Appraisal Document PAO Annual Operating Plan (Plan Annuel d’Opérations) PCG Central Pharmacy (Pharmacie Centrale de Guinée) PCN Project Concept Note PCU Project Coordinating Unit PDO Project Development Objective PER Public Expenditure Review PETS Public Expenditure Tracking Survey PHRD Policy and Human Resources Development Fund PMTCT Prevention of mother to child transmission PNDS National Health Development Plan (Plan National de Développement Sanitaire) PNLS National Program to Combat HIV/AIDS PPSG Population and Reproductive Health Project (Projet Population et Santé Génésique) PPF Project Preparation Facility PRCI Capacity Building for Service Delivery Project (Bank-financed project) PRSC Poverty Reduction Strategy Credit PRSP Poverty Reduction Strategy Paper PUIII Urban Project QAG Quality Assurance Group QER Quality Enhancement Review QSA Quality of Supervision Assessment RBF Results-Based Financing RF Results Framework SDR Special Drawing Rights SIL Specific Investment Loan SOE Statement of Expenditures SOUB Basic emergency obstetrical care (Soins Obstétriques de Base) SOUC Complementary emergency obstetrical (Soins Obstétriques d’Urgence Complémentaires) STI Sexually transmitted infections SWAP Sector-Wide Approach TA Technical Assistance TBA Traditional Birth Attendant TPI Intermittent Preventive Treatment for Malaria TT/TTL Task Team/Task Team Leader UN United Nations UNFPA United Nations Fund for Population Activities UNICEF United Nations Children's Fund USD US Dollar WHO World Health Organization REPUBLIC OF GUINEA HEALTH SECTOR SUPPORT PROJECT Table of Contents A. Basic Information i B. Key Dates i C. Ratings Summary i D. Sector and Theme Codes ii E. Bank Staff ii F. Results Framework Analysis ii G. Ratings of Project Performance in ISRs viii H. Restructuring viii I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 10 1.1 Context at Appraisal 10 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 11 1.5 Original Components 13 1.6 Revised Components 14 1.7 Other significant changes 14 2. Key Factors Affecting Implementation and Outcomes 15 2.1 Project Preparation, Design and Quality at Entry 15 2.2 Implementation 17 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 19 2.4 Safeguard and Fiduciary Compliance 21 2.5 Post-completion Operation/Next Phase 23 3.1 Relevance of Objectives, Design and Implementation 23 3.2 Achievement of Project Development Objectives 24 3.3 Efficiency 30 3.4 Justification of Overall Outcome Rating 32 3.5 Overarching Themes, Other Outcomes and Impacts 32 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 33 4. Assessment of Risk to Development Outcome 34 5. Assessment of Bank and Borrower Performance 34 5.1 Bank Performance 34 5.2 Borrower Performance 36 6. Lessons Learned 37 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 38 Annex 1: Project Costs and Financing 39 Annex 3: Economic and Financial Analysis 46 Annex 4: Bank Lending and Implementation Support/Supervision Processes 48 Annex 5: Beneficiary Survey Results 50 Annex 6: Stakeholder Workshop Report and Results 51 Annex 7: Summary of Borrower's ICR and/or Comments on Draft ICR 52 Annex 8: Comments of Co-financing partners and Other Partners/Stakeholders 55 Annex 9: List of Supporting Documents 56 MAP A. Basic Information Country: Guinea Project Name: Health Sector Support Project Project ID: P065126 L/C/TF Number: IDA 4072 GUI ICR Date: June 30, 2014 ICR Type: Core ICR Lending SIL Borrower: Government of Guinea Instrument: Original Total US$ 25.0 Million Disbursed Amount: US$ 23.46 Million Commitment: Environmental Category: B Implementing Agency: Ministry of Health and Public Hygiene Co-financiers and Other External Partners: N/A B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 31 March 2003 Effectiveness: 02 September 2005 26 June 2006 Oct. 4, 2011 (Lev 2) Appraisal: 11 June 2004 Restructuring: Jan 4, 2012 (Lev 1) May 21, 2013 (Lev 2) Approval: 02 June 2005 Mid-term Review: March 2008 Cancelled Closing: 30 September 2011 31 December 2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Moderately Quality at Entry: Government: Unsatisfactory Unsatisfactory Implementing Moderately Quality of Supervision: Satisfactory Agency/Agencies: Satisfactory Overall Bank Moderately Overall Borrower Moderately Performance: Satisfactory Performance: Satisfactory i C.3 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating Potential Problem Project at any time Yes No QER Quality at Entry (QEA): (Yes/No): Problem Project at any time (Yes/No): Yes Quality of Supervision No QSA Assessment (QSA): DO rating before Closing/Inactive status: MS D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 30% Health 100% 70% Theme Code (as % of total Bank financing) Health system performance 100% 34% Child health 33% Population and reproductive health 33% E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Gobind T. Nankani Country Director: Ousmane Diagana Mamadou Dia Sector Manager: Trina Haque Alexandre Abrantes Project Team Leader: Ibrahim Magazi Ibrahim Magazi ICR Team Leader: Ibrahim Magazi ICR Primary Author: Peter Bachrach F. Results Framework Analysis Project Development Objectives The objective of the Project is to contribute to reaching the maternal and under-five mortality reduction objectives of the Borrower’s Five-Year Plan, Program, and Poverty Reduction Strategy Paper, and the Millennium Development Goals. Revised Project Development Objectives (as approved by original approving authority) The objective of the Project is to improve the coverage and quality of maternal and child health services in targeted districts and targeted health centers in the peri-urban areas of Conakry. (a) PDO Indicator(s) Original credit Original Target Formally Actual Value Achieved at Values (from Indicator Baseline Value Revised Completion or Target approval Target Values Years documents) Indicator 1 : Number of deaths of pregnant women has dropped from 528 per 100,000 in 2005 to 200 per 100,000 in 2011 Value (Quantitative 980 200 Eliminated in 724 or Qualitative) 2011 Date achieved 12/31/05 12/31/12 Comments (incl. % Achieved. The indicator was poorly formulated, the initial baseline was erroneous achievement) (528), and the original target was below the MDG 2015 (220). Based on the PAD indicator of avoiding 12,000 deaths in the project area, the target was achieved as an estimated 15,596 deaths were avoided (or 130% of the target). Indicator 2 : Number of deaths of children under a year old has dropped from 88 per 1,000 in 2005 to 50 per 1,000 in 2011 Value (Quantitative 91 50 Eliminated in 67 or Qualitative) 2011 Date achieved 12/31/05 12/31/12 Comments (incl. % Partially achieved. The original baseline was slightly incorrect (88), and the original achievement) target was roughly the MDG 2015 (45), but overall, the target was partially achieved. Based on the PAD indicator of avoiding 27,500 deaths in the project area, the target was partially achieved as an estimated 14,542 deaths were avoided (or 53% of the target). Restructured credit Indicator 3 : Institutional deliveries assisted by trained health personnel Value (Quantitative 14.0% 25.0% 38.0% or Qualitative) Date achieved 12/31/06 12/31/08 Comments (incl. % Achieved. Data on the original indicator (assisted deliveries, including TBAs) do not achievement) exist after 2009 as the indicator was redefined to include only institutional deliveries. Institutional deliveries assisted by trained health personnel increased from 14.0% to 38.0% (or 152% of the target). Indicator 4 : Vaccination coverage (DTCP3) among children 0-11 months has increased from 70% in 2004 to 90% in 2010. Value (Quantitative 70.0% 90.0% 54.0% 86.0% or Qualitative) Date achieved 12/31/11 12/31/13 Comments (incl. % Achieved. Vaccination coverage increased substantially in the project zone, surpassing achievement) the revised target (159%) and nearly achieving the original target. Indicator 5 : Children 12-23 mos. old fully immunized (Penta3+R) Value (Quantitative 37.2% 41.0% 36.5% or Qualitative) Date achieved 12/31/05 12/31/12 Indicator Baseline Value Original Target Formally Actual Value Achieved at Values (from Revised Completion or Target approval Target Values Years documents) Comments (incl. % Not achieved. While 90% of the target was achieved, vaccination coverage actually achievement) declined between 2005 and 2012. Indicators 6 & 7: Number of pregnant women receiving at least 1 ANC visit to a health provider Percentage of pregnant women receiving at least 1 ANC visit to a health provider Value (Quantitative 194,475 267,774 249,030 or Qualitative) 85.0% 90.0% 93.0% Date achieved 12/31/11 12/31/13 Comments (incl. % Achieved. In absolute terms, the average number of women receiving at least one ANC achievement) rose significantly, from 119,064 over the period 2006-10 to 226,618 over the period 2011-13 (a 90% increase). Overall, 93% of pregnant women received at least 1 ANC (103% of the target was achieved). Indicator 8 : Number of direct project beneficiaries, of which female Value (Quantitative 1,000,000 of which 3,000,000 2,558,729 or Qualitative) 534,884 female 1,604,651 1,368,623 Date achieved 12/31/11 12/31/13 Comments (incl. % Substantially achieved. The targets were achieved at 85%. achievement) (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Achieved Values (from Revised Indicator Baseline Value at Completion or Target approval Target Years documents) Values Indicator 1 : Number of health facilities providing standard IMCI services, including: (i) prenatal care and counseling; (ii) childbirth services; and (iii) immunization Value (Quantitative 0 250 150 213 or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. The project contributed to operationalizing IMCI services in 213 health achievement) centers (142% of the target) by supporting: (i) training in clinical and community IMCI; (ii) distribution and use of the appropriate management tools; (iii) post-training supervision and monitoring; and (iv) medicines. Indicator 2 : At least 200 Health Centers provide basic emergency childbirth services. Value (Quantitative 0 200 175 213 or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. The project contributed to operationalizing basic emergency childbirth achievement) services in 213 health centers (122% of the target) by supporting: (i) training (on basic emergency services, essential obstetric care, and ante natal care); (ii) equipment (including “ventouses”); (iii) post-training supervision and monitoring; and (iv) Original Target Formally Actual Value Achieved Values (from Revised Indicator Baseline Value at Completion or Target approval Target Years documents) Values medicines. An evaluation by UNFPA in 2012 showed, however, that not all of the health centers met all of the seven criteria. Indicator 3 : At least 18 Prefectoral hospitals and 2 Comprehensive Health Centers (CHC) provide full emergency childbirth services. Value (Quantitative 0 20 20 22 or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. The project contributed to operationalizing full emergency childbirth services achievement) in 20 hospitals and 2 CHC (110% of the target) by supporting: (i) training; (ii) medical equipment; (iii) transport (ambulances, operations, etc.); (iv) functional blood banks; (v) electrical generators; and (vi) operational costs for caesarian sections. Indicator 4 : Caesarian-section deliveries as a percentage of all deliveries have increased from 1.8% in 2004 to 5% in 2010. Value (Quantitative 1.80% (2004) 5.00% 2.00% 1.54% or Qualitative) 1.52% (2010) Date achieved 12/31/04 12/31/13 Comments (incl. % Not achieved. The rate for caesarian section deliveries remained virtually unchanged achievement) from 2010 and less than in 2004. Indicator 5 : Percentage of pregnant women utilizing the Intermittent Treatment Program to prevent malaria has increased from 0% in 2005 to 80% in 2010. Value (Quantitative 0.0% 80.0% 80.0% 70.9% or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Partially achieved. The percentage of women utilizing ITP varied annually over the achievement) period 2011-13; on average 70.9% of women (89% of the target) utilized ITP. Indicator 6 : At least 130 new community health insurance schemes (mutuelles) have been created in the 18 targeted Prefectures between the Effective Date and 2011. Value (Quantitative 0 130 89 53 or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Partially achieved. Some 53 community health insurance schemes were initiated in achievement) 2013; this represents 60% of the target. The Borrower has carried out an evaluation of Health Center management by local Indicator 7 : authorities (Communes and CRDs) by 2011. Value (Quantitative 0 1 Eliminated in Completed or Qualitative) 2011 Date achieved 12/31/06 12/31/08 Comments (incl. % Achieved. The evaluation/feasibility study was conducted in 2008. achievement) Original Target Formally Actual Value Achieved Values (from Revised Indicator Baseline Value at Completion or Target approval Target Years documents) Values Indicator 8 : By 2008: (a) all Health Structures are subject to supervision at least every other month by the Prefectoral-level; (b) all Prefectoral-level Health Structures are subject to supervision at least every quarter by the Regional-level; and (c) all Regional-level Health Structures are subject to supervision at least every semester by the central-level. Value (Quantitative 20% 100% 80% 100% or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. In 2008, prior to the suspension, the project had achieved 100% (though this achievement) result declined to 0% during the suspension). After project restructuring, the project achieved 100% of the planned supervisions. Indicators 9 & 10: The Borrower has satisfactorily implemented the Five-Year Plan, as evidenced by the results of the annual reviews An annual review of implementation of the Five-Year Plan has been carried out jointly by the Borrower and the Association each year from and including 2007. Value (Quantitative 0 4 (2007-10) Eliminated in 0 or Qualitative) 2011 Date achieved 12/31/06 12/31/13 Comments (incl. % Not achieved. No joint annual reviews were conducted. achievement) Indicator 11 : Percentage of the national health budget allocated for Health Centers reaching such centers has increased from 30 in 2002 to 70 in 2010, as evidenced by the results of the health expenditure tracking survey. Value (Quantitative 30% 70% Eliminated in or Qualitative) 2011 Date achieved 12/31/02 Not measured Comments (incl. % Not measured. Annual health expenditure tracking surveys were not conducted until achievement) 2013; the 2013 survey does not provide the necessary information. Indicators 12 & 13: The number of contracts concluded between the MOPH and the private sector for purposes of health service delivery by midwives and for purposes of training of health personnel has increased from nil per annum in 2005 to 50 per annum in 2010. At least 90% of the contracts have been executed by 2011. Value (Quantitative 0 Number: 50 Eliminated in 15 or Qualitative) Percentage: 90% 2011 100% Date achieved 12/31/06 12/31/09 Comments (incl. % Partially achieved. The number of contracts was partially achieved (30% of the target); achievement) all of the 15 contracts were completed during 2008-09. Indicator 14 : At least 18 incinerators have been purchased and installed in the territory of the Borrower between the Effective Date and 2011. Value (Quantitative 0 18 18 14 (by 12/31/2010) or Qualitative) 18 (by 12/31/2013) Indicator Baseline Value Original Target Formally Actual Value Achieved Values (from Revised at Completion or Target approval Target Years documents) Values Date achieved Project data (2013) Comments (incl. % Achieved. Seventeen incinerators were purchased and installed by the project; one was achievement) purchased and installed by WHO. Project records indicate that 14 were installed by the end of 2010 and 18 installed by project closing. Indicator 15 : At least 844 medical personnel have been trained in MWM by 2011. Number of health personnel receiving training. Value (Quantitative 844 300 2,854 or Qualitative) Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. Originally, 844 personnel were to be trained in medical waste management, achievement) but training data was not disaggregated by subject area. Subsequently, 300 personnel were to be trained; project data show that 2,854 staff were trained in a variety of medical, technical, and administrative areas. Indicator 16 : MOPH has executed at least 70% of its annual budget allocation by the end of each Fiscal Year from and including Fiscal Year 2006 Value (Quantitative 72.7% 70.0% Eliminated in 88.1% (2010) or Qualitative) 2011 97.3% (2012) Date achieved 12/31/05 12/31/12 Comments (incl. % Achieved. Results from the PER show large annual variations, but except for 2011 achievement) (65.1%), the annual results from 2006-2012 were always greater than 70%. Indicator 17 : Number of pregnant women living with HIV receiving ARVs to reduce MTCT risk Value (Quantitative 519 800 0 or Qualitative) Date achieved 12/31/10 12/31/13 Comments (incl. % Not achieved. Services were expanded, training/supervision conducted, and 9763 achievement) pregnant women needing ARVs identified; but the project was unable to provide ARV drugs because the procurement process was too long for the time left until closing. Indicator 18 : 100 health centers have participated in the self-evaluation process by project mid-term. Value (Quantitative 0 100 Eliminated in 116 or Qualitative) 2011 Date achieved 12/31/06 12/31/13 Comments (incl. % Achieved. With technical support from GTZ (now GIZ), the self-evaluation process was achievement) conducted for 116 health centers (combined with a contest for the best quality services). Indicator 19 : Long-lasting insecticide-treated malaria nets purchased and/or distributed Value (Quantitative 0 795,000 Project: 234,000 or Qualitative) Total: 1,140,484 Date achieved 12/31/06 12/31/13 Comments (incl. % Partially achieved. Less than a third (29%) of the nets was distributed. The project’s achievement) objective was not to provide mass distribution but rather to link distribution with antenatal and child well visits. Indicator Baseline Value Original Target Formally Actual Value Achieved Values (from Revised at Completion or Target approval Target Years documents) Values Indicator 20 : Percentage of project annual budget executed. Value (Quantitative 29.0% 90.0% 98.0% or Qualitative) Date achieved 12/31/11 12/31/13 Comments (incl. % Achieved. Annual project expenditures for 2013 were 98% of the annual project budget, achievement) exceeding the target. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (XDR millions) 1 22-Nov-05 S S 0.00 (0.00%) 2/3 30-Jun-06 S MU 0.10 (0.60%) 4 11-Jan-07 S MU 1.16 (6.99%) 5 25-Jun-07 S MU 1.38 (8.31%) 6 27-Dec-07 U MU 2.57 (15.48%) 7 24-Jun-08 U MS 4.22 (25.42%) 8 4-Dec-08 U MS 5.77 (34.76%) 9 11-Jun-09 U MS 5.88 (35.42%) 10 21-Nov-09 U U 5.88 (35.42%) 11 18-Jun-10 U U 5.88 (35.42%) 12 1-Dec-10 U U 5.88 (35.42%) 13 24-Mar-11 U U 5.88 (35.42%) 14 21-Dec-11 U U 8.22 (49.52%) 15 11-Jun-12 U U 9.05 (54.52%) 16 10-Dec-12 MS MS 10.96 (66.02%) 17 25-Jun-13 MS MS 12.30 (74.10%) 18 31-Dec-13 MS MS 13.84 (83.37%) H. Restructuring Board ISR Ratings at Amount Restructuring Approved Restructuring Disbursed at Reason for Restructuring & Key Changes Date(s) PDO Restructuring Made DO IP Change in US$ millions Jan. 4, 2012 Revise unsuitable PDO; simplify the design; Yes U U2 12.58 Mar 20, 2012 1 modify the institutional arrangements; revise 1 Prior to submitting the project restructuring package, approved on January 4, 2012, to the Government, it was noticed that the signed documents were not the most recent version. A corrected package of documents and agreements was signed by the Country Director and the Minister of Finance on March 20, 2012. disbursement categories; and extend the closing date If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Modest Against Formally Revised PDO/Targets Substantial Overall (weighted) rating Substantial I. Disbursement Profile 2 The unsatisfactory ratings were the result of the inability to supervise the project during the more than two year period of its suspension after the December 2008 coup. 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context. After initiating implementation of the first poverty reduction strategy (PRSP-I) in 2002, Guinea continued to face a particularly difficult economic and social situation. The average annual economic growth rate was estimated at 2.3 percent, compared with the goal of 5 percent. Per capita GDP dropped from $379 in 2002 to $332 in 2006; at the same time, inflation rose from 6.1 percent in 2002 to 39.1 percent in 2006, further contributing to a substantial deterioration in the purchasing power of the population. Nationally, the poverty rate rose from 49.2 percent in 2002 to 53.6 percent in 2005; 19.1 percent of the estimated population of 10.3 million were considered in extreme poverty, with an income below US$116 per person per year. 2. Though essentially a rural phenomenon (60 percent of Guineans living in rural areas account for 86 percent of overall poverty), urban poverty was also a growing phenomenon, with the incidence of poverty at 21 percent in Conakry and 27 percent for the other urban areas. Upper Guinea (67.5 percent) and Middle Guinea (55.4 percent) were the country’s poorest regions, and included the country’s poorest prefectures: Tougué (72 percent), Mandiana (71.3 percent), Mali (71 percent), Koubia (70.8 percent), Dinguiraye (69.7 percent), and Lélouma (67.6 percent). 3. Weak economic growth, poor governance and corruption were key factors in the high incidence of poverty. Politically, the national dialogue begun in 2003 initially raised hopes of ending the tensions that had characterized politics since the first elections in 1993. However, despite the introduction of electoral reforms, the results of the municipal and regional elections (held in December 2005) were contested on the grounds that they lacked transparency. Further, while Guinea had the legislation and institutional framework needed, powerful interests continued to delay the implementation of these means to strengthen the rule of law, with a disproportionate impact of the poor and powerless. 4. Sectoral Context. Comparisons of the results of the Demographic and Health Surveys (DHS) showed mixed results between 1999 and 2005. Infant mortality declined from 98 to 91 per thousand live births and infant-child mortality was reduced from 177 to 163 per thousand, but maternal mortality increased from 528 to 980 per 100,000 live births, which is among the highest rates in West Africa. Use of modern contraceptive methods increased (from 4 percent to 6 percent, but so did fertility (from 5.5 to 5.7). Among pregnant women, the proportion receiving professional prenatal care and delivering in a health facility increased modestly from 71 percent to 82 percent and from 29 percent to 31 percent. During the same period, the proportion of children (12-23 months of age) completely vaccinated rose from 32 percent to 37 percent, but moderate malnutrition increased from 23 percent to 26 percent. 5. Despite the high incidence of poor nutrition, inadequate hygiene, and communicable and endemic diseases, utilization of health services was low as was the satisfaction of the recipients of health services. The 2007 PRSP noted that more than half (53.7 percent) of recipients (and especially those in rural areas) were dissatisfied with the services provided, particularly for the high cost and the low quality of treatment (including long waits and insufficient drugs). 6. Systemic problems in the sector related to: (i) inadequate sector financing. Between 2001 and 2005, the budget allocated to the health sector declined from 4.5 to 3.4 percent, and only 30 percent 10 of the allocated budget was actually executed (according to the 2003 Health Expenditure Tracking Survey); (ii) significant shortages of skilled health professionals in the country and a disproportionate number of these professionals located in Conakry; and (iii) unavailability of essential drugs and consumables. Stock-outs of essential drugs and vaccines undermined service quality. 3 7. Country Assistance Strategy and Rationale for Bank Involvement. The 2003-06 Country Assistance Strategy (CAS) supported the implementation of the country’s First Poverty Reduction Strategy Paper (PRSP) and included pillars to foster sustainable and equitable growth, to improve access to and the quality of basic services, and to strengthen governance and institutional and human capacity. The CAS was consistent with (i) the health priorities presented in the National Health Development Plan (2003-12) and the Five Year Health Development Plan (2003-07), and (ii) the proposed transition from a project approach to a programmatic approach by strengthening Guinea's fiduciary capacities and institutional arrangements for service delivery. 8. In addition to its extensive experience in the health project (with a series of health projects and a MAP over the period 1986-2005 and its projects in related areas of governance and local capacity building 4, the Bank’s experience and support of a programmatic approach made its participation important and useful. Given the existing status of the health sector, the Bank proposed: (i) to proceed with a classic investment project (emphasizing maternal mortality, which constitutes a useful proxy for health system performance) rather than playing the role of lender of last resort; and (ii) to assist the Ministry in establishing the conditions for a more programmatic approach. To this end, the Bank expected to collaborate closely with other partners in the sector (particularly, the European Union, UNICEF, France, and GIZ) to support the achievement of the PRSP goals by implementing MOH’s National Health Development Plan (2003-12) and the Five-Year Plan (2003- 07) to achieve the health specific MDGs as reflected in the PRSP objectives. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 9. The original Project Development Objective (PDO), as stated in the Development Credit Agreement (DCA), was to contribute to reaching the maternal and under-five mortality reduction objectives of the Borrower’s Five-Year Plan, Program, and Poverty Reduction Strategy Paper, and the Millennium Development Goals. Specifically, in the 18 targeted districts, the project was expected to help decrease: (i) maternal mortality from 528 per 100,000 live births in 1990 to slightly above 200 per 100,000 live births in 2011; and (ii) infant mortality from 98 deaths per 1,000 live births in 1999 to around 50 deaths per 1,000 live births in 2011. These key PDO results would be achieved by avoiding the deaths of 12,000 pregnant women and 27,500 children under 1 year old. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 10. The objective of the restructured Project, as stated in the revised DCA, was to improve the coverage and quality of maternal and child health services in targeted districts and targeted health 3 See World Bank, Country Status Report on Health and Poverty (2003), pp. 42-44. 4 These included the institutional strengthening project, the urban project, and the community support program (PRCI, PUIII, and PACV, respectively). 11 centers in the peri-urban areas of Conakry. 11. The revised PDO was justified by the need to limit the project’s ambitions since reductions in maternal and infant mortality will require: (i) more than the project lifetime to change; and (ii) actions beyond the health sector that are not part of project activities. As a consequence, the results chain was modified and more realistic indicators selected which could more effectively measure the project’s achievement and would be more likely to be met by project closing. The revised measures of project achievement for the period 2011-13 were defined as follows: • By 2013, the proportion of institutional deliveries assisted by trained health personnel increases from 21 percent to 25 percent • By 2013, the proportion of children 12-23 mos. old fully immunized (DPT3+M) increases from 39 percent to 41 percent • By 2013, the proportion of children 0-11 mos. immunized (DTC3) increases from 52 percent to 54 percent • By 2013, the number and proportion of pregnant women receiving at least 1 ANC visit to a health provider increases from 194 475 to 267 774 and from 85 percent to 90 percent respectively • By 2013, the number of project beneficiaries (of which female) increases from 1.96 million (1.05 million female) to 3.00 million (1.6 million female) 12. Data on the original and revised key outcome and intermediate outcome indicators are presented in Section F of the Data Sheet and discussed in more detail in Section 3.2 and Annex 2. 1.4 Main Beneficiaries and Benefits 13. The project’s main beneficiaries were expected to be pregnant women and children in the poorest areas of Guinea. The emphasis on pregnant women was especially justified both by the severity of maternal mortality in Guinea and by its significance as an overall indicator of health system performance. The emphasis on children, particularly those under one year of age, was justified by the need to promote a complete package of basic services to address the low utilization of health facilities and the poor health outcomes of this population. Sixteen of the 18 project prefectures were selected based on the national poverty map and have the worst health indicators; the two other prefectures (Gueckedou and Kissidougou) were added due to the influx of refugees from neighboring countries. 14. Other populations at all levels of the country were also expected to benefit from the project. These included: (i) the Ministry of Health, which would benefit from support in collecting and analyzing data (on monitoring health expenditures), developing health reforms (such as national insurance and performance contracting), and preparing for budget support; (ii) local authorities, who would benefit from the pilot effort to establish community-managed facilities; (iii) public and private health facilities, which would benefit from incentive grants to improve the quality of health services; and (iv) local communities, which would benefit from grants to increase village level awareness of critical health problems. 15. The project was expected to have other positive benefits, including: (i) reducing maternal and child mortality by increasing access to and improving the quality of health services; and (ii) contributing to essential sector management reforms at central, local, and community levels. 12 1.5 Original Components 16. Component 1: Strengthening Health Care Services (Appraisal: US$18.99 million equivalent; Restructuring: US$20.40 million; Actual: US$17.62 million or 93 percent of appraised cost). This component sought to strengthen the ability of health posts, health centers and hospitals in the 18 selected prefectures to provide a package 5 of quality services to at least one million pregnant women and 933,000 children less than one year of age. The project was expected to focus on three sub-components: (i) improving the quality of health care; (ii) supporting community mobilization; and (iii) instituting quality insurance. 17. Health care quality would be improved for pregnant women and children by: (i) reinforcing the chain of care for at-risk deliveries, starting with early identification by traditional birth attendants (TBAs); (ii) strengthening emergency delivery care at higher service levels, especially the ‘Centres de Santé Améliorés”, private clinics (including NGOs), and prefectoral hospitals; (iii) increasing the utilization of family planning services and IMCI activities (namely EPI, diarrhea control, acute respiratory illnesses, malaria, nutrition); (iv) strengthening the roll-back malaria program(through malaria prevention, treatment, and operational research via insecticide-treated bed-nets, anti-malarial drugs, and advisory services). According to the level involved, the project intended to invest in: (i) basic medical and office equipment; (ii) the referral system (ambulances and communications); and (iii) drugs, vaccines, and other medical supplies. To increase the availability of skilled midwives, the project intended to finance the existing master plan for human resources and to contract retired and other midwives to fill the gap (especially in rural areas) during the three-year training period required. The health care quality sub-component would be implemented mainly through performance-based agreements signed between MoH and health centers and district hospitals and contracts with private clinics and NGOs. 18. Community mobilization would be achieved through support for: (i) the existing community health insurance (mutual) system (MURIGA), which provides coverage mainly for women’s health risks related to child birth; (ii) pilot activities in six prefectures to test the feasibility of decentralizing management of selected health facilities to the urban and rural communes; (iii) increased public awareness and official accountability for actual Government health expenditures through publication of the planned budgets and actual expenditures of the health facilities in the project areas; and (iv) community sub-projects for village-level awareness activities related to the early detection of at-risk pregnancies and their follow-up, full DPCT3 immunization, and development of the MURIGAs. Depending on the activity involved, the project intended to invest in equipment, vehicles, training and technical advisory services, and sub-project financing. 19. Quality assurance, tested by GIZ and evaluated in 2004, would be expanded to the 18 project districts to: (i) finance self-evaluations of and plans for improving health facility performance; (ii) award quality incentive grants to the best plans; and (iii) assist and monitor the implementation of the plans. In addition to the quality incentive grants, the project was expected to finance equipment and training and technical advisory services. 5 The package of services comprised: (i) prenatal care, normal and emergency delivery for women; and (ii) integrated management of childhood diseases and control of communicable diseases (TB, malaria) for children. 13 20. Component 2: Institutional Strengthening (Appraisal: US$8.37 million equivalent; Restructuring: US$5.60 million; Actual: US$5.91 million or 71 percent of appraised cost). The project was expected to: (i) strengthen the Ministry’s leadership and policy-making capacity; (ii) support the ministry’s operational policies and practices; and (iii) finance project management and coordination. 21. Leadership and policy-making capacity as well as health sector reform would be reinforced through: (i) integration of maternal mortality into the health information system; (ii) assistance to the structure in charge of developing the national insurance scheme; (iii) financial support for carrying out an annual public health expenditure tracking survey. 22. Ministry operational policies and practices would be strengthened by: (i) technical support from a Fiduciary Management Agency (FMA) to help MOH’s Administrative Unit (DAAF) and decentralized units prepare for budgetary support; (ii) monitoring MOH’s procurement of works, goods, and services; (iii) jointly financing equipment for the Reproductive Health Training and Research Center in Conakry; and (iv) further implementing the country’s Medical Waste Management Plan (MWMP). 23. Project management and coordination would be implemented through: (i) establishment of a Pilot Committee for the Five-Year Plan and a Task Force to facilitate technical implementation of the project; and (ii) support (including incremental operating costs) for entities involved in Project implementation. 1.6 Revised Components 24. The Level 1 restructuring of the project (approved on January 4, 2012) modified both of the project’s components as indicated below. For Component 1: (i) the health care quality sub- component was expanded to include health centers in peri-urban areas of Conakry and PMTCT as part of the MCH service package and the integration of maternal mortality information was expanded to address strengthening M&E more generally; (ii) community mobilization was revised to eliminate support to MURIGAS (since deliveries were made free of charge); and (iii) quality assurance was dropped (as it was considered a medium-term issue to be addressed through economic and sector work (ESW). 25. For Component 2: (i) the leadership and policy-making sub-component was modified to eliminate development of national insurance (which became the focus of ESW), annual expenditure tracking surveys, and measures to strengthen health sector reform; (ii) the operational policies sub- component continued support for the Reproductive Health Training and Research Center and the Medical Waste Management Plan implementation; and (iii) the project management and coordination sub-component was expanded to incorporate support for the DAAF. 1.7 Other significant changes 26. In addition to the restructuring (discussed in Section 2.2), the DCA was amended to: (i) eliminate counterpart financing (March 2008) and to reallocate the proceeds (March 2008 and March 2012); and (ii) revise the closing date of the project (from September 30, 2011 to December 31, 2011 and then to December 31, 2013. 14 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 27. Project preparation. An initial project identification mission was conducted in May-June 2000. The project was later dropped, as the Bank decided to assist the country through a multi- sectoral effort (including a health component) rather than through a targeted intervention. Subsequently, the Bank reinstated the project 6, approved the Project Concept Note (March 2003), carried out preparation missions in May and November 2003, and pre-appraisal missions in February 2004, and March 2004. 7 28. The Decision Meeting (June 2004) proposed several important changes for the PAD to revise the PDO, increase the project cost from US$20 million to US$25 million, establish the amount of counterpart funding, change the risk ratings to reflect the recent experience in Guinea, and determine the conditions for negotiations and effectiveness. The appraisal mission was conducted immediately after the Decision Meeting, and negotiations were expected (in principle) to be held in July 2004. As a result of the readiness conditions agreed on at the Decision Meeting, however, negotiations were delayed until April 2005. The project was approved by the Board on June 2, 2005. 29. Soundness of the background analysis. The background analysis reflected the evolving nature of the project’s contribution to the CAS. The PHRD grant (approved in May 2000 for US$400,000 and amended in 2002) was intended to prepare the elements for a sector-wide approach; the Project Preparation Facility PPF (requested in June 2003 and approved in January 2004 in the amount of US$600,000) modified the approach to prepare the groundwork for a regular investment operation with the Bank identifying specific investments for project financing rather than positioning itself as the lender of last resort. The following table summarizes the planned background studies: Table 1: Background analyses for the project PHRD of May 15, 2000 PHRD as amended April 29, 2002 PPF of January 12, 2004 Health sector budget and expenditures Health infrastructure mapping/planning analysis exercise, including the private sector Health sector financing assessment Health sector financing assessment Analysis of health management capabilities at regional, district, and community levels Health facility utilization assessment Health facility utilization assessment Ten-year HR plan for frontline health workers Three-year and annual plan preparation Feasibility of contracting services through the at district and national levels private sector Analysis of issues related to the Integration of maternal/infant mortality and provision of drugs in the public sector private sector statistics National client satisfaction survey Feasibility analysis for community management of health centers 6 The project was included in the CAS as phase two of the existing APL rather than as a (more complicated) sector-wide approach (SWAP). The project was to be designed as a transition operation preparing the sector for funding of the third APL phase under a PRSC. 7 The initial Decision Meeting in February 2004 concluded that progress in preparing the project was not sufficient to conduct appraisal as scheduled in March 2004. 15 30. Virtually all of the studies were initiated under the PHRD grant and the PPF, but despite extensions for both 8, most were not completed by project effectiveness; however, except for the analysis of drug supply issues, they were all eventually completed during project implementation. 31. Project preparation also relied on the lessons learned from: (i) the three previous health projects financed by the World Bank since 1988; and (ii) successful operations (e.g., the PRCI, the PACV, and the Urban Project, which had piloted results agreements between the central and municipal government levels) which provided a model for MOH to follow in better defining the role of the regional and district health authorities with respect to the local authorities (municipalities and CRDs). 32. Assessment of the project design. Project design involved considerations of the appropriate instrument as well as the specific content. Over the lengthy period for project preparation, the proposed approaches for the project evolved in relation to the Bank’s appreciation of Guinea’s country context and implementation capabilities: • from an Adjustable Program Loan (APL) intended to: (i) complement the Population and Sexual Health Project (PPSG); (ii) provide flexibility in addressing health issues and introducing performance indicators; and (iii) establish the Bank as the donor of last resort; • to a program or sector-wide approach, based on support for the National Health Development Program (PNDS), experience with IDA-financed local level contracting (through PACV and PRCI), and the eventual transition to budget support; • to a regular investment operation based on the unfeasibility of pooling donor resources 9 and the determination that the World Bank should not be the lender of last resort but rather a contributor to specific outcomes in the poorest areas of the country. 33. Though a traditional investment operation was ultimately agreed on, several of the earlier intentions were maintained as the PAD included: (i) implementation arrangements to integrate the project within the existing ministry structures; and (ii) criteria and indicators in the Results Framework (RF) to measure the ministry’s readiness for graduation to a PRSC. 34. Within the context of a traditional operation, it was decided to focus the project on reducing maternal mortality rates because: (i) maternal mortality was particularly elevated; (ii) reductions in maternal mortality represented a proxy for overall improvement in health system performance; and (iii) improvements in this indicator would require the MOH to focus on the poorer populations. 35. Not surprisingly, the project design was somewhat of a hybrid comprising elements of both approaches in each of the final components. Component 1 (comprising more than 70 percent of the proceeds of the credit) intended both to strengthen MCH services and to introduce innovative approaches (e.g., community mobilization through mutual health insurance, quality assurance, etc.); while promising, these approaches were untested. Similarly, Component 2 envisioned a project integrated within the MOH under the leadership of the Secretary General. However, concerns about the ministry’s readiness resulted in several project-style conditions, such as preparation of a project implementation manual, recruitment of a Fiduciary Management Agency (FMA) to assist the 8 The original closing date of the PPF was initially extended from October 30, 2004 (the planned date for effectiveness) to June 30, 2005 and then to December 22, 2005. 9 As highlighted by the Public Expenditure Review and Public Expenditure Tracking Survey. 16 ministry’s Directorate for Finance and Administration (DAAF), etc. In addition, though decentralization was a key element of the project (with health structures to receive funds for eligible activities based on a performance agreement signed with MOH), the modalities were not adequately explained in the PAD. 36. Government commitment/Risk assessment and mitigation. The PAD indicated that the Government had adopted a number of relevant policies and strategies (PRSP, PNDS, Five-Year Plan, etc.) confirming its commitment, but the PAD did not otherwise explicitly address the issue of government commitment. Rather, much of the discussion in the PAD (and PAD reviews) focused on the potential risks of the design and of the environment, especially: (i) macroeconomic (and the likelihood of counterpart financing); (ii) political (and the possible effects of instability and conflict); and (iii) governance (and especially equity). 10 37. The initial risk assessment was inconsistent and the proposed mitigation measures were cursory. The Decision Meeting recommended changing the risk assessment from “modest” to "substantial" if not "high" and requested that the risks at the various levels (community participation, decentralization to regions and districts, weak ministerial leadership) be enumerated; the PAD did the former. 11 Other concerns about the project’s risk were reflected in the significant number of conditions for negotiation and effectiveness. 38. Quality at Entry. No quality at entry was carried out. 2.2 Implementation 39. The Development Credit Agreement was signed on September 29, 2005. Project effectiveness was delayed by the difficulties in meeting the conditions, including the project implementation manual, the initial deposit of counterpart funds, the contracts with the Fiduciary Management Agency and the independent financial auditors, and the conclusion of performance-based contracts with at least 10 health structures in the 18 targeted districts. Then, when these conditions were met (February 2006), the illness of the president delayed ratification of the credit, which became effective on June 26, 2006. 40. Disbursement overview. Under the financing plan for the project, the Bank was expected to finance US$25.0 million and the Borrower was expected to finance US$2.8 million equivalent bringing the total project cost to US$27.8 million equivalent. 41. As the following table shows (see the disbursement profile as well in Section I of the Data Sheet), the project’s disbursements lagged the PAD’s planned disbursements through the end of 2008, at which point the suspension brought spending to a virtual halt for 2009-10. After project restructuring, expenditures accelerated quickly and continued at a high level throughout the period 2011-13. 10 Though interested in the SWAP approach, Government commitment was tentative, possibly because of the reluctance of the other donors in the sector to join. 11 Of the four risks, three were rated as moderate and the fourth substantial, but the overall risk rating was high, reflecting the Decision Meeting’s view. 17 Table 2: Comparison of planned and actual IDA disbursements ('000 USD) (as of 3/27/14) Projected/Actual Financing 2006 2007 2008 2009 2010 2011 2012 2013 Total Projected financing (PAD) Annual total 1 500 2 500 5 000 7 000 6 000 3 000 Cumulative total 1 500 4 000 9 000 16 000 22 000 25 000 25 000 Actual disbursements Annual total 1 714 2 177 5 194 0 0 3 718 4 901 5 810 23 513 Cumulative total 1 714 3 891 9 084 9 084 9 084 12 802 17 703 23 513 % of planned 114% 97% 101% 57% 41% 51% 71% 94% Source: PAD and Client Connection 42. Implementation overview. The project began after a year’s delay and was implemented throughout in a very difficult socio-economic and political context: (i) violent street riots in 2007; (ii) a military coup in December 2008 leading to the suspension of Bank operations; (iii) presidential elections in 2010; (iv) Bank re-engagement and a restructuring of the project in 2011; and (v) on-going protests over the country’s deteriorating economic situation. 43. The project was confronted almost immediately after start-up by a number of internal constraints. First, the integration of implementation arrangements within the ministry met with institutional constraints: (i) because all decisions had to pass through the Secretary General, there were often delays in requesting required actions (e.g., for non-objections, reallocations of project funds); (ii) the project coordinating Task Force comprising high-level MOH staff was unable to maintain the planned meeting schedule and contribute effectively to the implementation of the project; and (iii) collaboration between the DAAF and the FMA was handicapped by the absence of the designated MOH staff. 12 Second, the unavailability of the requisite health personnel for the project area (which comprised eighteen of the poorest and most remote districts) required the local recruitment of health workers. 13 Third, an early supervision mission expressed concern that the Government’s critical underfinancing of the sector would alter the project’s role from contributing (along with other partners) to overall sector financing to substituting for the National Budget. 14 44. The Bank responded to these constraints by: (i) revising the Bank’s financial parameters to provide Bank financing for 100 percent of eligible expenditures; and (ii) contributing to the 2007 emergency plan by increasing (by 15 percent) the quantities of goods being procured (drugs, mosquito nets, equipment, etc.). 15 In addition, the Bank agreed to reallocate US$2 million from the training to grants for the Annual Operating Plans (PAO) of the regions, districts, hospitals, and health centers. Finally, the Government and the Bank concluded in October 2007 that the project 12 Despite repeated reminders from the supervision missions to the ministry about the DCA obligation to name counterparts to work with the FMA, MOH staff was never effectively available. As a consequence, the FMA contract was maintained throughout project implementation. 13 The Government also took action on a national level to address the problem by raising the retirement age (for those willing) for certain categories of personnel (e.g., midwives and nurses). 14 The Bank was also concerned with the difficulties of mobilizing counterpart funding and the implications for project implementation; for example, virtually the entire amount of the Government’s initial counterpart contribution was disbursed to meet its previous PPF obligations. 15 In October 2007, the Government instituted a free-service policy for cesarean sections; anti-retroviral drugs; drugs to treat opportunistic infections (tuberculosis), along with specific biological exams; and impregnated mosquito nets. As the Public Expenditure Review (p. 94) notes, funding for this policy was “miniscule.” 18 would need to be restructured after the mid-term review. 45. Restructuring began in February 2008 and the project was authorized to hire three full-time consultants to assist in the implementation of the project’s components. Subsequent supervision missions advanced the restructuring process, but it had not been completed at the time of the suspension in December 2008. A Level 1 restructuring was completed in 2011 with: (i) a revision of the PDOs and related intermediate outcome indicators; (ii) a streamlining of project activities; (iii) a revision of institutional arrangements; (iv) a reallocation of credit proceeds; and (v) an extension of the closing date. 46. In essence, the project was effectively implemented over two relatively brief periods (2007-08 and 2012-13 16 ), which were distinguished by: (i) the transfer of implementation responsibilities from the ministry during the initial phase to an autonomous unit during the latter phase; and (ii) the improvement in communication and decision-making between the project and the Bank. During this latter phase, a concerted effort was made to develop quantitative data on the project indicators: (i) publication of available (but unpublished) annual statistics for the years 2006-11 was financed; and (ii) based in part on these (and project) data, annual results for all of the indicators for the entire duration of the project were constituted. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 47. M&E design. A Results Framework with annual objectives was prepared, though without the definition of numerators and denominators for the KPIs. There were other problems as well with the KPIs. First, only two outcome indicators were linked to the services component and none to the institutional capacity component. Second, the intermediate outcome indicators were a mix of both population-based results and project outputs, and they were not well understood by the project. The restructuring paper recognized the flaws, replacing the 2 initial outcomes with 6 new or revised outcomes, eliminating 14 intermediate objectives, and replacing them with 12 new or revised intermediate outcomes. 48. The project’s M&E plan envisioned several layers: (i) monitoring of project expenditures by the FMA as well as internal (carried out by the FMA) and external audits at all levels; (ii) monitoring of project activities as defined in the annual operational plans and contracts/agreements funded by the project at the national, regional, district, and health facility levels; (iii) monitoring of project results through the health management information system (with a subset of data for the eighteen project districts); and (iv) evaluation of outcome and impact through Demographic and Health Surveys to be conducted at the beginning and end of the project. 49. Finally, other studies (e.g., health expenditure tracking, assessment of the quality of health services delivered, and a survey of client satisfaction) were envisioned on an annual basis. Prior to the Mid-Term Review, additional studies and surveys were planned to measure progress toward attaining the development objectives of the project. 16 Though the project was effective in June 2006, very few activities (see Annex 2) were carried out in 2006; similarly, a number of activities (based on the 2009 annual work plans) were restarted in 2011, but the project was not approved until December 28, 2011 and the revised DCA was not signed until March 20, 2012. 19 50. Implementation. The M&E ratings in the ISRs varied over time; (i) from Satisfactory over the first year; (ii) to Moderately Unsatisfactory (due mostly to the realization that the indicators were unsuitable) before recovering to Moderately Satisfactory during the period prior to the suspension; (iii) to Unsatisfactory over the period of project suspension; and (iv) to Satisfactory for the final period after restructuring. 51. Physical and financial monitoring. The combination of competent technical assistance (FMA), newly procured computer systems, and proven accounting software (TOMPRO) enabled the project to prepare the financial management reports required by the credit agreement. These reports were generally on time and acceptable to the Bank, though various supervision missions noted that they did not always adequately address physical results and the link between expenditures and outputs. 52. Project monitoring. As a result of the many parallel (and often competing) information systems comprising the ministry’s health management information system, project monitoring of the agreed-upon indicators was initially: (i) tardy and unreliable for the country as a whole; and (ii) not always applicable to the districts covered by the project. Throughout 2007-08, Bank supervision missions provided technical assistance to improve project monitoring and evaluation. In addition, the project financed preparation of a plan to strengthen the overall management information system through unified data collection formats, hardware and software for analysis, training, and support to ensure the use of the resulting information. Subsequently, the project financed the publication of the annual statistical data for the years 2007-2011. 53. The mid-term review of the project was initially planned for March 2008 but then postponed to November 2008 and subsequently to early 2009. With the suspension of the project in 2009, the mid-term review was cancelled. Though considered after restructuring, there were concerns that not enough time had elapsed between the restructuring and an eventual MTR; consequently, the Bank and the Government agreed to accept the restructuring as the MTR. 54. Surveys and research. Aside from the description of the PHRD and PPF requirements at the beginning of the project and the conduct of a follow-up DHS at the end of the project, the PAD did not identify the surveys and research to be carried out during project implementation. As indicated in Annex 9, the project produced an important number of studies and reports essentially in three waves: (i) prior to effectiveness when the PHRD studies were completed; (ii) in 2008 when the bulk of the PPF studies were completed; and (iii) in 2012-13 when several sector financing studies and technical evaluations were completed. The DHS results of 2004 and 2012 (funded in part by the project) as well as the MICS results of 2010 (using the DHS methodology but for a more limited number of variables) provide a reasonable series of data points for measuring project progress. 55. Utilization. While supervision missions noted a history (beginning with the Bamako Initiative) of conducting periodic monitoring missions and collecting information (particularly at health facility and district levels), they also pointed out the lack of analysis and use for decision- making at regional and central levels. Among the weaknesses noted in data utilization were: (i) concerns about the quality of the data which reduced the interest in and credibility of the information produced; (ii) the irregular meetings of the various coordinating structures, which 20 eliminated the occasions for exchanging results 17; and (iii) the lack of a communications strategy involving the dissemination of results. 56. M&E is rated modest. The range of information collected and disseminated during project implementation was substantial and included: (i) financial reports; (ii) consistent annual project and HMIS monitoring data (helped by the project’s hiring of and M&E specialist and contribution to the publication of the annual statistical digests); (iii) national surveys (DHS and MICS); and (iv) a number of studies and operational research reports on a variety of topics related to the project area. However, the functionality of the established institutions at central, regional, and district levels and the use of the information for project and sectoral management were not adequate. 2.4 Safeguard and Fiduciary Compliance 57. Environment. The environment category of the project at the time of appraisal was B. The National Medical Waste Management Plan (MWMP), adopted in May 2002 in the context of the Multi-Sectoral HIV/AIDS Project, identified a number of environmental risks, including: (i) inappropriate handling and disposal of medical waste by untrained staff; and (ii) inadequate management of the disposal sites. It proposed measures to mitigate the potential risks with adequate institutional arrangements to monitor proper medical waste management in the country; and it included: (i) an implementation schedule; and (ii) estimated costs for equipment, public awareness raising, and capacity building to effectively implement the plan. The report was reviewed, found satisfactory; and disclosed in country and in the Bank Info shop. 58. The project intended only to renovate some health posts and was therefore not expected to have substantial adverse environmental effects. Under the institutional strengthening component, responsibilities for implementing the MWMP were assigned to: (i) the Directorate of Public Hygiene (DNPH) at central level; and (ii) the heads of the hospitals, health centers and health posts for the decentralized levels. A revised action plan was requested for December 2007 and approved in January 2008. After project restructuring, a further revision of the MWMP was requested in 2011 and (after some delay) disclosed publicly in Guinea and in the Bank Info shop. 59. Prior to restructuring, the project financed (i) incinerators for the district hospitals; (ii) safety equipment for the health facility personnel at district and health center levels; (iii) training for hospital staff; and (iv) public awareness campaigns. After restructuring, the DNPH was particularly active: (i) formulating a national strategy and regional plans for bio-medical waste management; (ii) disseminating norms and procedures; (iii) installing the incinerators and distributing equipment and materials; and (iv) training and supervising staff. The ICR mission to the project area confirmed health staff’s awareness and implementation of appropriate waste management practices. 60. Except for the period during which the revised MWMP was being prepared, the overall safeguards ratings were Satisfactory both before and after restructuring. 61. Procurement, Disbursement, and Financial Management. The DAAF, assisted by the FMA, was expected to be responsible for procurement and financial management. The FMA was to 17 Annual reviews were envisioned by the DCA but were held only in 2007. Task Force meetings at the national level were rare; they seem to have been more frequent at regional level, but there is no record of the nature of the discussions. 21 be evaluated annually and judged in part on its success in ensuring the transfer of knowledge, skills, and methods to the DAAF. This transfer was never fully implemented, and the FMA continued to play the major role in fiduciary matters throughout project implementation. 62. Procurement. The PAD’s assessment of procurement capabilities concluded that there were weaknesses in each phase of the procurement process: bid preparation and evaluation, management of the contract awards, payments to contractors, and record keeping. At project start-up, the FMA had been recruited and a procurement plan prepared (though with weaknesses) for the initial eighteen months. Subsequent supervision missions repeatedly cited: (i) the need for timely updating of the procurement plan; (ii) the lengthy delays in conducting the tenders and in signing the contracts; (iii) the need to strengthen document preparation, handling, and archiving; and (ii) the importance of improving the capabilities of the procurement staff within the DAAF. 63. The project used several approaches for procurement: (i) tenders for minor renovations of health centers were organized through the Regional Development Councils (CRD) which had gained experience with a concurrent Bank project; (ii) bidding for large scale goods was the responsibility of the FMA while small-scale procurements were left to the regions; (iii) procurement of drugs and medical supplies was subcontracted to the Central Pharmacy; (iv) consultant contracts were handled by the FMA; and (iv) procurement of consumable materials was left to the decentralized health structures through the funds included in the Annual Operating Plan (PAO). 64. During the period of project suspension, procurement was rated Unsatisfactory; otherwise, procurement was rated Satisfactory both before and after restructuring. 65. Financial management. The FM assessment concluded that: (i) the minimum financial management tools were in place at the regional, district, and health facility levels; and (ii) the FMA would ensure the FM responsibilities under the supervision of the DAAF. At the time of project start-up, the basic elements of a financial management system were in place: (i) the accounting/financial management manual had been approved 18 and simplified guides prepared for the decentralized structures; (ii) the FMA had been recruited (though MOH counterpart staff were not functional); (iii) accompanying computer hardware and software (TOMPRO) were functional; and (iv) agreement had been reached on procedures for the Financial Monitoring Reports (FMR). 66. There were four recurring problems with the transfer of project funds to the implementing units at central, regional, district, and facility (hospitals, and health centers) levels: (i) the timely approval of annual plans and budgets; (ii) the availability of funds in the appropriate budget category; (iii) the difficulty of recovering justifications for the amounts spent (which reduced the funds available and the execution of planned activities; and (iv) the rejection of some of these justifications (initially by the FMA and subsequently by the Bank). 67. As previously noted, financial reports acceptable to the Bank were submitted in a timely manner. External audit arrangements were established as a condition for effectiveness, and annual audits (except for 2012) were conducted in a timely manner and generally without qualification 19. 18 The manual was subsequently revised in 2012 after the project was restructured. 19 The 2007 audit, for instance, included a qualification linked to the Bank’s modification of the financing parameters. 22 68. The quality of financial management was consistently rated Moderately Satisfactory and the risk assessed as Moderate by the FM specialists. The ISRs rated financial management as Satisfactory, except: (i) during the period of project suspension, when it was rated Unsatisfactory; and (ii) during the last ISR period, when it was rated Moderately Satisfactory due to the volume of outstanding justifications for expenditures. 2.5 Post-completion Operation/Next Phase 69. The original PAD expected that the project would: (i) contribute to the PRSP objectives by supporting MoH’s Five-Year Plan 2005-09; (ii) complement other existing Bank projects; (iii) establish planning and management capabilities at regional, district, and health facility levels; and (iv) serve as a transition to adequate budget support through a PRSC/sector approach. Though the project restructuring modified these objectives somewhat, most had by project closing been achieved in a manner that provides a basis for the future project currently being planned for FY16. 70. Specifically, policies, strategies, and protocols have been adopted in key technical areas, including MCH services, supply chain strengthening, training and human resource development, etc. Institutional arrangements have been adopted for decentralized planning and program management, and the implementing capabilities of health structures at regional, district, and facility levels were strengthened through the Annual Operational Planning process. The future project, which intends to introduce Performance-Based Financing, will benefit from these improvements. 71. Despite these positive elements, project closing will likely mean the return to a reliance on MoH’s chronically underfinanced share of the national budget impacting the improved practices (e.g., direct financing of health structures, autonomous facility management, training and supervision, timely data collection, etc.) introduced by the project. In addition, inadequate human resources remain an issue which constrains service delivery and sector management. 3. Assessment of Outcomes 72. To facilitate the assessment of outcomes, the following discussion is linked both to the summary of ratings by PDO in Table 10 (the Overall Outcome Ratings) and to the specific results of the PDO indicators and Intermediate Outcome indicators in the Section F of the Data Sheet. 3.1 Relevance of Objectives, Design and Implementation 73. Project relevance is rated Substantial for the project objectives, Modest for the project design and implementation, and Substantial overall. Prior to restructuring, the objectives were substantially in line with the existing political orientations and technical considerations, but the weaknesses in project design and implementation, noted previously, handicapped the translation of these orientations into effective actions; after restructuring, the project’s objectives, design, and implementation arrangements were significantly better aligned. 74. Relevance of the project objectives. As discussed earlier, the project’s objectives were substantially in line with the political orientations and technical considerations existing at project start-up. At project closing, the project’s objectives remain substantially relevant to the country’s global obligations and national objectives, through its support for: (i) the achievement of MDGs 4 and 5; (ii) the goals of the 2013 Poverty Reduction Strategy; and the priorities of the National 23 Health Policy and the National Health Development Plan (currently being updated). The project also remains relevant to the development of the sector through the roadmap for maternal and neonatal death (2008) and the 2011 National Gender Strategy. Finally, the project’s original principles remain relevant: (i) empowerment of MoH services for the implementation of the project, (ii) complementary funding from the project and the national budget; and (ii) reliance on enhanced local capacity to manage the implementation of project initiatives and participate in the proposed studies and evaluations. 75. The project also contributes to the achievement of the 2014-17 Country Partnership Strategy which seeks to balance fiscal austerity measures with mitigating actions in the social sector. Specifically, building on the free health policy of 2007, the government decided in 2012 to provide free pre-natal consultations and deliveries as well and in 2013 to include consultations for children under five years old. Based on evidence seen elsewhere, these policies can substantially increase the demand for these services, if the corresponding supply side needs (supported by the project) are provided, e.g., equipment, drugs, supplies, and training. 76. Relevance of the project design and implementation arrangements. The project design built on previous health project experience by focusing on: (i) vulnerable women and children in Guinea’s poorest districts; and (ii) proven essential interventions for pregnant women (risk identification and emergency care) and children (vaccination, malaria, etc.). The design also incorporated: (i) an approach for implementing quality assurance already piloted by GIZ; and (ii) a Government commitment to implement the project through agreements with decentralized health administration and health service providers. Finally, the design took into account the Guinean context for project implementation with the recruitment of the FMA and the medium-term transfer of skills to MoH. 77. While all of the above were strong elements of the initial project design, there were several shortcomings, including: (i) the ambitiousness of the PDO and the proposed targets; (ii) the number and complexity of the various interventions considered (quality assurance, compulsory health insurance, sector financing, human resource management, etc.); and (iii) the appropriateness of the proposed implementation arrangements. For these reasons, project design and implementation are considered Modest prior to restructuring. 78. Project restructuring was initiated in early 2008 to strengthen the project while maintaining the positive aspect of the design. The PDO were simplified, the project’s interventions were streamlined, the implementation arrangements (with accompanying changes in project personnel) were improved, and a performing monitoring and evaluation system was established. Though not formally completed until early 2012, the essential elements of project restructuring had been set in motion in 2011, and project design and implementation effectiveness were Substantial thereafter. 3.2 Achievement of Project Development Objectives 79. As shown in the overview below, fifteen of the project’s outcome and intermediate outcome targets were achieved (and often exceeded); seven were partially achieved; three were not achieved; and one could not be assessed due to a lack of information. Overall, the project fully or partially achieved 85 percent of its objectives. 24 Table 3: Summary of the achievement of the project's objectives Intermediate Outcomes outcomes Total Achievement No. % No. % No. % Achieved (95%+) 5 63% 10 56% 15 58% Partially achieved (36-95%) 3 38% 3 17% 6 23% Not achieved (0-36%) 0 0% 4 22% 4 15% Not measured 0 0% 1 6% 1 4% Total 8 100% 18 100% 26 100% 80. Based on: (i) the project's overall results shown in the Data Sheet, in Annexes 2a and b presenting the quantitative and qualitative outputs, and in the following paragraphs; and (ii) the project's weighted results as a proportion of actual disbursements at restructuring and at closing, as shown at the end of this section, project efficacy is rated Modest prior to the restructuring, Substantial/High after the restructuring and Substantial overall. Outcome Objective 1: Strengthening Health Care Services 81. Health care quality. To decrease maternal and infant mortality (under the original project) and improve the coverage and quality of maternal and child health services (under the restructured project), the project sought to improve the quality of care and increase the demand for health care. 82. Project inputs. As indicated in Annex 2, the project provided important inputs to strengthen health services, including minor health facility renovation, equipment, blood transfusion capabilities, drugs and vaccines, and training. Of particular importance for the quality of services were expenditures to: (i) recruit trained staff for facilities lacking midwives and nurses; (ii) strengthen communications and transport, including costs for evacuation for emergency obstetrical care; and (iii) provide comprehensive support for operational costs, implemented mainly through performance contracts between MOH, the district hospitals and the health centers. These agreements identified costs paid by central level (including equipment, vehicles, maintenance, salaries of contracted personnel) and paid at decentralized levels (training, supervision, and minor maintenance). 83. In addition to these inputs, the Government’s introduction of free health care services for pregnant women (2007) and children under five years (2012) would normally be considered as a significant contribution to results in the project area. However, as the Public Expenditure Review (PER) documents, these initiatives were never adequately funded (representing less than 2 percent of the ministry’s budget) over the period 2008-12. 20 84. Project outputs/results. As shown in Table 4 below, the project: (i) increased the coverage of health services for pregnant women and children at basic and first referral health facilities; and (ii) strengthened the technical skills (training and monitoring / post training evaluation) of clinical staff. For each of the indicators, the results show that the project achieved significant gains after the restructuring, where the actual results surpass the planned results. 20 World Bank, Public Expenditure Review (May 2014), p. 93. The PER also notes that “a large proportion of women of reproductive age still do not appear to be aware of the usefulness of prenatal visits.” (p. 53) 25 Table 4: Planned and actual results for strengthening health facilities Targets Results Orig. Restruc. Original Restructured Key performance indicator 2006-11 2011-13 2007 2008 2009 2010 2011 2012 2013 IOI 1 : No. of health centers providing standard health services for infant (IMCI) Planned (cumulative) 250 150 50 150 250 250 50 100 150 Actual 67 79 94 121 138 148 213 IOI 2 : No. of health centers providing basic emergency obstetric services (SOUB) Planned (annual) 200 175 50 100 200 200 120 150 175 Actual 0 135 135 135 167 208 213 IOI 3 : No. of hospitals and improved health centers providing full emergency obstetric services (SOUC) Planned (annual) 20 20 3 6 12 19 12 15 20 Actual 0 12 12 12 20 22 22 IOI 15 : No. of health personnel receiving training Planned (cumulative) 844 700 150 400 700 Actual nd nd nd nd 619 819 1 416 Source: Project data 85. Intermediate outcomes. Outcomes resulting from improved health services in the project area generally exceeded the targets. The number and proportion of women receiving at least one ante natal visit increased substantially, as did the proportion of institutional deliveries assisted by trained staff. The proportion of children receiving DTC progressed, but full immunization of children 12-23 months stagnated. Table 5: Planned and actual service delivery outcomes Targets Results Orig. Restruc. Original Restructured Key performance indicator 2006-11 2011-13 2007 2008 2009 2010 2011 2012 2013 OI 6 : No. of pregnant women receiving at least 1 ANC visit to a health provider ('000) Planned (annual) 198.6 267.8 181.2 186.8 192.6 198.6 204.7 259.7 267.8 Actual 94 125 125 133 194 236 249 OI 7: % of pregnant women receiving at least 1 ANC 90% 52% 67% 65% 67% 95% 91% 93% IOI 3 : Assisted delivery increased 65% 22% 37% OI 3 : Institutional deliveries assisted by trained 25% 11% 23% 19% 22% 24% 35% 38% OI 4: % of children 0-11 months immunized (DTC3) 54% 61% 80% 68% 66% 90% 84% 86% OI 5 : % of children 12-23 mos. old fully immunized 41% 38% 0% 0% 0% 43% 37% 37% Source: National HMIS. Immunization data for children 12-23 months are not available for the period 2008-10. 86. As shown below, results for several intermediate outcome indicators were not as positive. Particularly disappointing (in view of the Government’s decision to eliminate fees) were the rates for caesarian sections. While rates rose during the project from 0.60 percent to 1.54 percent, they did not reach the anticipated minimum rate of 5 percent or even the revised rate of 2 percent. 21 87. Treatment of HIV positive pregnant women was stymied by the project’s inability to purchase ARVs prior to project closing. The proportion of pregnant women benefiting from intermittent 21 The Ministry’s evaluation of emergency obstetric care emphasized this result and called for further research to determine the reasons. See Ministère de la Santé Public et de l’Hygiène, Evaluation des Besoins en Soins Obstétricaux et Néonatals d’urgence en République de Guinée (juillet 2013), p. 54. 26 preventive treatment for malaria (TPI) varied throughout the project. The number of LLINs distributed was less than originally planned, due in part to the contribution by other partners and in part to the fact that the project did not use mass distribution but rather targeted the distribution of nets to pregnant women appearing for ANC and to infants on the occasion of various vaccinations. Table 6: Planned and actual results for other selected health interventions Targets Results Orig. Restruc. Original Restructured Key performance indicator 2006-11 2011-13 2007 2008 2009 2010 2011 2012 2013 IOI 4: % of caesarian sections increased Planned (annual) 5.00% 2.00% Actual 0.60% 1.30% 1.49% 1.30% 1.30% 1.57% 1.54% IOI 5: % of pregnant women benefiting from TPI Planned (annual) 80% 80% Actual 57% 67% 42% 47% 58% 85% 67% IOI 19: No. of LLINs purchased and/or distributed ('000) Planned (cumulative) 795 190 455 795 Actual 115 234 Source: Project data 88. Outcomes. Based on data available from the 2004 and 2012 DHS, the ICR estimated the number of deaths avoided for pregnant women at 15,596 (or 130 percent of the target of avoiding 12,000 deaths) and children under 1 at 14,542 deaths (or 53 percent of the target of avoiding 27,500 deaths). 22 The project substantially achieved its target of 3 million beneficiaries, reaching 2.6 million persons (85 percent of the target), of which 1.4 million were female. 89. Finally, as indicated in the following table, which compares annual project results with periodic national results (as measured by the DHS and the MICS), the project zones lagged behind the country as a whole through 2010, but have made substantial progress on certain key indicators since project restructuring, generally catching up or surpassing the national averages. 23 Table 7: Comparison of results in the project zones with national survey results DHS Proj. Proj. Proj. MICS Proj. Proj. Proj. Proj. DHS Proj. 2005 2006 2007 2008 2008 2009 2010 2011 2012 2012 2013 % of pregnant women receiving at 82.1% 67.0% 52.0% 67.0% 88.4% 65.0% 67.0% 95.0% 91.0% 85.2% 93.0% least 1 ANC visit to a health Institutional deliveries assisted by 38.0% 14.0% 11.0% 23.0% 46.1% 19.0% 22.0% 24.0% 35.0% 40.3% 38.0% trained health personnel Proportion of caesarian sections 1.7% 0.73% 0.60% 1.30% 2.4% 1.49% 1.30% 1.30% 1.57% 2.40% 1.54% Children 0-11 months immunized 48.9% 78.0% 61.0% 80.0% 43.1% 68.0% 66.0% 90.0% 84.0% 47.2% 86.0% Children 12-23 mos. old fully 37.2% 38.0% 38.2% 0.0% 38.2% 0.0% 0.0% 43.0% 37.0% 36.5% 37.0% immunized (Penta3+R) 22 As noted in the data sheet, the initial baseline data for both indicators were incorrect, and the targets were very ambitious (at or below the targets set for the MDGs). 23 For a full review of the national averages of key outcomes indicators also see World Bank, Public Expenditure Review (May 2014). 27 90. Community mobilization. To mobilize community support for and utilization of the improved health services, the project relied on three quite different approaches. 91. Project inputs: As indicated in Annex 2, the project financed: (i) pilot activities in six regions to test the feasibility of decentralizing management of selected health facilities to the urban and rural communes; (ii) community sub-projects for village-level awareness of MCH-related activities; and (iii) the expansion of the existing community health insurance system providing coverage mainly for women’s health risks related to child birth. 92. Project outputs/results. Early studies concluded that compulsory health insurance and the decentralization of health facility management to the urban and rural municipal governments (IOI 7) were not feasible and they were dropped. 93. Subprojects (IOI 12 and 13), implemented by community-based organizations to promote the benefits of preventive health services, were financed before the project suspension but not restarted after restructuring. The impact of the sub-projects was evaluated in 2012 and showed that: (i) knowledge of the interventions had increased substantially in the selected areas (from 30 percent to 90 percent for prenatal care, from 50 percent to 80 percent for vaccination, and from 9 percent to 78 percent for family planning); and (ii) utilization (of a third prenatal consultation, of the DTC3, and modern family planning methods) lagged far behind the level of knowledge. 94. Studies of community-organized health insurance (IOI 6) were promising, and promotion of the approach was initiated early; although interrupted by the suspension, it was restarted after project restructuring and at project closing, 53 (of a planned 89) schemes had been initiated. A preliminary progress report for the community health schemes was prepared in December 2013 and showed that 7,232 households had subscribed (covering 17,708 beneficiaries or approximately 0.3 percent of the population in the project area) and that 91.2 million FGN had been mobilized. It should be noted that while these are very small numbers, they were achieved during the last five months before the end of the project. 95. Quality assurance. Beginning in 2002, German Technical Assistance (then GTZ, now GIZ) organized, in collaboration with the ministry and the Faculty of Medicine, an annual review of the quality of health care services (“concours”) provided by hospitals and health centers in selected regions. Although some of the regions in the project zone participated in the exercise, none were targeted by the GIZ project; it was therefore anticipated that the project would finance GIZ to expand the assessment of quality for the health facilities. 96. Project inputs. A workshop to introduce the ideas was financed by the project in 2008, but the contract with GIZ was never finalized. After restructuring, workshops were held in 2011 and 2012 to refine the quality assessment tool, but the tool itself was never formally introduced. The project financed the introduction of training (in-school and continuing) and follow-up supervision to measure the retention of training skills provided for emergency obstetric care and for the treatment of childhood illnesses. 97. Project outputs/Results. Self-evaluation exercises (IOI 18) were organized in eight of the project regions in 2003, expanding to nine in 2007, and eleven in 2009; cumulatively, the number of participating facilities increased from 56 in 2003 to 95 in 2007 to 116 in 2009. The mission was unable to find the results of these exercises. The sub-component was dropped at restructuring. 28 98. The quality of obstetric care was measured at the national level through evaluations led by the UN Fund for Population Activities (UNFPA) in 2003 and 2012. 24 The comparison of results was mixed: (i) the completeness and coverage of the services offered had not materially improved; but (ii) all of the other indicators (treatment of obstetrical complications, caesarian sections, and maternal death rates) improved. The quality of treatment of childhood illnesses was measured during the supervision of a sample of health centers in the project zone, which found that 80 percent of the facilities were evaluating and treating/referring cases correctly. Outcome Objective 2: Strengthening institutional capacity 99. Under the original project, the project’s intermediate objectives were intended to prepare the ministry (through support for and monitoring of the PNDS) to meet the criteria for graduating to a PRSC approach for financing the sector. Under the restructured credit, the objectives were limited to strengthening specific sectoral functions, including planning and financial management (through the DAF, supported by the FMA), monitoring and evaluation, and project coordination (through the PMU). Support for waste management and training and research (through the reproductive health training and research center) were continued. 100. Project inputs. As shown in Annex 2, the project financed personnel and material for coordinating project implementation and activities in four important areas: (i) formulation of broad policies (including the revision of the National Health Development Plan) and specific strategies; (ii) development of the health management information system and support for the 2012 DHS; (iii) strengthening of the planning and management of human resources; and (iv) contributing to the national implementation of measures to control medical waste. 101. Project outputs/results. Prior to project restructuring, the results framework was programmatic in nature, and set seven conditions as targets for the transition to budget support 25. Of these, three were monitored through the key performance indicators: (i) annual monitoring of a complete five- year plan (IOI 9&10) was not carried out; (ii) health expenditure tracking surveys to verify the budgeted amounts actually reaching health centers (IOI 11) were not conducted (although three studies, including a public expenditure review, expenditure tracking survey and beneficiary survey, were done after restructuring); and (iii) MOH budget execution of more than 70 percent (IOI 16) was achieved, as results from the PER show that, except for 2011 (65.1 percent) all of the annual results from 2006-2012 were greater than 70 percent. 102. After project restructuring, the results framework emphasized project oriented achievements and showed that: (i) all of the health centers and district hospitals were supervised (IOI 8) at least twice annually; and (ii) the proportion of the annual project budgeted executed (IOI 20) rose to 98 percent in 2013. Finally, as noted previously, 17 incinerators were purchased and installed (IOI 14) by the project (one was purchased and installed by WHO); and project records show that 14 were installed by the end of 2010. 24 Ministère de la Santé Public et de l’Hygiène, Evaluation des Besoins en Soins Obstétricaux et Néonatals d’urgence, p. 54. 25 See PAD, p. 21. 29 103. Overall achievement result. Based on the restructured project's revised indicators and the ICR guidelines (requiring separate outcome ratings weighted in proportion to the share of actual credit disbursements made in the periods before and after formal restructuring), the following table assesses the project's efficacy as modest prior to the restructuring, substantial/high after the restructuring, and Substantial overall. 26 Table 8: Combined overall project achievement ratings / Efficacy Against Against Original Restructured Considerations PDOs PDOs Overall 1 Rating value 1.85 3.50 2 Amount disbursed 12.58 10.95 23.53 Weight (% disbursed 3 before/after PDO change) 53% 47% 100% 4 Weighted value (1 x 3) 0.99 1.63 2.62 5 Final rating (rounded) Substantial Rating values: ICR worksheet available from the TTL. Methodology: OPCS, ICRR Guidelines (rev. June 2007), Appendix B, pp. 42ff. 104. As indicated in the table, the restructuring and extension of the project contributed significantly to the achievement of the PDO. In addition, a number of the project’s achievements are not captured in the results reported above, including: (i) the participation of health workers, NGOs and members of the community in raising awareness about preventive care and the increased availability of improved services; (ii) improving the functioning and contractual quality of staff at the health centers and district hospitals; (iii) the establishment of a referral and counter referral system (with ambulances and radios); (iv) support for the recapitalization of the Central Pharmacy (PCG); (v) contribution to the management of human resources; (vi) the implementation measures for medical waste management (including with incinerators in hospitals); and (vii) strengthening the coordinating bodies of MOH. 3.3 Efficiency 105. As discussed in the PER (2014), the project was implemented within a sectoral context of poor allocative and technical efficiency. First, public funds were concentrated at: (i) central level and in Conakry rather than in rural areas; and (ii) referral hospital level rather than primary health center level. This was due mostly to the disconnect between the needs for and deployment of health personnel and services. Second, as seen earlier, spending priorities did not give sufficient emphasis to basic public health programs with high potential effectiveness and positive externalities; of the eighteen priority public health programs operating in the country, the PER examined five which were large enough to be traceable: Expanded Program on Immunization (EPI), tuberculosis prevention pro-gram (TB), malaria prevention program (PALU), program against AIDS (AIDS) and Comprehensive Care for Diseases of Newborns and Children (PECIMNE). In 2012, these programs 26 The worksheet detailing the calculations leading to these results is available from the Project Files. 30 received only 2.7 percent of total ministry expenditures. 27 106. Further, the PER noted the inefficiencies of budget execution, resulting from the low levels of budget execution for personnel and the consequence that, despite budget availability, less staff is on the ground than planned with deleterious effects on the quality and coverage of health care. 107. Allocative and technical efficiency. Annex 3 of the PAD anticipated these issues in its economic and financial analysis and proposed strategies to: (i) improve allocative efficiency (to needier regions and more vulnerable populations); (ii) implement a cost-effective package of priority services; and (iii) increase technical efficiency in the delivery of health services (through outreach, follow-up of vaccination, etc.). 108. In contrast to the sector’s demonstrated inefficiencies, the project’s design promoted both allocative and technical efficiencies. The project’s allocative efficiency was substantial both before and after the restructuring, based on: (i) the level of poverty in the targeted areas and the accompanying inequitable access and quality of health services; and (ii) the focus on mothers and children (who bear a disproportionate burden of morbidity and mortality). Similarly, the project’s design favored technical efficiency by financing proven interventions to improve maternal and child health services and control communicable diseases (including vaccination coverage). 28 Technical efficiency is rated modest prior to the restructuring, based on the scope of the project’s activities, a number of which were financed once or twice and then dropped. Though potentially important for long-term reform, these activities tended to compete for attention and resources from the essential objectives of improving the coverage and quality of maternal and child health services. After the restructuring, the project’s activities were more focused and technical efficiency was substantial. 109. Managerial efficiency. Prior to the restructuring, the implementation arrangements, which sought to combine internal government structures and methods with an external fiduciary management agency’s procedures created delays in procuring and delivering goods and services to the service delivery point and in disbursing the project’s funds. In addition, the late effectiveness (2006), the slow start spending (2007), the lack of timely availability of counterpart funding, and the suspension of the project (2009-11) all detracted from the project’s efficiency. With difficulty, the project coped with the problem of coordinating the timing, nature, and modalities of the resources required to achieve the anticipated results. After project restructuring, as can be seen in the volume of activities achieved over the final two years of the project, the revised implementation arrangements contributed significantly to managerial efficiency. More than 50 percent of project funds were disbursed during calendar years 2011-13. 110. Efficiency before and after project restructuring is rated as follows: (i) allocative (substantial- substantial); (ii) technical (modest-substantial); and (iii) managerial (modest-substantial). Overall, based on these dimensions of efficiency, the project’s efficiency is rated as Modest. 27 PER, pp. 96. The PER indicates that public funding for these programmatic areas (which is less than GNF 3 billion for all eighteen programs and less than about 1 billion for the five priority areas) is almost irrelevant in comparison to the needs of the country. In 2012, the total cost of running these programs was GNF 180 billion, all of which was funded by the international community. 28 The package of interventions implemented by the project were known to be cost-effective at US$82-142 per DALY. 31 3.4 Justification of Overall Outcome Rating 111. Based on considerations of the various ratings criteria and indicators and disbursement prior to and after project restructuring, the following table presents the overall outcome rating for the project as Moderately Satisfactory. Table 9: Summary of project ratings Original Restructured project project Rating criteria 2006-10 2011-13 Overall Relevance Substantial Substantial Substantial Objectives Substantial Substantial Design/Implementation Modest Substantial Efficacy Modest Substantial/ Substantial High* Efficiency Modest Substantial Modest Overall MU S MS *Table 8 attributes an efficacy score of 3.5 to the restructured project. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 112. The project contributed to the government’s poverty reduction strategy and, based on the national poverty map established with Bank support, explicitly targeted the country’s poorest regions. The PDOs were articulated around two groups known to be the most vulnerable: pregnant women and children under the age of five. During the last year of the project, an estimated 1.4 million women were direct beneficiaries of the project. As previously indicated in Table 7, the proportion of pregnant women benefiting from antenatal care and institutional deliveries (with assistance from trained health personnel) increased steadily and substantially in the project areas while remaining more or less static for the country as a whole. 113. According to the PER (2014), Guinea established two pro-poor programs, the Indigents Fund (2005) and the Caesarian Fund expanded to include children under five (2007, 2012), which were established to support the free delivery of certain health services to the poorest Guineans. The project supported these poverty oriented initiatives indirectly by providing goods and services to the health facilities rather than to the funds themselves. 114. Finally, the project contributed to the establishment of more than 50 community health financing mechanisms (mutuelles de santé) which could ensure the continuation of at least some of the services strengthened under the project. As described to the ICR mission by several respondents, despite a long history in Guinea of attempts to support them (and the existence of a legal code), these mutuelles must still overcome a number of technical difficulties and a persistent lack of knowledge and distrust on the part of the populations, which have handicapped their expansion. 32 (b) Institutional Change/Strengthening 115. Though the project did not achieve its very ambitious sector reform agenda (including the steps for graduation to a budget support approach), this was due in large measure to constraints beyond the control of the health sector. The project did achieve other, more narrowly defined sector reforms, which can be categorized in three groups: (i) sustained technical (safe motherhood, blood transfusion, IMCI, medical waste management, etc.) and administrative support (decentralization, annual planning, pharmaceutical management, etc.); (ii) periodic interventions which have not yet been fully embraced (community health insurance, human resource planning and management, public expenditure tracing, etc.); and (iii) pilot experiences which were implemented but were not deemed feasible (national health insurance, community ownership of health facilities, quality assurance, CRD sub-contracting, etc.). 116. Within the sector, (i) task forces have been established at regional and district levels to plan, implement, and evaluate activities, and community participation for health center management has been reinforced; (ii) performance contracting has enhanced the autonomy of all levels of the health system with the accompanying strengthening of planning and budgeting methods, financial accounting, and reporting practices; (iii) health information systems and M&E have been updated (and annual statistical reports brought up to date); and (iv) tools for assessing the technical competence of health workers and the effects of training have been introduced. 117. One of the project’s major contributions was in establishing expectations for the performance of the health system. Health staff and beneficiaries saw that, with adequate resources and support, the health system in Guinea could function adequately. For health service providers, the introduction of the performance contract provided a mechanism for planning results, projecting costs, receiving funding directly into facility bank accounts, and disbursing for pre-approved expenses. 29 Facility autonomy and results were enhanced. For beneficiaries, pregnant women, for example, received and came to expect: (i) free transfer of pregnant women to the district hospital in case of need (telephone communication, ambulance, etc.); and (ii) the availability of the requisite services at the district hospital (surgical care, blood, etc.). Similarly the availability of drugs and vaccines 30 ; periodic supervision at all levels, outreach with community health workers, etc. all represented significant improvements in the functioning of the health care system. (c) Other Unintended Outcomes and Impacts (positive or negative) 118. From a positive perspective, the contract signed with the Central Pharmacy (PCG) to procure and distribute drugs had the intended effects of increasing the availability and improving the management of drugs and other medical supplies at health facility level, but it also had the somewhat unintended positive effect of providing a means to recapitalize the PCG without providing a direct project subsidy. 119. From a negative perspective, while in almost all countries where caesarian sections are 29 The Ministry of Finance approved these measures on an exceptional basis (par dérogation) in large part because of the presence of the FMA. 30 The ICR mission to the field confirmed both the availability of drugs and supplies and the use of proper stock management techniques through the assistance of the Central Pharmacy (PCG). 33 provided free, coverage has increased, this was not the case in Guinea either at the national level or in the project area. The 2012 evaluation of emergency obstetric services did not provide an explanation, but it may well be due to the Government’s inability to finance the declared policy, with the result that the policy was not implemented by the health services. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 120. Beneficiary assessments were conducted in 2002 as part of the project preparation and in 2013 at project closing. The ICR mission was unable to obtain access to the initial beneficiary assessment; the latest beneficiary assessment is summarized in Annex 7. 4. Assessment of Risk to Development Outcome Rating: High 121. While durable progress has been achieved and should continue in many areas as indicated above, three types of risk should be noted. First, a chronic problem of leadership exists within the ministry of health as demonstrated by the number of reform initiatives which were not brought to a satisfactory conclusion. Several promising pilots were simply abandoned (e.g., quality assurance, community awareness sub-projects, national insurance, etc.) while other proven successes (e.g., emergency obstetric services) are likely to disappear without additional support. 122. Second, in the short term, insufficient funding from the national budget since project closing has already had negative effects on: (i) the stability of contractual staff of which an estimated 70% have left their positions); and (ii) the functionality of services for which operational expenses (including inputs, regular supervision, etc.) are no longer adequate. In the longer term, insufficient numbers of qualified staff combined with a lack of motivation and rapid turnover will lead to a declining interest in the methods and tools introduced by the project. 123. Third, the participation (through performance contracting and direct financing) of regional, district, and health facility officials in the implementation of annual operational plans developed the tools and methods for effective decentralization of health service delivery. Without these key elements, continuation of this participation will be jeopardized. 124. Finally, given the potential for continuing political instability, the fragility of the economy, and the uncertainty over the future leadership and direction of health service delivery in Guinea, the risk to the development outcomes is considered High. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately unsatisfactory 125. Despite the multiple changes in the overall project framework for the delivery of services, the project preparation process consistently focused on certain key principles, including the poorest geographical areas, the most vulnerable population groups, and the most effective interventions. It 34 also sought to build on the Government’s commitment to decentralization, demonstrated in other successful Bank operations (e.g., the PRCI, the PACV, and the Urban Project), and focused on enhancing the autonomy of health administration and health service providers to implement the project. At the same time, the project design did not ignore the constraints of the Guinean context for project implementation and included the recruitment of the FMA as a condition of effectiveness. 126. Unfortunately, the PAD did not entirely reconcile these different elements with the result that the project combined many ideas and specific activities for strengthening MCH services with institutional arrangements which were not entirely suited to the implementation of these initiatives. There were delays in project preparation due to the project’s design: (i) the conditions imposed for effectiveness; (ii) misunderstandings between the DAAF and the FMA which handicapped their cooperation for the duration of the project; and (iii) the breadth of the planned mix of program and project activities expected to be carried out by MOH’s technical services. 127. There were other weaknesses as well. First, while a number of the background analyses (at least for the PPF) proved to be relevant, they were generally not available until long after project effectiveness and therefore did not influence the project design. Second, a better assessment of the potential risks might have anticipated some of these difficulties. Third, the project design’s internal inconsistencies were reflected in the results framework, which established very ambitious targets for indicators which were neither easy to measure nor particularly relevant to the PDO. (b) Quality of Supervision Rating: Satisfactory 128. Though supervision was hampered by the recurring restrictions on travel to and within the country, it was enhanced by the continuity of the TTL, the thorough treatment of the essential issues (technical, legal, fiduciary) by the Task Team members, and the systematic review of actions taken on recommendations of the previous missions. With a couple of exceptions, the supervision mission aides-mémoires were comprehensive, and the implementation status reports (ISR) were candid. 129. Supervision was characterized by flexibility and proactivity as measures were planned early in the project to address: (i) weaknesses in the institutional arrangements for project implementation; (ii) inadequate health personnel in the project area; and (iii) insufficient data to monitor the project’s progress. Project restructuring, initially planned after the mid-term review, was accelerated when it became apparent that waiting would have a negative effect on project performance. Restructuring was relatively advanced when the Bank suspended activities in Guinea. 130. After restructuring, each of the issues above was adequately addressed. Mission assistance was particularly valuable in ensuring the availability of statistical data for the country as a whole and for monitoring the project. Although a formal exit strategy was not formally adopted, planning for a follow-on project was initiated prior to project closing. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 131. Following ICR guidance indicating that, when ratings for the two dimensions are in different 35 ranges (moderately unsatisfactory for quality at entry and satisfactory for supervision), the rating for overall Bank performance depends on the outcome rating and is therefore rated Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 132. Socio-economic instability and political conflict preoccupied authorities during project implementation and, in combination with frequent ministerial changes and shortcomings in the institutional arrangements, weakened political and technical leadership for the project within the Ministry. First, while the basic policies and directives for the sector were established, the daily operations to monitor implementation were ineffective: the steering committee functioned poorly, the reform initiatives were not followed up, and the lack of central-level support was felt particularly at regional and district levels. Second, though the project demonstrated that satisfactory results can be obtained with a certain level of financial resources and autonomy (to recruit staff locally, to spend money on specific inputs, etc.), the Government’s long-term underfinancing of the sector (as shown by the latest PER) has seriously undermined sector performance. 31 (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 133. Project Coordination Unit. The PCU had responsibility for the overall implementation of the project; specific project implementation responsibilities were shared with the appropriate Ministry services for technical matters and with the FMA and the DAAF for financial and administrative matters. The ISRs consistently rated project management as satisfactory or moderately satisfactory 32. Assessment of implementation progress varied considerably, from moderately unsatisfactory over the period 2006-07, to moderately satisfactory for 2008 and then for the entire period of the restructured project. 134. As noted previously, the PCU’s performance was handicapped by the original implementation arrangements, but the modification of these arrangements, the recruitment of limited (but dedicated and competent) staff; and the enthusiasm of central and decentralized MOH personnel contributed to the project’s achievement of many of its expected results. 135. Fiduciary Management Agency. The FMA succeeded in its principal task of ensuring the fiduciary responsibilities of the country in a transparent manner as described in the DCA. Although there were problems cited with respect to the timeliness and justification of the FMA’s actions and to the limited number of its supervision missions 33, the ISRs consistently rated FM as satisfactory. 31 The Government’s counterpart contribution was paid in full, but not in a timely manner. 32 Except for the latter half of 2007 and the suspension period. 33 For instance, in 2008, the decentralized structures received only one of two planned advances because the FMA had not conducted the supervision missions to confirm the expenditures incurred. 36 The FMA also succeeded in strengthening financial management capabilities at regional and district levels. The FMA did not succeed in the planned transfer of fiduciary capabilities to the appropriate financial and administrative services within MOH. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 136. Following ICR guidance indicating that, when ratings for the two dimensions are in different ranges (moderately unsatisfactory for government performance and moderately satisfactory for the implementing agency performance), the rating for overall Borrower performance depends on the outcome rating and is therefore rated Moderately Satisfactory. 6. Lessons Learned 137. Project design must balance a range of considerations. These include the desired outcomes, the financing instrument, and the implementation arrangements. While certain instruments (such as APLs or sector support) may offer an innovative approach for addressing identified weaknesses in a traditional project approach, their selection may raise other, more significant problems. Therefore, project design should be simple and focused on the Government’s priorities and commitment and not on the latest development trend. For the project, perhaps too much attention was focused on the financing instrument (and the resulting institutional arrangements) and not enough on a more complete understanding of the previous results and the existing socio-political situation. 138. Risk assessment is an essential aspect of project design. While overall risk was rated as high, the components of this rating were underestimated and the mitigation measures were insufficient. A more rigorous assessment might have concluded that: (i) significant risks (political, socio-cultural, technical, and financial) existed; and (ii) the feasibility of completing the project as designed was questionable. While project preparation cannot be expected to pre-identify all of the potential problems and their solutions, the proposed mitigation measures for the operation should have been justified and not just asserted. 139. Where ministry leadership is inadequate, other champions must be found to advance the project’s objectives. Guinea offers an example of where progress was achieved in the context of political instability and insufficient leadership. Significant advances were made where champions initiated service improvements (e.g. prevention of childhood illness, bio-medical waste, blood transfusion, etc.). Project support from these champions was able to compensate (at least in part) for varying government commitment and weak capacity. Consequently, proactive project supervision (and effective management) should seek to identify opportunities and encourage them where possible. 140. The Bank's comparative advantages need to be recognized and exploited. Guinea demonstrates several of the Bank’s comparative advantages, which will be very important for the proposed follow-on project to introduce Results-Based Financing (RBF), namely its flexibility and willingness to support innovative strategies, and in particular to: (i) adopt measures to strengthen health systems and health personnel; (ii) support decentralization of services and program coordination; and (iii) promote initiatives to improve services, experiment with sectoral reforms, and improve health care financing (e.g., better transparency, community health insurance, etc.). In 37 these efforts, the Bank's non-financial support was often as essential as its financial support for persuading all stakeholders in the sector. 141. There is a need to temper expectations that a single health project can impact the entire health sector. Even good projects cannot necessarily change the sector context. While a single project financed by the Bank may be a necessary condition for sectoral change, it is not sufficient without greater political leadership and financial commitment from the Guinean authorities. Inadequate sector financing and shortage and distribution of health personnel remain an issue in Guinea that can only be tackled with increased Government spending for the sector. This is all the more important in the context of the introduction of an RBF approach to strengthen the sector’s focus on results towards improved availability and quality of health services. 142. Monitoring and evaluation can be improved throughout the project. Though the general prescription is that M&E systems, including arrangements for collecting and analyzing data, should be established prior to project launch, Guinea shows that projects can recover from inadequate M&E design and early implementation weaknesses. While the timing of the DHS (in 2005 and 2012) was fortuitous for evaluating the project, the project’s sustained efforts to establish the annual HMIS reports (2007-2013) and reconstruct project data for the entire period were essential in providing an evidentiary trail for the project. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 143. (a) Borrower/implementing agencies: 144. The task team and the Borrower reviewed and agreed on the results of the indicators reported in the Data Sheet. 34 The Borrower has prepared a comprehensive final evaluation report in French; as it did not contain a summary, one was prepared and is presented in Annex 7 in English. The Borrower’s final evaluation report is available from the Project Files. 145. (b) Co-financing: Not applicable (c) Other partners and stakeholders: Not applicable 34 A technical note was prepared at the completion of the ICR mission and is available in the project file. 38 Annex 1: Project Costs and Financing (a) Project Cost by Component Appraisal Estimate Additional Actual/Latest Percentage of Components Original credit financing Estimate Appraisal (US$ millions) (US$ millions) (US$ millions) * Component 1 18.99 17.62 93% Component 2 8.37 5.91 71% PPF 0.43 Total Project Cost 27.79 23.53 85% Sources: Annex 2 of PAD and Client Connection. *At project restructuring, fluctuations in the SDR to dollar exchange rate had increased the original credit to US$27.8 million; the reduction of US$1.8 million in the estimated project cost resulted from dropping the subcomponents on quality assurance, development of national insurance scheme, and tracking survey of health expenditures. (b) Financing Appraisal Actual/Latest Percentage Type of Co Source of Funds Estimate Estimate * of financing (US$ millions) (US$ millions) Appraisal International Development Association 25.00 23.53 94% (IDA) Borrower 2.80 2.80 100% Local Communities Total Project Financing 27.80 26.33 95% * Increases in total project financing also reflect exchange rate fluctuation in the SDR to dollar exchange rate over the project life. 39 Annex 2: Summary of the main physical results of the Project Component 1: Strengthening of health services Quality of Care Strengthening health services 2007-13 • Financing of health center (CS) operating costs (management tools, fuel for motorcycles, refrigerators, repair costs for motorcycles; financial motivation for staff. etc.) 2007 • Technical Evaluation of the renovation of 25 CS health centers for 2008 • Acquisition of 8 4x4 vehicles, ambulances, etc. • Acquisition of fifty (50 Motorcycles) • Acquisition of two hundred fifty (250) Bicycles for health stations • Distribution at: (i) health center level annual management tools comprising 100,000 immunization records; 100,000 health cards; and 90,000 antenatal care records; and (ii) hospital level 50,000 programs 2008 • Acquisition of medical furniture, electrical generators, and other equipment • Acquisition and installation on site of 47 solar-powered radios (with accessories) 2012-13 • Payment of salaries of 400 contracted workers at hospital and health center levels • Complete salary payments for community-contracted health center personnel in the project area for the full four years Strengthening drugs 2008 • Agreement with the Central Pharmacy of Guinea (PCG) for: (i) the procurement of drugs and medical supplies (bed nets, ARV, basic and hospital emergency medical equipment, drugs, etc.); and (ii) receiving, repackaging and routing pharmaceuticals and medical material to health facilities • Acquisition of medical consumables (ACIA) • Workshop to develop a simplified manual for management of drugs 2012-13 • Acquisition of drugs and medical consumables • Management tools for evaluating Hospital logistics systems • Training of 18 officers in applied training techniques for logistics management by the National Directorate of Pharmacies and Laboratories (DNPL) • Régional workshop on the management of medicines for the DRS, DPS, HP, CSA, SAF & Pharmacist • Training of 15 Pharmacy Inspectors Strengthening blood transfusion 2008 • Training / Supportive supervision for staff of 18 Hospital Blood Transfusion Units on standards of quality assurance for blood transfusion • Purchase and provision of inputs for blood transfusion (15,000 blood bags and reagents for biological tests) 40 2011 • Conception / Distribution of promotional material for donating blood for the CNTS 2012-13 • Financing of equipment for 18 Hospital Blood Transfusion Units, including electrical energy for storage of blood and reagents; • Financing of solar energy for Ratoma Community Medical Centre (CMC) blood bank • Training 40 hospital clinicians (Maternity / Pediatrics) of the project area on the rational use of blood and blood products • Training of 148 agents in safe blood transfusion of blood donors • Training of blood donor recruiters • Purchase and provision of food for blood donors • Acquisition / Installation of a solar powered blood transfusion unit Safe motherhood (monitoring risk cases, reference, blood, etc.). 2008 • Study on unsafe abortions: magnitude, causes and percentage in maternal mortality é • Study tour / Survey / Workshop on nosocomial infections in project maternities 2009 • Acquisition of SOUB and SOUC equipment (Lots 1-4) (in part) 2011 • Acquisition of equipment (biomedical, instrumentation, medical material) (Lot 2); and SOUC equipment and consumables • Training of 60 providers in organizing emergency hospitalization • Training of 92 training providers on essential obstetrical care • Reproduction of training materials for emergency neo-natal care (SONU) 2012-13 • Development / Validation / Dissemination of the national roadmap for accelerating the reduction of Maternal, Neonatal and Infanto-Juvénile mortality • Technical assistance for the installation of SOUB and SOUC equipment • Training of 122 staff in ANC / delivery with SONU s in the districts of Koubia, Dalaba, Kérouané, Kissidougou and Gaoual • Acquisition of ventouses Increase PF and IMCI 2008 • Workshop / Training / Monitoring post of 24 providers of clinical IMCI clinical in Siguiri Dalaba, Pita, Kouroussa, Kérouane, Gaoual and Koundara 2011 • Purchase of vaccines for EPI / Refund of Government commitment to UNICEF / GAVI • Training workshop for the updating prevention of infections and contraceptive technology • Training workshop for 24 participants in clinical IMCI in Koubia, Lélouma, Beyla and Mandiana • Training of 104 providers in the project zones on RH norms and standards 2012-13 • Acquisition of vaccines to immunize more than 335,000 children aged 0-11 months and 350,000 pregnant women 41 • Vaccine Procurement (U.S. $ 520,000) • Conception and production of management tools EPI, ANC and PF • Provision of 198 kerosene fueled refrigerators (SIBIR) • Production of training materials for PCIMNE • Training workshops in clinical IMCI for service providers in the project area • Post training follow up of providers trained in 2011 & 2012 in clinical IMCI • Inputs (management tools PEV / ANC / PF, equipment, material, medical consumables, and bed nets • Training of 460 community workers in integrated package of community IMCI services Malaria prevention 2007 • Participation in the seminar on the acquisition of products against malaria in Burkina Faso from 12 to 15 December 2007 2008 • Acquisition / Transport 115,000 Mosquito nets (LLINs A) in the project areas 2012-13 • Acquisition / 119,000 transport Mosquito nets (LLINs) PMTCT 2012-13 • Training / retraining of 120 providers of PMTCT services in 30 sites • Support for the management of PMTCT activities in the project area Community mobilization Support for the development of community health insurance 2007 • National forum on community health insurance (“mutuelles) 2008 • Development of the National Strategy for the Promotion of Health Mutuelles • Study on the definition and feasibility of mandatory health insurance in Guinea 2012-13 • Recruitment / Training / Supervision of operators of health mutuelles • Supervision of operators of health mutuelles in the project area Ownership of the management of health facilities by the communities 2008 • Study for the feasibility of community ownership of health centers • Workshop on the development of community action plans for local authorities and health personnel • Approval / Finalisation of feasibility study for community ownership of health facilities Support to the private sector and sub-projects to raise awareness of the risks 2008 • Workshop on public and private sector partnership • Training of trainers on management of sub-projects in Mamou in 2008 • Supervision of the implementation of the recommendations of the workshop contracting 2009 • Funding 15 subprojects 42 2011 • Workshop for 65 agents on contracting in the private sector 2012-13 • Evaluation of 12 sub-projects in the project area in 2008 PRSS (Tougué, Koubia, Dalaba, Kissidougou) • Workshop for the preparation of educational messages on IMCI, MSR and the management of biomedical waste for 30 participants Quality assurance 2008 • Workshop assessing the performance of hospitals and CMC 2011 • Workshop for 30 participants on appropriating the tool for measuring integrated quality 2012-13 • Workshop to revise the inspection instrument for the General Inspection for Health Component 2: Institutional Support Institutional strengthening Support policy development and reflections 2011 • Support for program review of MOH • Workshop for the reflection on the organization of primary health care 2012-13 • Support for the organization of workshops to prepare the evaluation of the National Health Development Plan and the Etats Généraux Support for strengthening the NHIS 2011 • Support for the preparation of statistical yearbooks 2007-10 2012-13 • Acquisition of computer equipment • Revision of HMIS tools to integrate reproductive health and child nutritional indicators • Workshop to revise and adopt indicators for monitoring health centers • Contribution to realization of the DHS 2012 Support the development of human resources 2008 • Study on the motivation of health staff in unfavorable locations (Health Focus) • Training of 5 staff of the Division Human Resources in Human Resource Management • Census of health staff outside Conakry 2011 • Workshop to review the results of the census of health staff 2008 • Development of the national strategy for the development of human resources 2012-13 • Acquisition of computer material for the computerization of the Human Resource Management System • Establishment of a database for the management health staff 43 Support for the management of bio medical waste 2008 • Training workshop on the management of bio medical waste 2011 • Training of 150 staff of public/private/mixed facilities on the treatment of bio medical waste • Dissemination of a plan to address key issues in bio medical waste management • Organization of 7 regional workshops to disseminate documents on the management of bio-waste • Edition / Dissemination of the National Policy on Public Hygiene 2012-13 • Development / Diffusion in the Official Journal of the National, Regional, and District Plans for the Management of bio medical waste • Acquisition of 17 incinerators for the districts in the Project area • Supervision of the management of bio medical waste in health facilities in the project area Increased transparency of public expenditures 2011 • Finalization of the National Health Accounts study for the year 2010 2012-13 • Public health expenditure tracking survey • Public expenditure review of the health sector (2005-2012) Support the functioning of coordination bodies (CTP, CTRS, TF, PMU, etc.). 2006-08 • Renovation of the conference room of the Project Coordination Unit (PCU) • Purchase of office furniture and computer equipment for the PCU • Acquisition of material for the PCU: electrical generator, phones, server, etc. • Purchase of televisions, screens, DVD Players and electrical generators for the central and regional-level Task Forces • Recruitment of specialists for the PCU: Administrative Assistant; Accounting; Specialist in decentralization and community health insurance; Specialist in safe motherhood; Specialist in infant and neo-natal health • Acquisition logistics and transport • Acquisition of three supervision vehicles • Four (11) all-wheel drive vehicles • Eighteen (18) all-wheel drive pick-ups • Recruitment of the Fiduciary Management Agency (AGF) • Acquisition / Installation / Training for TOMPRO project accounting system • Training in financial management for the SAAF of the DRS, DPS, hospitals, and CMC hospitals in the whole country in 2007 and 2008 • Development of financial management tools; • Training of 18 officers responsible for the management and maintenance of equipment in the Project area • Recruitment of the external auditor 2008-13 • Support for the preparation of the Annual Operating Plans (PAO) 44 2011 • Support for the meetings of the various project coordinating structures (CTC, CTRS, etc.). • Renewal of the contract of the AGF 2012-13 • Inventory of goods provided to structures by the project • Training of more than 160 staff in various subjects (service quality, contracts, public- private partnerships, community health insurance, etc.). 45 Annex 3: Economic and Financial Analysis 1. The PAD did not provide a cost-benefit analysis for the proposed project; rather it argued that in view of severe budget limitations, the project would implement strategies based on: (i) improving allocative efficiency (to needier regions and more vulnerable populations and to a more cost-effective package of essential services); (ii) increasing technical efficiency in the delivery of health services (through outreach, follow-up of vaccination, etc.); and (iii) enhancing the efficiency of current financial resources and the potential of out of pocket expenditures (whether to the public or private sectors) through better cost-sharing schemes and the introduction of community health insurance. 2. Allocative efficiency. The PAD identified mismatches between: (i) priorities in the Five- Year Plan and budget allocations; (ii) budget allocations and expenditures; and (iii) insufficient expenditures on operational and maintenance costs of the health infrastructure. The project envisioned the following actions for increasing allocative efficiency: • re-allocating resources to poorer regions with larger disease burden from urban areas; • increasing preventive outreach and revitalizing primary health care; and • improving hospital coverage; and • strengthening the capacity of the different levels to improve budgetary execution. 3. Allocative efficiency was significant throughout based on: (i) the level of poverty in the targeted areas and the accompanying inequitable access and quality of health services; and (ii) the focus on mothers and children (who bear a disproportionate burden of morbidity and mortality) and on effective interventions to improve communicable disease control (including vaccination coverage) 35. The intended emphasis on instituting a quality of care instrument did not prove to be feasible, but comprehensive training and regular supervision were financed by the project. 4. Strengthening of planning, budgeting, and budgetary execution at the different levels through the use of the Annual Operating Plans (PAO) was somewhat more problematic. While greatly appreciated by the local health authorities and the service providers as a means of financing key operational costs, the PAO, could only be expected to have a very indirect effect on results. In addition, the Government's inability to adequately finance the national health budget meant that the project substituted for (rather than complemented) key elements of the Government’s operating budget in the project area. 5. Technical efficiency. The PAD analysis does not address Government inefficiencies in health spending, namely: (i) the very low level of health spending; (ii) the delays in procuring and delivering goods and services to the service delivery point; and (iii) the known leakages between the central level, regional and district health authorities, and hospitals and health centers. Data are available on the first two points. 35 The package of interventions implemented by the project were known to be cost-effective at US$82-142 per DALY averted. 46 6. Data from the WHO National Health Accounts website show that both total health expenditure as a proportion of Gross Domestic Product and general government spending on health as a proportion of total general government spending have remained stable at 5-6% and 6-8% respectively since 2003. Data from the Survey to Monitor Public Expenditures to Destination (2012) provide information on the extensive delays in procurement and on the consequent lag in deliveries and payments. 7. From the project’s perspective, technical efficiency was modest throughout, based on the timing, the nature, and the modalities of the project’s financial support. First, as a result of procurement delays and the socio-political situation, the project had difficulty coordinating the availability of all of the resources required to achieve the anticipated results. Second, the scope of the project’s activities somewhat diverted resources from its essential objectives of improving the coverage and quality of maternal and child health services. Among these activities, a number were financed once or twice and then discontinued. 8. Enhancing fiscal space. The PAD noted that in 2001, 83.5 percent of total health spending in Guinea was from private sources, and almost 99% of private spending on health care was out-of- pocket (OOP) at the point of service. While the benefits of prepayment for services had been recognized, the existing social security system covered a very limited number of individuals in Conakry, and no such mechanism had been developed to cover populations in poor, rural areas. The project envisioned the following actions to enhance efficiency and increase fiscal space: • improving risk pooling through the community health insurance (mutuelles); • ensuring that existing public and private resources support public health goals; and • initiating a program approach to: (i) consolidate the ministry’s budget to increase the amount and avoid duplications in donor financing; and (ii) shift the ministry’s focus from project implementation progress to results. 9. As noted on the WHO website, since 2003 the share of private spending on health has declined somewhat from 99% to 93%. OOP spending has increased since 2003 from US$15 per capita to USS$21 per capita. Private insurance finally attained 1% of total private health expenditures in 2010 and has remained at that level. As with risk pooling, the coordination of the various sources of financing to achieve public health goals has only just begun to be initiated. 10. Managerial efficiency. Managerially, the implementation arrangements during the project’s initial phase, which sought to combine internal government structures and methods with an external fiduciary management agency’s procedures created disbursement problems from the beginning. In addition, the delay in implementation (2006), the slow start spending (2007), the lack of timely availability of counterpart funding, and the suspension of the project (2009-11) all contributed negatively to the project’s efficiency. Finally, as can be seen in the volume of activities achieved over the final two years of the project, the revised implementation arrangements after project restructuring contributed significantly to the rate of disbursement. More than 50 percent of the project proceeds were disbursed during calendar years 2012-2013. 47 Annex 4: Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility Lending (a) Original Credit Bella Lelouma Diallo Sr. Financial Management Specialist AFTMW Financial Management Sameena Dost Senior Counsel LEGES Legal Salimatou Drame-Bah Program Assistant LCCHT Team assistance Edeltraut Gilgan-Hunt Sr. Environmental Specialist AFTHW Safeguards Daniele Jaekel OOperations Analyst AFTHW Operations Soukeyna Kane Sr. Financial Management Specialist AFTMW Financial Management Ibrahim Magazi Task Team Leader AFTHW Health Tonia Marek Public Health Specialist AFTHW Health Myrina McCullough OOperations Analyst AFTHW Health Alexander Preker Public Health Specialist AFTHW Health Eric A. Tinkt de Roodenbeke Health Specialist AFTHW Health Hugues Agossou Financial Management Specialist AFTMW Financial Management Siaka Bakayoko Financial Management Specialist AFTMW Financial Management Françoise Brunet Consultant AFTSW Consultant AFTHW Community-led Mohsen Farza Consultant Initiatives Madefing Kaba Driver C AFMGN Transportation Mohamed Ould Ahmed Consultant C AFTHW Consultant Mohamed Ould Nezhir Consultant AFTHW Consultant Justine Agness Soubahoro Consultant AFTHW Consultant Bhanoumatee Ayoung Procurement Specialist OPSOR Procurement Ok Pannenborg Manager P AFTHW Management Moussoukoro Soukoule Program Assistant EACVQ Team assistance William Dakpo Consultant AFTHW Consultant Assiata Houedanou-Soro Procurement Assistant AFCF2 Team assistance Gnoleba Mathieu Megube Procurement specialist AFCF2 Team assistance Amadou Tidiane Toure Procurement Specialist AFTPC Procurement Ousmane Diadié Haidara Health Specialist AFTHW Health (b) Restructuring Ibrahim Magazi Task Team Leader AFTHW Health Jean Jacques de St Antoine Lead Public health Specialist AFTHW Health Vincent Turbat Sr Health specialist AFTHW Health Chris Atim Sr. Health Specialist AFTHW Health Eby Ould Cheikh Health Specialist H AFTHW Health Dominic Haazen Lead Health Policy Specialist AFTHW Health Omar Ramses Zang Sidjou Consultant AFTHW Health 48 Names Title Unit Responsibility Nicole Hamon Program Language Assistant AFTHW Operations and Team Assistance Sariette Jippe Program Assistant AFTHW Operations and Team Assistance Ibrahim Magazi Team Leader AFTHW Health Chris Atim Sr. Health Specialist AFTHW Health Eby Ould Cheikh Consultant AFTHW Consultant Dominic Haazen Lead Health Policy Specialist AFTHW Health Omar Ramses Zang Sidjou Consultant AFTHW Consultant Ousmane Diadié Haidara Sr. Health Specialist AFTHW Health Peter Bachrach Consultant AFTHW Consultant Robert A. Soeters Consultant AFTHW Health (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including travel and No. of staff weeks consultant costs) Lending FY02 4.6 20.1 FY03 0 0 FY04 32.23 206.8 FY05 30.57 105.7 Total: 67.4 332.6 Note: Breakdown by fiscal year is not available. Supervision/ICR FY06 27.14 70.4 FY07 30.76 62.1 FY08 30.06 125.9 FY09 14.61 45.9 FY10 5.73 13.9 FY11 14.61 71.7 FY12 16.80 70.5 FY13 17.60 91.9 FY14 18.47 104.7 Total: 87.82 657.0 49 Annex 5: Beneficiary Survey Results Methodology In each prefecture investigators visited two towns (an urban district and a rural commune); occasionally, districts far from the capital of the municipality were visited to account for the effect of distances. A total of 71 Focus group discussions involving 535 participants (male and female) were organized; and 1245 questionnaires were administered to 321 patients and 924 health care facilities, including: Urban Hospitals and prefectural CS , CS houses, national Hospital ) and of the health services (DRS , DPS and DCS , urban and rural CS , Community health Workers , national Direction ( family Health Community Health Directorate in Charge of Hospitals , Office of Strategy and Development). The questions concerned the following services provided by the project: Antenatal care; treatment of common diseases during pregnancy; complications related to pregnancy; assisted delivery; Family Planning; care and services for the integrated support of childhood illness; vaccination of small children; nutrition and diet; care and services for the Reference and Counter reference; primary curative consultation; mobilization and community participation; and the status of infrastructure and associated equipment. Overall conclusion Based on an analysis of the various beneficiary groups surveyed (men and women, patients encountered in health facilities, and health personnel at various levels of the health pyramid): • People feel a sense of satisfaction with the relief provided by the care and services in public health facilities through the efforts of government and the support of numerous technical and financial partners, despite the persistent socio-political difficulties; • However this level of beneficiary satisfaction apparently is still at a high level means taking into account the actual gap between care and health services in some areas compared with standards and procedures on the one hand, and on the other hand many limitations/constraints raised by the beneficiaries themselves (and confirming some opinions of health personnel), as well as the expectations expressed during the field survey. Recommendations To enhance the level of satisfaction of the beneficiaries of care and health services in the Republic of Guinea, in accordance with the standards and procedures, it is recommended that: • A substantial renovation of infrastructure (buildings and equipment associated water, energy, toilet) would improve the work of health personnel; • A strengthened infrastructure development plan would improve services by building and equipping new health facilities closer to the to the beneficiaries of services and would increase access to a greater number by eliminating the constraints of distance and isolation; • The continued improvement in the level of care equipment and services and the supply of medicines and care products in the health facilities; • The development and implementation of an integrated IEC based on the technical guidance documents developed in the context of health promotion action plan; • The substantial implementation of measures for human resource envisaged in the National Health Development Plan but only modestly implemented as noted in the evaluation report; • The application of measures to improve the governance of health facilities to better reassure beneficiaries and encourage more to attend health facilities. 50 Annex 6: Stakeholder Workshop Report and Results Not applicable 51 Annex 7: Summary of Borrower's ICR and/or Comments on Draft ICR Introduction The Project was originally planned to cover the period 2006-2011 but was extended until December 2013 because of political developments in Guinea in late 2008 which led to a two-year suspension (2009-2010) of project activities. The main objective of the project, based on the five-year plan, the Poverty Reduction Strategy Paper, and the Millennium Development (MDGs), is to contribute to the objectives of reducing maternal mortality and child mortality. The specific objectives of the project are: • Reducing maternal mortality, neo natal and infant in the project area ; • Strengthen community participation for ownership of health facilities by the community; • Strengthen the management capacity of the structures to provide at any time quality services. The two components of the project are to strengthen health services and institutional capacity. Results Relevance of the project (consistency of project strategies with the strategic axes of the PNDS): The analysis of the various components of the project shows that they are a perfect fit with the strategic axes 1 and 2 respectively of the PNDS for integrated fight against disease and maternal mortality and institutional strengthening and management. This justifies the relevance of the project and its perfect harmony with national priorities described in the strategy document of poverty reduction and the priorities of the World Bank, which aims to support countries in reducing poverty. Assessment of project effectiveness (level of achievement of objectives): A gradual evolution of immunization coverage as well as coverage of other activities related to maternal and neo natal care was observed for: vaccination across the four antigens, antenatal consultations, assisted deliveries, caesarean sections, etc. This development was probably the result of health center implementation of their activities including support for outreach sessions; contracting health workers and the payment of bonuses to vaccinators; the financing of operating costs for refrigerators, regular supervision, etc.; and support for institutional strengthening of prefectural health departments. Coverage: A gradual evolution of BCG coverage (83% to 93%) was observed in all health centers since the actual start of the project in 2007. The fall observed since 2009 could be explained by the termination of funding from that date following the political developments in Guinea in December 2008. Another drop in BCG coverage in 2012 was due to a significant decrease availability of BCG vaccine during this period. Use of MNCH services: As measured by ANC1 coverage (for first contact) and ANC3 coverage (for acceptability/continuity), there was a gradual evolution of the coverage in the area, from less than 27% in 2006 to 91% in 2012. CPN1 coverage is well above the national coverage described in the DHS 2012 where it is 85.2%. ANC3 coverage also followed the pace of progress in the project area from 25% in 2006 to 75% in 2012 while it was 56.6% at the national average according to the DHS 2012. 52 Deliveries in health facilities: They increased significantly during the project period, from 11% in the project area in 2006 to 43% in 2013, while the national rate is 40.3 % according to the DHS 2012. Training for emergency obstetric care providers in the project area as well as the equipment of certain structures in materials explains this performance. It should be noted that this rapid growth may not be the mere fact of the project but rather the result of a combination of contributing factors, including the decree establishing the free delivery of 2010, although the accompanying measures in place for its implementation have not been to the full satisfaction of service providers in terms of the need for drugs and consumables necessary for the normal functioning of health facilities. Coverage rates for caesarean sections: Given the national goal is of covering 5% for all cesarean deliveries expected, the project made steady progress between 2006 (where only 0.4 % of caesarean sections were performed) to 2008 (1.30%) before declining between 2008 and 2010. This rise in coverage between 2007 and 2008 followed the implementation of the government's 2007 decision establishing the free caesarean section. Blood transfusion: Staffing hospitals in the area of equipment and consumables, needed for collection, storage and administration of blood and blood products, improved progressively and the number of blood transfusions increased from 188 in 2006 to 1944 in 2013. Effects Benefits in quality and care: Integration of the main diseases that cause child mortality (diarrheal diseases, malaria and respiratory infections) into child health care services; training on emergency obstetrical care provider for health workers at health center and hospital levels; improved quality of care for beneficiaries. Similarly, the provision of delivery kits and caesarean kits and equipment for the management of bio-medical waste, including incinerators for hospitals and sanitation equipment at the small centers health have also helped to improve the quality of care and strengthening the institutional capacity of structures. Benefits in institutional strengthening: Project interventions in terms of institutional capacity building covered all levels of the health pyramid. • Central level: all departments and central services have benefited from the support of the project, including financing their participation in important national meetings of MOH, such as the Regional Technical Committees of Health (CTRS ) and support financing of supervision teams and intermediate structures at the central level. Also some Directions received logistical support by providing vehicle support fuel and vehicle maintenance provided. • Intermediate and peripheral: Most targeted areas received vehicle supervision and financial support for supervision and monitoring of health facilities. In addition, the supply of ambulances for hospitals as well as the provision for DRS, DPS and DH of the project area for telephone service strengthened operational capacity of these structures especially in communication with front-line health structures for effective management of obstetric referrals and timely transmission of information on the epidemiological situation. It should also be noted that the decentralized level, the agents received training in the development of operational action plans and monitoring its implementation. • Community level: Capacity building at the community level is done through training of members of the health committee and health management resources and training of 53 community-based health workers on Community PCIMNE well as the training of village midwives to recognize danger signs of pregnancy and childbirth and refer cases to the health center. Sustainability: The sustainability strategies developed by the project have included: the involvement of communities in health center management; training midwives; financial support for certain health center budget items; development and implementation of participatory annual plans and budgets with health centers/hospitals; introduction of tools for managing resources; logistical support including vehicles, equipment and operating costs; training of health center and hospital personnel; transfer of financial management responsibilities to health centers and hospitals; networking of community-based associations. Lessons Learned • Project transparency through accountability in the management of CCS • Upgrading agents and AV provides support quality patient • Business planning helps streamline efforts • Good training delivery agents can qualify services • Proper training of service providers MNCH ensures proper management of women • The proper formation of AV and AC facilitates the mobilization of women around the MNCH services • Proper training of AV and AC allows an increase in attendance at the CS • Community empowerment in the management of funds promotes good governance • The allocation of funds to beneficiaries structures increases the credibility of structures • Contracting officers with the community can improve services • Substantial financial support led to an increase in the level and quality of services • Contracting of local staff is a guarantee for retention agents • The establishment of the Steering Committee has minimized duplication. The synthesis of lessons learned from the project shows that the two basic elements that emerge are those related to human resources and management arrangements of the project. • With regard to human resources, health workers were the most important investment made by the project at health center level. The staffing of health facilities including recruitment and management have been made at Community level filled a deficit when the project promoted local recruitment. Even despite the delayed payment of their wages, contracted staff remained in their posts for the duration of their contract with the communities. This experience should be used to address the chronic shortage of health personnel especially in rural areas. • The second element is emerged from discussions accountability of health centers and health committee and health as a motivator and a guarantee of good governance in these structures. 54 Annex 8: Comments of Co-financing partners and Other Partners/Stakeholders Not applicable 55 Annex 9: List of Supporting Documents National documents (laws, policies, strategies, norms and protocols, etc.) Laws, decrees, and regulatory texts o Arrêté No. 858/MSHP/CAB/DRH du 18 février 2011 portant création, attributions et fonctionnement du Comité Technique de Pilotage du Projet d’Appui au Plan National de Développement Sanitaire (CTP/APNDS) o Arrêté No. 859/MSHP/CAB/DRH du 18 février 2011 portant création, attributions et fonctionnement du l’Unité de Gestion du Projet d’Appui au Plan National de Développement Sanitaire (UGP/APNDS) Policies, strategies and regulatory measures o Plan National de Développement Sanitaire de Guinée 2003-2012 (Juin 2003). o Programme Quinquennal de Développement Sanitaire 2003-2007 (Juin 2003). o Manuel de procédure de gestion, administrative, financière et comptable du programme quinquennal 2003-2007 (Août 2003). o Politique Nationale de l’Hygiène Publique (Avril 2010) Strategic plans o Atelier de relance des soins de santé primaires en Guinée : Rapport technique (Décembre 2008). o Feuille de route pour l’accélération de la réduction de la mortalité maternelle et néonatale en Guinée XXX (XXXX). o Plan stratégique de développement des ressources humaines pour la sante en Guinée 20XX - 20 (XXXX). o Stratégie Nationale de promotion des mutuelles de santé (Décembre 2008). o Mécanisme de coordination et de suivi du secteur santé (Janvier 2009). o Rapport de synthèse du système de santé en Guinée (Mars 2012) Normes and protocoles • Division de la Santé de la Reproduction, Normes et Procédures en Santé de la Reproduction (Septembre 2009). • Révision du Protocole d’accord pour, l’acquisition, la Réception, le Reconditionnement et l’acheminement de produits pharmaceutiques et matériels médicaux par la Pharmacie Centrale de Guinee (PCG) (2012). • DNPL, Formulaire Simplifié des Médicaments Essentiels (2013)(Diffusion). Etudes de faisabilité • Rapport final du forum de santé sur les mutuelles de santé (Juin 2007). • Etude pour l’élaboration d’un document d’orientation pour l’appropriation des centres de santé par la communauté (XXXX). • Etude sur la définition et la faisabilité de l’assurance maladie obligatoire en Guinée (Octobre 2008). • Health Focus, Etude sur plan planification et l’utilisation des ressources humaines pour les dix prochaines années dans les services de première ligne (Novembre 2008). Project documents Legal o PAD, Health Sector Support Project (Report No. 28046-GUI) (May 2005). o Restructuring Paper, Health Sector Support Project (Report No. No: 65769-GN) (December 2011). 56 o DCA o Aides mémoires and Implementation Status Reports Environmental and fiduciary o RSF o External audits / Rapports annuels d’audit o o Plan National de Gestion des déchets bio-médicaux en Guinée (May 2002). o Cadre réglementaire de recasement et de compensation des populations affectées par les travaux d'infrastructures de santé (April 2004). Technical / Components • PHRD o Alhassane Sow, Evaluation des attentes des bénéficiaires du PASS ( o SOFRECO, Plan de développement à moyen terme des infrastructures sanitaires ( o Evaluation nationale des soins obstétricaux d’urgence en Guinée (Août 2003). o MCDI, Revue des dépenses du secteur de la santé en Guinée (Mai 2003). • Component 1 o Management et Formation pour le Développement, Etude sur les avortements à risque : Ampleur, causes et pourcentage dans la mortalité maternelle (Septembre 2008). o Rapport de mission de supervision des opérateurs des mutuelles de santé de la zone APNDS (Août 2013). o Rapport technique de l’atelier de formation des chefs de CS de la zone APNDS en gestion logistique des médicaments essentiels, des consommables médicaux et outils de gestion (Novembre 2013). • Component 2 o Bureau de Stratégie et de Développement, Comptes Nationaux de la Santé pour l’exercice 2010 (2012). o Dr. Sékou Condé, et al., Rapport d’évaluation des sous projets de la zone APNDS (Novembre 2012). o Oumar Bah et Ousmane Diop, 2eme Enquête sur le Suivi des Depenses Publiques Jusqu’à Destination (ESDD) dans le Secteur de la Sante en Guinee (Avril 2013). o Dr. Télly Diallo, Etude sur la satisfaction des bénéficiaires des soins et services de santé en République de Guinée (Mai 2013). o M. Alpha Ousmane Diallo and M. Amara Drame, Revue des dépenses publique du secteur de la sante de (2005-2012) (Mai 2013) o Enquête d’évaluation finale de l’APNDS (Février 2014). Managerial • Projet de manuel d’exécution (Avril 2005). • APNDS, Rapport Annuel 2007-2009, 2011-2012. Monitoring and evaluation • Guinée Enquête Démographique et de Santé 2005 (2006). • Guinée Enquête Démographique et de Santé 2011 (2012). • Multi-Indicator Cluster Survey (MICS) (2010) (2011). • Annuaire des statistiques sanitaires de Guinée (Années 2006-2011). 57 MAP 58