Document of The World Bank Report No: ICR 89189-MX IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-78600) ON A LOAN IN THE AMOUNT OF US$1,250 MILLION TO THE UNITED MEXICAN STATES FOR A SOCIAL PROTECTION SYSTEM IN HEALTH PROJECT June 27, 2014 Human Development Sector Management Unit Mexico and Colombia Country Management Unit Latin America and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2013) Currency Unit = Mexican Peso (MXP) MXP$13.0648 = US$1.00 US$1.00 = MXP$.07648 FISCAL YEAR January 1- December 31 ABBREVIATIONS AND ACRONYMS BP Bank Procedure CDI Commission for Indigenous Development (Comisión Nacional para el Desarrollo de los Pueblos Indígenas) CIDE Center for Economic Research and Teaching (Centro de Investigación y Docencia Económica) CNPSS or Commission National Commission of Social Protection in Health (Comisión Nacional de Protección Social en Salud) COFEPRIS Federal Commission for Protection against Health Risks (Comisión Federal para la Protección contra Riesgos Sanitarios) CONAPO National Population Council (Consejo Nacional de Población) CONEVAL National Council for the Evaluation of Social Development Policy (Consejo Nacional de Evaluación de la Política de Desarrollo Social) CPS Country Partnership Strategy DALY Disability Adjusted Life Years ENIGH National Household Income and Expenditures Survey (Encuesta Nacional de Ingresos y Gastos de los Hogares) ENSANUT National Health and Nutrition Survey (Encuesta Nacional de Salud y Nutrición) FM Financial Management FY Fiscal Year GDP Gross Domestic Product GOM Government of Mexico IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report IFR Interim Financial Report IMSS Mexican Social Insurance Institute (Instituto Mexicano del Seguro Social) INEGI National Institute for Statistics, Geography and Informatics (Instituto Nacional de Estadística Geografía e Informática) INSP National Public Health Institute (Instituto Nacional de Salud Pública) IPP Indigenous Peoples Plan ISR Implementation Supervision Report ISSSTE National Institute for the Social Security of Government Workers (Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado) LAC Latin America and the Caribbean Region ii M Million MDGs Millennium Development Goals M&E Monitoring and Evaluation MxFLs Mexican Family Life Survey MXP Mexican Peso NAFIN Mexican Goverment Financial Agent (Nacional Financiera, S.N.C., I.B.D.) NDP National Development Plan OECD Organization for Economic Co-operation and Development OM Operational Manual OOP Out-of-Pocket OP Operational Policy PAC Program for Coverage Extension (Programa de Ampliación de Cobertura de Salud) PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PDI Project Development Indicator PHI Popular Health Insurance (Seguro Popular) PPP Purchasing Power Parity PROSESA National Health Sector Program (Programa Sectorial de Salud) REPSS State Regimes of Social Protection System in Health (Regímenes Estatales de Protección Social en Salud) SHCP Ministry of Finance and Public Credit (Secretaría de Hacienda y CréditoPúblico) SIL Specific Investment Loan SINAIS National Health Information System (Sistema Nacional de Información en Salud) SINOS Personalized Health Registry (El Sistema Nominal en Salud) SPSS Social Protection System in Health (Sistema de Protección Social en Salud) TA Technical Assistance TF Trust Fund UHC Universal Health Coverage US United States USD United States Dollar WBG The World Bank Group WHO World Health Organization Vice President: Jorge Familiar Calderón Country Director: Gloria M. Grandolini Sector Manager: Joana Godinho Project Team Leader: Claudia Macias ICR Team Leader: Christel Vermeersch ICR Primary Author Christel Vermeersch and Natasha Zamecnik iii MEXICO Social Protection System in Health TABLE OF CONTENTS Data Sheet A. Basic Information....................................................................................................... v B. Key Dates ................................................................................................................... v C. Ratings Summary ....................................................................................................... v D. Sector and Theme Codes .......................................................................................... vi E. Bank Staff .................................................................................................................. vi F. Results Framework Analysis ..................................................................................... vi G. Ratings of Project Performance in ISRs ................................................................... xi H. Restructuring (if any) ................................................................................................ xi I. Disbursement Profile ................................................................................................ xii 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 11 4. Assessment of Risk to Development Outcome......................................................... 17 5. Assessment of Bank and Borrower Performance ..................................................... 18 6. Lessons Learned ....................................................................................................... 19 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 21 Annex 1. Project Costs and Financing .......................................................................... 22 Annex 2. Outputs by Component ................................................................................. 23 Annex 3. Economic and Financial Analysis ................................................................. 43 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 46 Annex 5. Beneficiary Survey Results ........................................................................... 48 Annex 6. Stakeholder Workshop Report and Results................................................... 50 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 51 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 61 Annex 9. List of Supporting Documents ...................................................................... 62 Annex 10. Map IBRD 33447R ..................................................................................... 66 iv A. Basic Information Support to the Social Country: Final Mexico Project Name: Protection System in Health Project ID: P116226 L/C/TF Number(s): IBRD-78600 ICR Date: 06/27/2014 ICR Type: Core ICR UNITED MEXICAN Lending Instrument: SIL Borrower: STATES Original Total USD 1,250.00M Disbursed Amount: USD 1,250.00M Commitment: Revised Amount: USD 1,250.00M Environmental Category: C Implementing Agencies: Secretariat of Health Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/10/2009 Effectiveness: 12/29/2010 12/17/2010 11/22/2011 Appraisal: 02/18/2010 Restructuring(s): 02/28/2013 Approval: 03/25/2010 Mid-term Review: 09/24/2012 11/08/2012 Closing: 12/31/2013 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: v C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 95 95 Public administration- Health 5 5 Theme Code (as % of total Bank financing) Health system performance 90 90 Social safety nets 10 10 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Gloria M. Grandolini Gloria M. Grandolini Sector Manager: Joana Godinho Keith E. Hansen Project Team Leader: Claudia Macias Christoph Kurowski ICR Team Leader: Christel M. J. Vermeersch ICR Primary Author: Natasha Zamecnik Christel M. J. Vermeersch F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is to (i) initially preserve and later expand the Popular Health Insurance's coverage of people without social security and (ii) strengthen the capacity of the Commission (for social protection in health) and State Health Systems to effectively administer the entitlements of the Popular Health Insurance. vi Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Number of individuals affiliated with Popular Health Insurance as a percentage of the Indicator 1 : total number of individuals that are not affiliated with a contributory social security system. Value 64% 85% 115% quantitative or (31.1 M / 48.4 M) (41.1 M / 48.4 M) (55.6 M / 48.4 M) Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Source: CNPSS Comments Target surpassed. (incl. % % achievement for numerator: achievement) 245% = (55.6-31.1)/(41.1-31.1) See Table 7 in Annex 2 for additional information. Percentage of recommended actions implemented by federal entities resulting from the Indicator 2 : supervision action plan carried out by the Commission in action areas (Affiliation and Operation, Health services management, Financing and Oportunidades program). Value quantitative or 0 N/A 71% 90.3% Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 12/31/2013 Source: CNPSS. Comments Description of indicator revised during first restructuring. See Annex 2. (incl. % Target surpassed. achievement) % Achievement: 127%=90.3/71 Number of individuals affiliated with the Popular Health Insurance who have received a Indicator 3 : "Health Risk Screening". Value quantitative or 0 10,000,000 22,800,000 Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Source: CNPSS. Comments Description of indicator revised during first restructuring. (incl. % Target surpassed. achievement) % achievement: 228%=22,800,000/10,000,000 Number of individuals affiliated with the Popular Health Insurance that report having Indicator 4 : received enough information to know their rights and obligations as a percentage of the total number of individuals affiliated with the Popular Health Insurance. Value quantitative or 71.2% 80% 74.3% Qualitative) 12/31/2013 Date achieved 12/31/2009 12/31/2013 vii Comments Source: ENSANUT for both numerator and denominator. (incl. % Progress made but target not achieved. achievement) % achievement: 35%=(74.3-71.2)/(80-71.2) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Federal and State expenditure on the Popular Health Insurance (US$ constant exchange Indicator 1 : rate 2009) Value (quantitative $1,649.8 M $6,481.69 M $4,503.08 M $5,236.48 M or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 12/31/2013 Comments Source: CNPSS (incl. % Target surpassed. achievement) % achievement: 126% =(5,236.48-1,649.8)/(4,503.08-1,649.8) Number of federal entities that contribute with their State Solidarity Contribution (ASE) Indicator 2 : to the Popular Health Insurance according to the General Health Law. Value (quantitative 31 31 32 or Qualitative) Date achieved 01/26/2010 12/31/2013 12/31/2013 Comments Source: CNPSS (incl. % Target achieved. achievement) % achievement: 103%=32/31 Number of individuals in deciles 1 and 2 affiliated with Popular Health Insurance's Indicator 3 : subsidized regime as a percentage of total number of individuals in deciles 1 and 2 that are not affiliated with a contributory social security system. Value 48% 77% 97% (quantitative (9.9 M / 20.7 M) (16 M / 20.7 M) (20.2 M / 20.7 M) or Qualitative) Date achieved 12/31/2008 12/31/2013 12/31/2013 Comments Source at PAD: ENIGH. Results reported by Government based on ENSANUT. (incl. % See Table 7 in Annex 2 for additional information. achievement) Number of Oportunidades beneficiary families affiliated with the Popular Health Indicator 4 : Insurance as a percentage of the total number of Oportunidades beneficiary families. Value 60.7% 80% 113% (quantitative (3.06 M / 5.03 M) (4 M / 5.03 M) (5.7 M / 5.03 M) or Qualitative) Date achieved 12/31/2008 12/31/2013 12/31/2013 Source: CNPSS (numerator); Opportunidades (denominator). Comments Baseline was adjusted in first restructuring. (incl. % ICR uses numerator only. (See Annex 2) achievement) Target surpassed. viii Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years % achievement for numerator: 280% = (5.7-3.06)/(4-3.06) Number of individuals affiliated with PHI residing in areas where more than 40% of the Indicator 5 : population speaks an indigenous language as a % of total # of indivs. that are not affiliated with a contributory social security system that are residing in these area Value 92% 78.5% 50% 131% (quantitative (4.84 M / 5.27 (4.13 M / 5.27 M) (4.85 M / 9.67 M) (6.9 M / 5.27 M) or Qualitative) M) Date achieved 12/31/2009 12/31/2013 12/31/2013 12/31/2013 Source: CNPSS (numerator), INEGI (denominator). Comments Indicator adjusted at restructuring. (incl. % See Table 7 in Annex 2 for additional information. achievement) Target surpassed. % achievement for numerator: 413%=(6.9-4.13)/(4.8-4.13) Number of women and girls affiliated with the Popular Health Insurance as a percentage Indicator 6 : of the total number of women and girls who are not affiliated with a contributory social security system. Value 68% 88.7% 120% (quantitative (16.94 M / 24.8 M) (22 M / 24.8 M) (30 M / 24.8 M) or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Source: CNPSS/SINAIS (numerator). Comments See Table 7 in Annex 2 for additional information. (incl. % Target surpassed. achievement) % achievement for numerator: 258%=(30-16.94)/(22-16.94) Number of federal entities supervised by the Commission in any [of] its four core action Indicator 7 : areas (Affiliation and Operation, Health services management, Financing and Oportunidades program) during a calendar year. Value (quantitative 0 32 32 or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: CNPSS. (incl. % Indicator adjusted at first restructuring. achievement) Target achieved. Amount of funds (US$ constant exchange rate (2009)) allocated for technical assistance Indicator 8 : to improve the collection and analysis of State Health System results information. Value (quantitative $1.77 M N/A $2.51 M $8.02 M or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 12/31/2013 Source: CNPSS. Comments Target surpassed. (incl. % % achievement: 845%=(8.02-1.77)/(2.51-1.77) achievement) ix Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Number of individuals affiliated with the Popular health Insurance who report having Indicator 9 : received a bill of rights and responsibilities at the time of affiliation as a percentage of total number of individuals affiliated with the Popular Health Insurance. Value (quantitative 81.9% 88% 77.7% or Qualitative) Date achieved 12/31/2006 12/31/2013 12/31/2013 Comments Source: ENSANUT (incl. % Target not achieved but there is significant variation in achievement among the States. achievement) Number of individuals affiliated with Popular Health Insurance who report having Indicator 10 : received a catalogue of their benefits package at the time of affiliation as a percentage of the total number of individuals affiliated with the Popular Health Insurance. Value (quantitative 77% 83% 74.3% or Qualitative) Date achieved 12/31/2006 12/31/2013 12/31/2013 Comments Source: ENSANUT (incl. % Target not achieved but there is significant variation in achievement among the States. achievement) Number of individuals who report on the satisfaction survey to have received Indicator 11 : information at the time of affiliation with regard to their right to not pay service fees, as a % of [those] who participate in the satisfaction survey. Value (quantitative 53.4% 80% 89.2% or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: Encuesta de Satisfacción (incl. % Target surpassed but there is significant variation in achievement among the States. achievement) % achievement: 135% =(89.2-53.4)/(80-53.4). Information materials on Popular Health Insurance rights, responsibilities and affiliation Indicator 12 : processes designed for distribution by the Oportunidades program. Designed and Designed and Value distributed in the distributed in the (quantitative Not designed. states where the states where the or Qualitative) program operates. program operates. Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: CNPSS. (incl. % Target achieved. achievement) Indicator 13 : Health risk management program guidelines have been designed and rolled out. Value (quantitative No Yes Yes or Qualitative) x Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: CNPSS. (incl. % Target achieved: Program rolled out in 32 states. achievement) Number of States in which the health risk management program IT systems for data Indicator 14 : collection have been rolled out. Value (quantitative 0 31 32 or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: CNPSS. (incl. % Target achieved. achievement) % achievement: 103%=32/31 Number of States that capture biometric information of individuals affiliated with the Indicator 15 : Popular Health Insurance. Value (quantitative 0 31 32 or Qualitative) Date achieved 12/31/2009 12/31/2013 12/31/2013 Comments Source: CNPSS (incl. % Target achieved. achievement) % achievement: 103%=32/31 G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 06/28/2010 Satisfactory Satisfactory 0.00 2 02/23/2011 Satisfactory Satisfactory 815.50 3 07/26/2011 Satisfactory Satisfactory 815.50 4 12/27/2011 Satisfactory Satisfactory 990.98 5 07/11/2012 Satisfactory Satisfactory 1136.72 6 02/25/2013 Satisfactory Satisfactory 1238.31 7 11/25/2013 Satisfactory Satisfactory 1246.88 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions 11/22/2011 S S 905.46 Modification of the M&E xi ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions framework Reallocation of loan proceeds from 02/28/2013 S S 1238.31 Component 2 to Component 1 I. Disbursement Profile xii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. In an effort to expand health care to those without social security and improve health inequities, the Government of Mexico (GOM) revised the General Health Law and established the Social Protection System in Health (Sistema de Protección Social en Salud -- SPSS) and its main pillar the Popular Health Insurance (PHI), also called Seguro Popular.1 The 2003 health reform had several objectives: (a) to increase funds earmarked to the public health system and decrease the inequities in public expenditures across subsystems and states, (b) to improve health outcomes and reduce out-of-pocket (OOP) payments for health services and provide protection against catastrophic health expenditure, 2 and (c) to reform the organization and functioning of the state health system to ensure better PHI management, and establish incentives to promote equality, technical efficiency, and responsiveness. (Kurowski and Ortiz 2012) 2. By 2009, the PHI had produced a number of positive results in Mexico’s health system. An increase in public health expenditure, from 2.5 percent of GDP in 2003 to 3.0 percent of GDP in 2009, raised health spending on the population lacking health insurance, reduced the differential on public expenditure between those covered by formal insurance schemes and the uninsured (from 2.1/1 to 1.2/1), and narrowed differences across state spending. (Bonilla-Chacín and Aguilera 2013) Investment in public health care reduced financial barriers to health service access, increased utilization rates, and reduced catastrophic and impoverishing health expenditures in new affiliates. 3. Nevertheless, the PHI faced some important challenges regarding coverage expansion and beneficiary entitlement awareness. The 2009 economic crisis, which resulted in a brief though deep contraction of over 6 percent of GDP, and A/H1N1 influenza outbreak threatened the health gains achieved in the country, increased poverty and put pressure on the finances of poor and vulnerable families. An expansion or at least maintenance of the PHI was necessary in order to protect the health and finances of new affiliates relying on the system and 1 In 2002 only half of the Mexican population benefitted from social insurance protection, largely via their employment condition: the formal private sector workers through the Mexican Institute for Social Security (Instituto Mexicano del Seguro Social - IMSS), and federal public sector workers through the National Institute for Social Security and Services for Civil Servants (Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado -- ISSSTE). The self-employed, underemployed and unemployed workers accessed health services through the State Ministry of Health or paid for private care. Public health care, through the State Ministry of Health was funded from uncertain, residual budget allocations lacking explicit entitlements. (Knaul et al. 2012) 2 Unequal access to health care reduced equality of opportunity for a large portion of the population and left the majority of the uninsured vulnerable to health care shocks. Unequal access to services partly reflected an unequal distribution of public health spending across different institutions and among states. Public resources allocated to social security institutions were more than double per capita of those assigned to those without social security. (Pueblita 2013) Furthermore, more than half of all health expenditure in Mexico was financed OOP, much of it paid by poor and uninsured families. (Secretaría de Salud 2004) High out- of-pocket costs reduced health care access and utilization for those with the highest need and exacerbated inequality. 1 those still lacking health insurance coverage. At the same time, beneficiary surveys in Mexico revealed that many PHI affiliates lacked knowledge regarding their entitlements. 4. The PHI also faced important challenges regarding the effective administration of entitlements. The rapid increase in coverage did not automatically translate into the availability of quality health care services. The limited number of pre-approved medical interventions, fragmented nature of the Mexican health care system, combined with the lack of portability and service convergence between health insurance programs and between states restricted the benefit to affiliates and hindered the system’s performance. To improve administration capacity, organization and managerial changes to the State Health Systems were necessary, in particular expanding autonomy of the State Regimes of Social Protection System in Health (Regímenes Estatales de Protección Social en Salud – REPSS) and strengthening their functions. The lack of accountability regarding PHI federal funds also suggested stronger performance management arrangements between the Commission and State Health Systems. 5. The Project was aligned with the World Bank Group’s Country Partnership Strategy (CPS) 2008 - 2013 (Report No. 42846-MX) discussed by the Board of Directors on March 4, 2008, with the Government’s National Development Plan (NDP) 2007-2012 and supported progress on health-related Millennium Development Goal (MDGs). The Project was closely aligned with two of the strategic development challenges identified in the CPS: sustainable growth through investment in human capital and institutional strengthening. The World Bank’s experience with the Mexican health sector and with health insurance programs worldwide made it a natural partner in expanding PHI coverage and strengthening entitlement administration. Further, across the globe, World Bank projects had begun to co-finance health insurance premiums. The Project was a logical extension of this practice, financing the insurance premium of the developing world‘s largest (both in terms of population and benefits) and highly visible health insurance for people without contributory social security. 1.2 Original Project Development Objectives (PDO) and Key Indicators 6. The PDO was to (i) initially preserve and later expand the PHI’s coverage of people without contributory social security, and (ii) strengthen the capacity of the Commission 3 and State Health Systems to effectively administer the entitlements of the PHI. The Project’s results were measured via four PDO indicators and 15 intermediate results indicators. 4 The Project’s PDO indicators were: 5 • PDO Indicator 1: Number of individuals affiliated with the PHI as a percentage of the total number of individuals that are not affiliated with a contributory social security system. 3 National Commission of Social Protection in Health (CNPSS or “the Commission”). 4 The numbering and sequence of indicators used in the ICR are the ones from the 2011 restructuring paper and are different from the ones in the Project Apraisal Document (PAD). For the sake of brevity, the ICR uses “PDO Indicators” for “PDO Level Results Indicators” and “Intermediate Indicators” for “Intermediate Results Indicators”. 5 PDO indicators two and three were adjusted almost one year after Project effectiveness. See Section 1.3. 2 • PDO Indicator 2: Number of State Health Systems that collect information on system results (including beneficiary satisfaction and number, quality and cost of services delivered) that is validated, widely publicized and used for performance improvement recommendations by the Commission. • PDO Indicator 3: Number of individuals affiliated with the PHI who have received a “Health Risk Screening” as a percentage of the total number of individuals affiliated with the PHI. • PDO Indicator 4: Number of individuals affiliated with the PHI that report having received enough information to know their rights and obligations as a percentage of the total number of individuals affiliated with the PHI. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 7. The original PDO remained valid and unchanged throughout Project implementation. However, two PDO indicators were revised in November 2011 during the first Project Restructuring (See Section 1.7). PDI Original indicator Modified indicator Reason for change 2 Number of State Health Systems Percentage of recommended Provide a greater focus on that collect information on system actions implemented by federal actions to improve PHI results (including beneficiary entities resulting from the administration. satisfaction and number, quality and supervision action plan carried out cost of services delivered) that is by the Commission in its four validated, widely publicized and action areas (Affiliation and used for performance improvement Operation, Health services recommendations by the management, Financing and Commission. Oportunidades program). 6 3 Number of individuals affiliated Number of individuals affiliated The accelerated expansion of with the Popular Health Insurance with the Popular Health Insurance PHI coverage led to a who have received a “Health Risk who have received a “Health Risk continuous modification in the Screening” as a percentage of the Screening.” denominator of the original total number of individuals affiliated indicator, and continuous target with the Popular Health Insurance. adjustment. The modified indicator measures progress in absolute terms rather than in relative terms. 1.4 Main Beneficiaries 8. The Project’s intended beneficiaries were individuals not affiliated with a contributory social security system. For this population, the Project specifically intended to benefit (i) 31.1 million individuals already covered by PHI: by preserving their coverage; (ii) 10 million individuals not yet covered: by providing new PHI coverage; (iii) 10 million PHI affiliates (new or existing): from health risk screenings; and (iv) all 41.1 million PHI affiliates: from improved information about and understanding of their rights under the PHI. 6 The original indicator lacked baseline information and targets, which were established for the revised indicator. 3 9. The Project’s actual beneficiaries included: (i) 31.1 million individuals already covered by PHI: by preserving their coverage; (ii) 24 million new affiliates who joined the PHI during Project implementation; (iii) 22.8 million affiliates (new or existing) who received a health risk screening; and (iv) 55.6 million (i.e. all) affiliates who benefited from improved information about and understanding of their rights under the PHI. In addition, both new and existing PHI affiliates benefitted from improvements in entitlement administration and from enhancements in PHI performance and capacity. As shown in Section 3.2 and in Annex 2, the majority of new PHI affiliates were members of vulnerable groups in Mexico. 1.5 Original Components 10. The Project consisted of two components, with the following initial loan allocation (final expenditure in parentheses) and main activities (Annex 2): 11. Component 1: Popular Health Insurance’s Coverage of People without Contributory Social Security. US$1,239 million (US$1,250 million). The component aimed to initially preserve and later expand the PHI’s coverage of eligible beneficiaries. Specifically, Component 1 financed 70 percent of the federal social contributions to the PHI premium as referred to in Article 77 of the General Health Law. 12. Component 2: Capacity of the Commission and of State Health Systems to Administer the Popular Health Insurance Entitlements. US$7.875 million (US$0.00). The component aimed to (i) support enhancements in performance management in the administration of the PHI, and (ii) support State Health Systems in preparing and carrying out reforms in the administration of the PHI. The activities under the component were technical assistance including, inter alia, the review and refinement of the federal monitoring and audit system (including Coordination Agreements), assessments of the capacity of State Health Systems to monitor the delivery of health services to different groups of affiliates (including vulnerable groups), the development of instruments to capture and validate information, the development of performance benchmarks, the design of a disclosure policy for the PHI, the development of platforms to transfer experiences and knowledge across State Health Systems, and the development of financing mechanisms for technical assistance to State Health Systems to effectively carry out key health system functions, and specifically, their responsibilities as the administrators of the PHI. 1.6 Revised Components 13. The components were not revised during implementation. However, their financing was revised as described in Section 1.7. 1.7 Other Significant Changes 14. The Project had two Level II restructurings. The first restructuring (November 22, 2011) modified the Monitoring Framework in order to allow for more accurate measurement of Project progress and impact. The modifications helped fill some gaps in the original monitoring framework, took advantage of improvements in data availability, corrected for inadequate 4 sources of information and simplified some indicators. The second restructuring (February 28, 2013) reallocated loan proceeds from Component 2 to Component 1. While the Commission initially planned to use loan proceeds to finance Component 2 activities, after Project approval it received sufficient own budget funds to carry out the activities without the loan proceeds. Using own resources allowed the Commission to avoid duplicate procurement processes. 7 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 15. The Project responded to the Government’s policy and priorities for the health sector. The Mexican Government and the Commission were highly committed to the Project, as the previous administration aimed to make the right to healthcare a concrete entitlement. The Project supported the Government’s 2007-2012 National Health Sector Program (Programa Sectorial de Salud - PROSESA), which aimed to increase health service utilization, reduce out- of-pocket expenditures and reduce mortality. It was also aligned with the National Development Plan (NDP) 2007-2012, which focused on reducing inequalities and providing equal opportunities. In addition, the Project supported progress on health-related MDGs. 16. The Project represented one of the largest health loans ever made by the World Bank and one of the largest in the country. As reflected by the Independent Evaluation Group (IEG) crisis response analysis, IEG, The World Bank Groups’ Response to the Global Economic Crisis, Phase II (2012), the large amount of lending to Mexico was a reflection of country demand and the existence of a mature social protection program. 17. The World Bank provided technical expertise, policy advice, convening services, and supported a number of studies in Mexico, focusing on coordination, regulation, supervision, prevention and beneficiary empowerment in the public health system. 8 The World Bank’s extensive experience in designing, implementing and strengthening health insurance programs, in particular aimed at coverage expansion for the poor, informal workers and other vulnerable groups and their families, provided relevant and timely information. 18. Two essential lessons served as a backbone to Project design: the importance of empowering affiliates of insurance systems serving the poor and vulnerable, and the need for comprehensive incentive and support systems to foster reform in highly decentralized 7 When using loan funds for procurement of studies and technical assistance, the Commission had to use the internal clearance and approval processes as well as the World Bank review process. The use of a financial agent/intermediary also implies additional processing, which added to the overall administrative burden. 8 At the time of Project implementation, the World Bank was supporting the introduction and scale-up of non-contributory or subsidized-contributory health insurance schemes in more than 40 countries, including Argentina, the Dominican Republic, Honduras, Paraguay and Nicaragua. The World Bank assisted the documentation and dissemination of information regarding PHI in national and international workshops, seminars and conferences in and outside Mexico. The World Bank arranged workshops and high-level consultations, such as the December 2009 Workshop – Strategies for Continued Success: Taking PHI’s Achievements Forward, which provided an opportunity for the exchange of ideas and experiences for further strengthening of PHI, and the World Bank Knowledge Exchange: Friend or Foe? Mexico’s System of Social Protection in Health and the Formal Sector. The latter brought together a broad set of researchers, which presented findings and studies regarding PHI. 5 contexts. Studies in Mexico had shown that many affiliates lacked knowledge regarding their entitlements, and evidence from around the globe revealed that beneficiary entitlement knowledgeable was crucial in encouraging health service utilization and in holding providers accountable by using complaint, mediation, and arbitration systems. Experience from countries such as Argentina, China, Indonesia, Poland, and Turkey showed the importance of results monitoring, setting benchmarks, making financing conditional on results, creating experience exchange platforms, and building capacity to ensure administration effectiveness of entitlements. At the time of Project design, it was not politically feasible to link payment to the States to targets of service provision; however the Commission was committed to improve the administration of entitlements through technical assistance to the States, as well as negotiated agreements and supervision. While Component 2 of the Project only accounted for a small portion of the original loan amount, it addressed some initial but key steps to support the reform of the health financing system. 19. The Project supported Mexico’s progress towards Universal Health Coverage (UHC), which consists of three inter-related components: (i) coverage for the entire population; (ii) financial protection from out-of-pocket payment for health services when consumed; and (iii) the full spectrum of quality health services according to need. (WHO 2010) Project funds were mostly used to address the first two dimensions. The Project had an explicit aim to increase the number of people in PHI. The aim of coverage from financial risk was an implicit one, in that PHI benefits specify that its affiliates do not need to contribute for services at the point of service delivery; in theory, this means that PHI affiliates are covered from financial risk arising from needing health services; however, in practice much depends on whether the corresponding public services are available and of sufficient quality, or whether they need to seek alternative providers that do charge for services. Working on this third dimension (breadth and quality of services by PHI) is more complicated because service delivery is mostly decentralized to the States, and it would have required intervention at the state level, either directly or through incentive mechanisms between the federal government and the States. As noted above, it was not an option at the time of Project design to introduce such incentive mechanisms, though it remains a pending agenda. Instead, the Project opted to support the reform of the health financing systems through small but essential steps such as technical assistance, agreements and supervision. (See Supra and Annex 2) 20. The Mexican health system is fragmented, and in the PAD there was a concern that the Project may contribute to further fragmentation by supporting an additional system of insurance (the PHI). In reality, at the time of Project design, PHI was already covering over 25 percent of the population, so the Project did not create an additional program. At the time of creation of the PHI, the Government did not find it possible to expand one of the existing health insurance schemes (eg. IMSS or ISSSTE) instead of creating the PHI. 21. A Specific Investment Loan (SIL) was considered the most appropriate lending instrument. The loan mechanism combined a fast-disbursing component, which helped maintain and increase federal funding levels following the 2009 economic crisis by financing eligible federal social contributions associated with proof of new affiliation (measured via PDO indicator 1), with a technical assistance component that aimed to support the reform over the medium term. Component 2 was reassigned, the Commission used its own resources to implement the 6 activities of the component and gained from regular supervision and frequent interaction with World Bank staff. 22. The Project’s disbursement modality was fully aligned with the GOM’s existing financing arrangements for PHI. The PHI financing arrangements specify that the Federal Government makes two contributions: the Social Contribution and the Federal Solidarity Contribution. States make their own contribution called the State Solidarity Contribution. The World Bank reimbursed up to 70 percent of the federal social contribution transfers to the States, which itself accounts for approximately 1/3rd of the cost of the PHI scheme. 9 The financing did not discriminate between existing or new affiliates; rather than trying to identify the affiliates who would benefit directly from Project financing, the Project helped the Program to achieve its goals of preserving and expanding the number of affiliates. 2.2 Implementation 23. In 2010, PHI changed the unit used for calculating transfers to the States. Initially, PHI based the transfers to the States on the number of affiliated family units in each State. Family units included the head of household, spouse, children and parents. This gave States an incentive to try and game the system by subdividing family units in the enrollment registers. 10 In 2009, the General Health Law was modified so as to use the individual as the basis of calculation of the transfers. This change led to a steep rise in affiliation after 2009. (Annex 2) 24. Due to financing constraints stemming from the crisis, the loan disbursed rapidly and provided necessary financial support to Mexico’s safety net system. GDP fell by more than 6 percent in 2009 and though it rebounded quickly, poverty rose from 47.4 percent in 2008 to 51.3 percent in 2010. (CONEVAL) The timeliness of the Project is evident by its quick disbursement: the Project became effective on December 17, 2010, and within one year, 80 percent of the total loan amount was disbursed. 25. Project activities were coordinated and implemented using the organizational structures and staff of the Commission, which had been responsible for implementing the PHI since 2004. Though the Program expanded dramatically, both financially and in terms of coverage, the financial architecture, control mechanisms and management structure were in place and supported the further expansion. 9 The General Health Law establishes that (i) the premium of the PHI is a share9 of combined Federal and State contributions and (ii) the Federal Solidarity Contribution amounts to 150 percent of the Social Contribution while the State Solidarity Contribution amounts to 50 percent of the same. 10 By 2009, evidence showed that PHI included a larger than expected number of single person and small family units, presumably because states subdivided family units in order to increase the federal resource transfer. The original transfer rule also did not capture differences in family sizes among states, and often allocated more funds per person to the wealthier states with relatively smaller family units, thereby exacerbating health-financing inequities among states. 7 26. Notwithstanding the Project success, the decentralized nature of Mexico’s Health System resulted in unequal implementation performance. The PHI gave the States incentives to increase enrollment but not necessarily to improve the effectiveness and efficiency of service provision, as federal entities have a large degree of autonomy in the use of resources and in the organization and management of their health systems. Furthermore, state-level political commitment and health sector capabilities vary widely among the different States. Despite reductions in inequality in recent years, there are still large differences between the States in terms of health funding, availability of health services per PHI affiliate and health outcomes (CONEVAL 2013, OMM 2014). 27. The World Bank’s involvement over the life of the Project was beneficial in several areas. First, while the World Bank loan only financed a portion of the cost of the PHI program, it did signal the World Bank’s support to the Program and helped protect its funding during the financial crisis. In addition, the Project allowed for exchange and dissemination of knowledge, both nationally and internationally. (Annex 2) Finally, the design, supervision and implementation of the Indigenous People’s Plan led to a substantial shift in implementation strategy and tangible benefits for indigenous populations. (See Section 1.4) Finally, the Project’s supervision in the area of financial management supported the Government’s efforts to enhance performance management. 28. Despite the reallocation of the total loan proceeds from Component 2 to Component 1, the World Bank team decided to keep Component 2 as part of the Project and to continue supervising its achievements. The decision to retain this component can be justified with the following considerations: (i) It allowed the Bank to continue engaging in the area of reform of incentive and support systems to the States in the administration of PHI; while this reform is a long-term proposition, it is essential to ensure the effectiveness of the PHI funds; (ii) a number of Component 2 indicators were achieved through synchronous implementation with Component 1: in particular, new affiliates received the preventive health checkups and promotional materials of rights; (iii) the activities outlined in Component 2 were carried out by the Commission with support from the World Bank, albeit only using counterpart funding; and (iv) a significant portion of the World Bank’s own supervision funds were used to provide support and convening services to the Component 2 activities. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 29. During the implementation of the Project, the Commission invested heavily in improving its M&E capabilities. These included the development of indicators for monitoring at the state level, technical assistance to improve the collection and analysis of information in the States, investments in health information systems such as an individualized health information system to support preventive health screenings, and the rollout of a biometric identification system for affiliates. The Commission carried out a wide range of studies that included various evaluations of the PHI program. (Annex 2) 30. PHI was evaluated using a randomized evaluation methodology before the approval of the Project. (King et al. 2009) At the time of the Project’s design, it was not deemed possible to carry out a second impact evaluation of the Program, despite the fact that the 8 first evaluation did not demonstrate impacts on service utilization or health outcomes, possibly due to a rather short evaluation timeframe. During the implementation of the Project, the Commission contracted several external evaluations of the PHI program. (Eg. Lazcano 2012 and Lozano 2013) 31. The Project M&E Framework used a multi-pronged approach that included existing administrative data and regular survey data. At Project closing, the combination of these approaches allowed for measurement of PDOs. At the same time, the M&E framework could have made fuller use of regular survey data, and there were weaknesses in the indicators’ design and definition, which resulted in unclear and imprecise measurement of certain outcomes during the Project’s implementation. 32. Better definition of indicators and inclusion of indicators measuring effective access would have enhanced the Project M&E Framework. First, the Project Appraisal Document (PAD) and the Operational Manual (OM) lacked detailed information about indicator calculation. For a number of indicators, including PDO indicator 1 and intermediate indicators 3, 4 and 5, the monitoring framework was ambivalent about whether the indicator’s target was the percentage or the numerator and did not give sufficient specification of the denominator. Second, a number of indicators were rather complex, yet the design documents were incomplete as to the source of data and methods of calculation. Finally, while the Project’s monitoring framework included a wide array of indicators measuring affiliation, PDO indicator 3 is the only one that captures utilization of services by PHI affiliates. The Program’s success towards increasing effective access to health services could have been measured through: (i) public health spending per beneficiary, (ii) composition of PHI affiliates by income quintiles based on household surveys, and (iii) service availability and utilization. (See Table 7 in Annex 2) 33. During implementation, the lack of clarity in the design of the indicators resulted in imprecise measurement of a number of outcomes. A number of indicators that were expressed as percentages reached over 100 percent. For example the percentage of Oportunidades families affiliated with the PHI (Intermediate indicator 4) reached over 100 percent, seemingly because the Oportunidades family unit was defined differently in the numerator and denominator. Other indicators reached high levels but were not confirmed by alternative data sources. For example, intermediate indicator 3 reported that coverage of PHI among the non-insured in deciles 1 and 2 reached 97 percent by 2013. Still, the 2012 National Health and Nutrition Survey (Encuesta Nacional de Salud y Nutrición – ENSANUT) finds that approximately 20 percent of individuals in those deciles were not covered by any health insurance, which gives a rather different picture of the situation. As a result, this indicator does not convey accurate information about the reach of the program (Table 7 on Annex 2 and Annex 7). Still, because of the existence of regular household surveys, at the time of the ICR there were sufficient data available to estimate the efficacy of the Project. 2.4 Safeguard and Fiduciary Compliance 34. Environmental Safeguard. The proposed Project did not have potentially adverse environmental effects and therefore did not trigger the Bank’s environmental safeguard policy (OP/BP 4.01). It was classified as Environmental Category C. 9 35. Indigenous People Safeguard. The Indigenous People safeguard was triggered, and as mandated by OP/BP 4.10, the Government prepared an Indigenous Peoples Plan (IPP) to ensure that indigenous peoples benefited from the Project in a culturally appropriate manner. The IPP aimed to support indigenous people’s affiliation to PHI and ensure closer monitoring of affiliation, services and satisfaction in indigenous localities. (Annex 2) This is important because the health status of indigenous peoples has historically been poor in Mexico, with access to health services constrained due to remote and rural residence and lack of contributory social security enrollment. The social assessment prepared during the Project’s design identified many of the issues hindering indigenous peoples PHI enrollment. Based on these finding, the Project prepared and implemented a strategy that went beyond the Government’s initial cultural and communication plans for targeting the population. (See Annex 2) The significant increase in enrollment among this marginalized population reflects the success of the framework. Between 2006 and 2012, the gap in insurance coverage rates between indigenous and non-indigenous populations was virtually eliminated, and significant advances were made in narrowing service coverage gaps (eg. hospital based births) and even health outcomes (eg. stunting), even though there remain significant service utilization differences. (Leyva et al. 2012) Overall, the Project’s Indigenous Plan proved to be useful as an instrument to start to address the constraints faced by the indigenous population in health service access. 36. Fiduciary. Financial Management (FM) arrangements in terms of accounting, budgeting, flow of funds, internal control and financial reporting were moderately satisfactory throughout the Project’s life in providing reasonable assurance that loan proceeds were used for intended purposes. Some moderate FM shortcomings, such as delays in hiring the technical auditor and in submission of Project audits, were identified during Project implementation and negatively impacted the FM ISR rating, which at one point was downgraded to Moderately Unsatisfactory (MU). The final FM ISR rating of the Project (prior to closing date) was Moderately Satisfactory (MS), as most of the FM related shortcomings were timely followed-up and addressed. Most of the Project’s Interim Financial Reports (IFRs) were prepared and delivered in a timely manner, while the final financial audit was submitted with major delays, it was deemed acceptable to the World Bank. Nacional Financiera (NAFIN), in its capacity as the Project’s financial agent, provided the implementation support and oversight based on its many years of experience with Bank-financed projects. 2.5 Post-completion Operation/Next Phase 37. As of Project completion, PHI’s phase of expanding enrollment has been virtually completed and the focus ahead is bound to change towards the organization and functioning of the PHI. In particular, to ensure the efficient administration of the PHI benefits, there is a need to continue working with the REPSS who administer PHI benefits at the state level, and to reform the incentives and accountability mechanisms linking these agencies with the Federal Government, States, and health providers, particularly primary health care providers. 10 In 2013, the legislative chambers approved a change in the General Health Law 11 which aims to improve the efficiency of the allocation of Seguro Popular resources in the States, while strengthening the capacity of the Commission to supervise the use of the resources in the States. An additional agenda is to start to address the fragmentation of the Health System. While the World Bank currently does not have a loan in Mexico, it continues its policy dialogue with the Ministry of Finance, the Ministry of Health and IMSS around improving the efficiency and coordination between the different insurance schemes in the short term, and possibly integration of the system in the long run. The Bank engages in these areas through analytical work and convening services under the umbrella of a Programmatic Knowledge Services program. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 38. The Project’s objectives remain highly relevant for Mexico at the time of completion. Progress towards effective universal healthcare, reduction of health disparities, and achievement of the MDGs are all part of the new administration’s goal of an inclusive Mexico. The 2013-2019 National Development Plan reinforced the importance of health and included the following objectives: (i) to secure access to health services, including progress toward the establishment of a universal health system, (ii) make protection, promotion and prevention a priority for the improvement of the population’s health profile, (iii) improve health attention for the vulnerable population, (iv) guarantee the effective access of quality health services, and (v) promote international health cooperation. 39. In recent years, there has been a growing movement across the globe for UHC ensuring that everyone who needs health services is able to get them, without undue financial hardship. (WHO and WBG 2013) UHC is being pushed as one of the possible goals of the post-2014 development agenda, and therefore insurance schemes like the PHI are bound to become the center of a vivid internationally debate. (The Lancet Commission 2013) The fact that the Mexican Government moved on UHC 10 years ago makes it a pioneer in this global agenda. 40. Project objectives support key health MDGs and are tightly intertwined with the World Bank’s new goals to reduce extreme poverty and promote shared prosperity. The FY 2014-2019 CPS (Report No 80800-MX, October 23, 2013) calls for a series of tailored engagements, which allow the World Bank to play an effective role in reducing extreme poverty and promoting shared prosperity. Access to health is crucial to promote shared prosperity, as the effort expended in reducing opportunity inequality will lead to less outcome inequality in the future. 41. The PHI has made a marked contribution towards closing gaps in health coverage and financing between socio-economic groups. While 10 years ago half of the population lacked health insurance and the poor were significantly more likely to lack coverage, 11 Articles 77 bis 5, bis 6, bis 11, bis 15, and bis 16. 11 currently the gaps in health insurance enrollment between socio-economic quintiles have been virtually eliminated. (Gutierrez and Hernandez-Ávila 2013) The Social Protection System in Health and specifically the PHI significantly narrowed health-spending gaps. This narrowing of coverage and financing gaps remains essential to reduce poverty and promote inclusion. (See Section 3.2) 42. As evidenced in the PAD, there is generally some concern that programs like the PHI may generate some labor market distortions. The concern is that by offering benefits that are comparable to those of the contributory regime, workers may choose to stay in the informal sector even when they have an option of moving to the formal sector which has a contributory health insurance. Several studies were undertaken during the life of the Project to evaluate this concern. Most found a relatively small impact overall, with a greater reduction in the probability of formal employment in rural areas. (Duval Hernandez and Smith Ramirez 2011) 3.2 Achievement of Project Development Objective 43. Following the Project Development Objectives, the Project efficacy should be evaluated for the achievement of the following outcomes: (i) number of people without contributory social security that are affiliated to PHI; and (ii) capacity of the Commission and State Health Systems to effectively administer the entitlements of the PHI. Given the respective initial allocations of funds between the two objectives, the ICR puts a higher weight on PDO 1 to calculate overall Project efficacy. 44. The Project’s efficacy was high in its objective to preserve and expand the coverage of the PHI for people without contributory social security. By the Project’s completion, the PHI counted 55.6 million affiliates, a 78 percent increase over the baseline and far exceeding the end-of-Project target of 48.4 million (PDO indicator 1). The rapid growth in affiliation was facilitated by the change in coverage focus from the family unit to the individual. The majority of PHI affiliates appear to be individuals who have no access to alternative health service coverage, as required by the PHI eligibility criteria. On the basis of the Household Income and Expenditures Survey- Encuesta Nacional de Ingresos y Gastos de los Hogares - ENIGH), we estimate that the amount of filtration (affiliation to PHI of individuals with contributory health insurance) was 9.5 percent in 2010 and 11.3 percent in 2012. 12 This number is consistent with Technical Audit of the Project, which finds that the list of affiliates is relatively reliable. (COLMEX 2014) (Annex 2) 45. The Project contributed to the PHI reaching a high number of affiliates within a shorter timeframe. While the World Bank loan only financed a portion of the cost of the PHI program, it did signal the World Bank’s support to the Program and helped protect its funding during the financial crisis. Without the Project, PHI coverage would most likely have progressed at a slower pace. 12 Authors’own calculations. 12 46. The expansion in PHI coverage disproportionately benefited vulnerable groups and fostered equity of health care coverage. The Project identified four vulnerable groups whose affiliation to PHI was more closely monitored: the poor (i.e. individuals in deciles 1 and 2), beneficiaries of the Oportunidades program, indigenous populations, and females. As shown in Annex 2, the majority of PHI affiliates belong to socio-economic quintiles 1 and 2, and affiliation rates rose steeply in the lowest quintiles. The number of Oportunidades beneficiary families affiliated with the PHI (intermediate indicator 4) grew from 3.06 million to 5.7 million, compared to the original target of 4 million. The number of individuals affiliated with the PHI residing in areas where more than 40 percent of the population speaks an indigenous language (intermediate indicator 5) grew from 4.13 million to 6.9 million, against the original target of 4.8 million. Finally, the number of women and girls affiliated with the PHI (intermediate indicator 6) increased from 16.9 million to 30 million, much above the original target of 22 million. 47. The Project’s efficacy was substantial in its objective to improve the capacity of the Commission and State Health Systems to effectively administer the entitlements of the PHI. We use the following outcome measures to make this assessment: effectiveness at providing preventive screenings, affiliates’ understanding of the PHI program, and concrete actions to improve administration of entitlements. This assessment keeps in mind that the original objective of the Project in this area was a relatively modest one. 48. PDO indicator 3 can be viewed as a tracer indicator for effectiveness of coverage and measures the number of individuals affiliated with the PHI who have received a preventive health screening. Health prevention is particularly important in Mexico as the country experiences an epidemiological shift, with the population increasingly struggling with obesity and diabetes, and as health care costs rise. Over the longer term prevention strategies can enhance the health profile of the population and lower health care cost. During the implementation of the Project, the number of PHI affiliates that had received health screenings increased from zero in 2009 to 22.9 million, amply surpassing the 10 million target. 49. For PHI coverage to be effective, affiliates need to understand their rights and obligations. While the Project indicators reveal some progress in this area, there is still an outstanding agenda to ensure beneficiary awareness. According to the 2012 ENSANUT survey, 74.3 percent of PHI affiliates reported having received enough information to know their rights and obligations under the PHI. (PDO Indicator 4) This is above the baseline number of 71.2 percent but below the Project’s target of 80 percent. The Project’s intermediate indicators show that several interventions were undertaken to increase affiliates’ understanding of their rights, but that this did not necessarily translate into “sufficient” knowledge, according to beneficiaries. The Technical Audit finds that the degree to which affiliates know their rights depends on the type of benefit, in the sense that basic rights are better known than rights related to catastrophic services. (Colmex 2014) Lazcano et al. (2012) find that affiliates in the poorest States generally received much less information about their rights than those in richer states. 50. During Project implementation, the Commission implemented a number of strategies to improve the administration of the PHI entitlements. First, the Commission increased its supervision of PHI in the States in four core action areas (affiliation and operation, health service management, financing and Oportunidades program). With this strategy, the 13 Commission aimed to strengthen the implementation of the health norms signaled by the administration, and in particular those related to health prevention and rights of the affiliates. Second, the Commission took a number of concrete steps to strengthen the administration of the Program in the States, including the hiring of medical managers, management agreements with the states, and the development of indicators for monitoring at the state level. Third, the Commission invested in improving health information systems. It allocated increased amounts of funding for technical assistance to improve the collection and analysis of information in the states. The corresponding intermediate indicator 8 surpassed its target. It also invested in improving its health information systems. For example, by 2013, the Commission had rolled out an individualized health information system to support preventive health screenings in the 32 States (intermediate indicator 14), while 32 States had started capturing biometric information of individuals affiliated to the PHI Program. 51. The Project also contributed to the Commission securing funding for a wide range of technical assistance and studies. Approximately 11 studies were mapped to the Project though the Commission actually carried out more than 45 different studies in 2010-2012 valued at more than US$26 million, which focused on a range of issues, including the knowledge of beneficiaries regarding their entitlements, the design of existing mechanisms, and the evaluation of the PHI in general. A number of these studies led to the review and refinement of the federal monitoring system, assessed the capacity of the State Health Systems to monitor the delivery of health services to different groups of affiliates (in particular the indigenous population), supported disclosure (through satisfaction surveys and external evaluations) and the development of performance benchmarks (through the establishment of State-level base lines for a number of PHI related indicators). Component 2 also supported the development of instruments to capture and validate information through the expansion of Consulta Segura (PDO indicator 3), which captures affiliates health related information, and the expansion of SINOS. The Commission worked closely with both the Bank and the Center for Economic Research and Teaching (Centro de Investigación y Docencia Económica - CIDE) on dissemination of the studies. While the loan did not disburse the funds that were allocated to Component 2, nevertheless the Project contributed to the achievement of the objectives outlined in the component. Contribution to higher level outcomes 52. As mentioned in Section 2.1, the Project aimed to support Mexico in achieving universal health coverage. Therefore it is useful to look beyond the Project’s original indicators (which mostly look at population coverage) to evaluate whether there were any impacts on financial protection and on the quantity and quality of health services, and on final health outcomes. 53. As the number of PHI affiliates increased, public spending on health increased while beneficiary financial protection was improved. Federal and State expenditure on PHI (intermediate indicator 1) increased from US$3.4 billion in 2009 to over US$5.2 billion in 2013, significantly above the target of US$4.5 billion. Public spending on health grew from 2.55 percent of GDP in 2003 to 3.1 percent in 2012. (Table 3) While the trend is positive, total health expenditure per capita (US$977) and as a percentage of GDP (6.2 percent) remain significantly below the OECD averages (US$3,322 and 9.3 percent respectively). (OECD 2013) At the 14 individual level, the expansion of PHI improved beneficiary financial protection. A randomized study by King et al. (2009) estimated that the reduction in OOP expenditures was 915 pesos per year. Avila-Burgos et al (2012) find that PHI reduced the probability that a household would spend OOP for health by 2 percentage points, from a baseline probability of 52 percent. In addition, they find that PHI reduced the likelihood of catastrophic health expenditures by 36.8 percent. 13 54. The range of covered services and their utilization rose considerably during the Project’s implementation. Between 2009 and 2013, the gap in the benefits packages offered by Social Security and PHI gradually reduced, as PHI started to cover an increasing number of health interventions and medicines. Still, the availability of services continues to be an issue. According to CONEVAL (2014), the supply of services has not kept up with the number of affiliates. On the utilization side, there is some evidence of increased utilization of services, and that the Program has led to a shift in services from the private sector to the public sector. However, further information would be needed on the quality of services in the public sector to evaluate whether this shift was a positive development. 55. The evidence regarding the impact of PHI on health outcomes is mixed. A randomized study by King et al. (2009) found a 23 percent reduction from the baseline in catastrophic expenditures, but no effects on medication spending, utilization or health outcomes, possibly due to the short duration of the evaluation (10 months). All other evaluations of the Program use non-experimental methods and results vary. Most studies do not find large effects on standard indicators of maternal and child care usage, possibly because the baseline levels were high to start with. By contrast, in a recent paper, Pfutze (2014) uses a weighted exogenous sampling maximum likelihood (WESML) estimator and finds that PHI can be expected to reduce Mexico’s infant mortality rate by close to 5 out of 1,000 births. A few evaluations look at the impact of PHI in those areas that now form 90 percent of the burden of disease, i.e. non- communicable diseases and injury. (Knaul et al.) Teruel et al (2014) use the 2002 and 2009-2011 rounds of the Mexican Family Life Survey (MxFLS) and find positive impact of the Program on health services, detection and diagnosis for urban women and rural men. They also find an improvement of health status for urban women, rural men, and children in areas related to chronic conditions. (See Annex 4) 56. Part of the explanation for the mixed evidence on the link between PHI and health outcomes may be that funding to the states for PHI ends up strengthening the entire health system in the state, which benefits not only PHI beneficiaries. The comparison between PHI affiliates to non-affiliates within States may be biased because non-affiliates may benefit from improvements in health services as well. For example, some states such as Puebla have made particularly impressive progress on the MDGs such as maternal mortality (Annex 2, Box 1) and have used PHI funds to finance their intervention strategies. In practice, women who need emergency services and are not affiliated are affiliated at the time they need the service. This kind of strategy makes it challenging to compare affiliated and non-affiliated populations to 13 Avila et al (2012) define catastophic health expenditures those that exceed 30 percent of the total expenditures of the household. 15 extract the impact of the program. Still, overall the numbers seem promising. For Example, Knaul et al. (2012) find that the gaps in maternal and under-5 mortality have been reduced between those who have access to social security (i.e. the formally employed and their immediate family members), and those who don’t. 3.3 Efficiency 57. To break-even, the PHI program should have achieved an 11 percent reduction in the overall burden of disease, which appears to be modestly likely given the available evidence. A number of studies have looked into the impact of PHI on health outcomes. A randomized study by King et al. (2009) assessed the impact of rolling out PHI, and found a 23 percent reduction in catastrophic expenditures. The evaluation found that the Program’s resources reached the poor, but that the program did not show other effects, possibly due to the short duration of the evaluation (10 months). All other evaluations of the program used non- experimental methods and results vary. For example, Gakidou et al (2006) and Scott (2006) find that SP beneficiaries have higher utilization rates than the uninsured. Bernal et al (2010) found a 3 percentage point increase in the probability that deliveries in the rural areas were attended by a medical doctor. Estimates suffer from attenuation bias and are likely to underestimate the impact of the Program. In a recent paper, Pfutze (2014) found that PHI can be expected to reduce Mexico’s infant mortality rate by close to 5 out of 1,000 births. (Annex 3) 3.4 Justification of Overall Outcome Rating 58. The overall outcome rating is Satisfactory. The rating is based on the following: (i) Project consistency with Government priorities; (ii) high relevance of the objectives and design to the Mexican country context and the global context; (iii) high efficacy in achieving PDO 1 and substantial efficacy in achieving PDO 2; and (iv) modest efficiency. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 59. The Project provided health coverage to those lacking health insurance, which are often the most vulnerable members of society. The PHI program is highly progressive and has contributed to closing inequities in health financing. In fact, the PHI and Oportunidades are among the most progressive programs in Mexico. 60. Women have benefitted from the additional programs financed through the PHI aimed at women and children. The PHI provides improved access to health services, and gives pregnant women, mothers and children priority access to health care services through a Strategy for Healthy Pregnancies (Estrategia de Embarazo Saludable) and the Health Insurance for a New Generation (Seguro Médico para una Nueva Generación) as well as priority enrollment to the PHI. 61. The Project had a significant impact on the Indigenous population. As Section 2.4, the IPP worked to overcome identified obstacles to affiliation of Indigenous populations. The 16 Commission promoted an intercultural approach, and cooperated with the Commission for Indigenous Development (Comisión Nacional para el Desarrollo de los Pueblos Indígenas – CDI) to launch visual and radio advertisements in different languages. The strategy was clearly successful as reflected in the increase in affiliation. (b) Institutional Change/Strengthening 62. The Project helped to strengthen the stewardship role of the Ministry of Health. The expansion of the role of the General Health Council (Conseco de Salubridad General), a collective decision making body that spans all participants in the health sector, including the Commission, was key in reinforcing the stewardship role of the Ministry of Health. In addition to setting the National Health Priorities, the General Health Council assesses the inclusion of interventions into CAUSES and the Catastrophic Fund. The 2013 change in the General Health Law (See Paragraph 37) aims to strengthen the capacity of the Commission to supervise the use of the resources in the States. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 63. The 2012 satisfaction survey of PHI affiliates 14 found a 97 percent satisfaction with PHI, with a high satisfaction for all service components (affiliation process, medical attention, etc.). Ninety-nine percent of those surveyed planned to re-affiliate, with most providing the fact that service is free, that they receive good attention and that they don’t have access to another health insurance as the main reasons for re-affiliation. Most interviewed felt they received a detailed explanation of what they were suffering. The average wait time for receiving attention was 104 minutes, with variations between states from 69 minutes to 157 minutes. Most affiliates say they would return to the same health center. However, less than half of the affiliates received information concerning their rights at the time of affiliation. Indigenous affiliates rated satisfaction with PHI in the satisfaction survey between 8.5 and 9.7 (total of 10). While the surveys show high satisfaction levels, the survey does not fully represent the population of PHI affiliates because it selected PHI affiliates at the point of service exit. Hence the survey implicitly excluded PHI affiliates who did not choose to attend PHI health centers or those that went to alternative providers; arguably these could be the PHI affiliates that are less satisfied or have given up on receiving services from PHI providers. 4. Assessment of Risk to Development Outcome Rating: Negligible 64. Reduced financing of eligible federal contributions to the PHI premiums would have a high impact on the PHI. However, the likelihood of this happening is low. Though the 14 The 2012 survey is the latest available. The Ministry of Health carries out a satisfaction survey for PHI affiliates on a biannual basis. 17 current administration may try to change aspects of the SPSS, the elimination of the PHI is unlikely. The PHI is grounded on the rights and principles enshrined in the Mexican Constitution and established by the 2003 General Health Law, which established the PHI to ensure that all individuals, including those lacking contributory social security, are equally able to exercise this right. The PHI enjoys broad, non-partisan political support, in its eleventh year of operation and has survived two presidential election cycles. The 2013 change in General Health law (See Paragraph 37) continues to support the role of PHI and the Commission in reaching the population’s health needs. According to the 2013-2018 NDP and PROSESA, the Commission will continue working to guarantee individual’s access to quality and effective health care and to narrow inequalities in terms of coverage between the PHI, the Mexican Social Insurance Institute (Instituto Mexicano del Seguro Social – IMSS) and the National Institute for the Social Security of Government Workers (Instituto de Seguridad y Servicios Sociales de los Trabajadores del Estado - ISSSTE). 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 65. The Project’s design built on findings of earlier sector work and the objectives were relevant and strongly supported Government and global priorities. The Project was prepared and approved in a six-month period, as agreed with the Government, which required a fast paced design and the selection of indicators available at the time. Weaknesses in the M&E framework negatively weigh on the rating, although they did not lead to issues in terms of the implementation and neither did they prevent the measurement of project achievement at the time of the ICR. (b) Quality of Supervision Rating: Moderately Satisfactory 66. The Project benefitted from a committed World Bank team and a close relationship with the SPSS, despite the fact that it had three different TTLs. World Bank staff worked closely with the Commission discussing and modifying areas requested by the GOM for the two restructurings, and mobilizing global knowledge on various topics through convening services. The World Bank also supervised and provided assistance in the implementation of technical and financial audits, and restructured the Project in 2011 to adjust the M&E Framework. While some improvements were made, the improvements could have been more thorough to include the issues identified in Section 2.3. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 67. For the reasons indicated above, the Overall Bank Performance is rated Moderately Satisfactory. 18 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 68. Government’s performance is rated Satisfactory due to the high level of support given to the Project. As described previously, the Government was very committed to the Project. Commitment waned somewhat following the change in administration in 2013, but by that time the Project had reached the end of its cycle since it had been nearly fully disbursed. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 69. Implementation agency performance is rated Moderately Satisfactory. The Commission was responsible for the Project’s implementation. During the Project, the Commission managed a 24.5M increase in affiliates without major obstacles, because the financial architecture, control mechanisms and management structure were already in place. The Commission’s agenda of studies was ambitious and revealed strong commitment to the PHI. Delays in the delivery of both technical and audit reports by the Commission revealed weaknesses in the Project’s management. Finally, as discussed in Section 2.3, there were some weaknesses in the reporting of indicators. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 70. For reasons outlined above, the overall Borrower Performance is rated Moderately Satisfactory. 6. Lessons Learned UHC Financing in a Decentralized Context 71. Universal Health Coverage consists of three inter-related components: (i) coverage for the entire population; (ii) financial protection from direct payment for health services when consumed; and (iii) the full spectrum of quality health services according to need. (WHO 2010) The Project funds were mostly used to address the first two dimensions, though the Project did contribute with small steps towards the reform of the health financing systems within the States. Projects that aim at UHC should consider the 3 dimensions of the UHC “cube”, and include monitoring indicators for all three dimensions. Empowering affiliates of insurance systems serving the poor and vulnerable. 72. Introduction of the SPSS did not eliminate the fragmentation of the Mexican health system, which includes several health insurance schemes covering formal workers, the SPSS, IMSS-Oportunidades and the private sector. The horizontal segmentation of the health system is considered one of the main impediments to improving its performance, in 19 particular in some states. Though the system has been reorganized, with an alignment of the financial architecture and federal stewardship, reform across functions would improve efficiency. In addition, now that a majority of the vulnerable population has been affiliated, the Government should combine programs such as Siglo XXI, the health part of Oportunidades, etc. 73. While PHI has succeeded in narrowing health funding differences between the States, there are still significant differences in performance and outcomes, which largely mimic socio-economic differences amongst the states. Linking funding to an increase in enrollment based on a uniform per capita amount will not succeed in bringing up the poorest States, because they have a significant lag in services and infrastructure. To achieve UHC, PHI may require a differentiated per capita amount and specific performance targets to bring those states up to minimum service delivery levels. 74. There is a need to rethink incentives for the states and their providers to improve health care performance. Although this was one of the objectives of the 2003 reform, there has not been much progress in this regard (Kurowski and Villar-Uribe 2012). At the state level, in most cases provider payments continue to be based on historical budgets rather than on some measure of performance. States are not held accountable on how or where the money is spent, and the States’ weak record in service delivery may be partly due to the lack of accountability mechanisms. The Commission is, however, evaluating options to use the existing information systems, including the Personalized Health Registry (SINOS) as the basis to generate these incentives. Overall, there is still a need for a comprehensive incentive and support system to support reform in the highly decentralized Mexican context. Project’s Design 75. The Project had a dual aim of protecting financing for PHI (which required fast disbursement) and providing technical assistance (which required long-term engagement). While Development Policy Loans (DPLs) are fast-disbursing, their disburse-and-close modality is not well suited for long-term engagement and technical assistance to complex reform processes. At the same time, Investment Lending allows for longer engagement and can also include fast-disbursing components, as was the case in this Project. However, in loans that incorporate a significant component of technical assistance, procurement arrangements are crucial to ensure that the component will be executed as expected. Where appropriate, procurement arrangements should use country systems. 76. Even in Programs that aim at universal coverage, there is a need for specific approaches and strategies to reach the most vulnerable segments of the population. As demonstrated in this Project, effective Indigenous Peoples Plan and other targeting strategies can help make universal health programs inclusive. 77. Evidence generation and regular monitoring and evaluation are crucial for policy, strategy and program decision-making. The numerous surveys carried out in Mexico, such as the ENSANUT, the ENIGH, satisfaction surveys, and other surveys provide significant information on the health profile of the society and the provision of health services. Yet this information was not necessarily used to the fullest in the Project’s M&E setup. Overall, the 20 M&E framework would have benefited from having fewer and simpler indicators that covered the main dimensions of UHC (coverage of people, financial protection and availability of quality services). However, the Government was unwilling to make significant changes to the monitoring framework during project implementation. Documentation of the source of data and method of calculation are essential to ensure clarity about project achievements. Globally, having more agreement on metrics for universal health coverage would facilitate M&E at the country level and foster comparability among countries. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers N/A (c) Other partners and stakeholders N/A 21 Annex 1. Project Costs and Financing (a) Project Cost by Component in USD Million equivalent. Amount in brackets refers to the World Bank financing. Appraisal Actual/Latest Percentage of Estimate (USD Estimate (USD Appraisal millions) millions) Component 1:Inititally preserve and later expand the Popular Health Insurance 26,825.000 26,043.130 15 97% coverage of people without contributory (1,239.000) (1,246.875) social security Component 2: Strengthen the capacity of the National Commission for Social Protection in Health (the Commission) and 32.875 26.000 79% State Health Systems to effectively (7.875) (0.000) administer the entitlements of the Popular Health Insurance Total Baseline Cost 26,857.875 26,069.130 97% Physical Contingencies 0 0 - Price Contingencies 0 0 - Total Project Costs 26,857.875 26,069.130 97% Front-end fee PPF - - - 3.125 3.125 Front-end fee IBRD 100% (3.125) (3.125) 26,861.000 26,072.255 Total Financing Required 97% (1,250) (1,250) (b) Financing Appraisal Actual/Latest Percentage of Source of Funds Estimate Estimate Appraisal (USD millions) (USD millions) Government 25,611.000 24,822.255 97% International Bank for 1,250.000 1,250.000 100% Reconstruction and Development Total Financing 26,861.000 26,072.255 97% 15 Total spending on Premiums was estimated by combining the amount transferred from the Federal level to the states in 2010, 2011, 2012, and 2013, and the estimated contribution of the states for the same years. Source: SPSS (2013, graphs 7.4 and 7.5) 22 Annex 2. Outputs by Component (a) Achievement of PDO and intermediate indicators 78. The Project had four PDO indicators and 15 intermediate indicators to track Project progress. By the Project’s closing, three of the four PDO indicators had exceeded end- of-Project targets, while the fourth was not achieved, although some progress toward the target can be observed. Twelve of the 15 intermediate indicators were successfully achieved. Table 1: Project PDO Indicators Assessment PDO Indicators Baseline Target Final % of Value Achieve achievement -ment 1 Number of individuals affiliated 31.1M 41.1M 55.6M 245% Surpassed with PHI as a percentage of the total number of individuals that are not (+10M) (+24.5M) affiliated with a contributory social security system. 2 Percentage of recommended actions 0% 71% 90.3% 127 % Surpassed implemented by federal entities resulting from the supervision action plan carried out by the Commission in its four action areas 3 Number of individuals affiliated 0 10 M 22.8M 228 % Surpassed with the PHI who have received a “Health Risk Screening” 4 Number of individuals affiliated 71.2% 80% 74.3% 35 % Not with the PHI that report having achieved received enough information to (+8.8 pp) (+3.1 pp) know their rights and obligations as a percentage of the total number of individuals affiliated with the PHI. 79. The Project’s efficacy was high in its objective to preserve and expand the coverage of the PHI for people without contributory social security. By the Project’s completion, the PHI counted 55.6 million affiliates, a 78 percent increase over the baseline and far exceeding the end-of-Project target of 48.4 million (PDO indicator 1). The rapid growth in affiliation was facilitated by the change in coverage focus from the family unit to the individual. The majority of PHI affiliates appear to be individuals who have no access to alternative health service coverage, as required by the PHI eligibility criteria. On the basis of ENIGH, we estimate that the amount of filtration (affiliation to PHI of individuals with contributory health insurance) was 9.5 percent in 2010 and 11.3 percent in 2012. 16 This number is consistent with Technical Audit of the Project, which finds that the list of affiliates is relatively reliable. (COLMEX 2014) Specifically, the Audit finds that in 13.8 percent of the family units covered by PHI, at least one member was covered by an alternative health insurance scheme. However, this does not mean 16 Authors’ own calculations. 23 that all members of the household are ineligible for PHI, since other health insurance schemes do not necessarily allow the entire family unit to be enrolled, even if one member is enrolled. Therefore, the number of ineligible individuals that are enrolled in PHI is likely lower than 13.8 percent. Figure 1: Number of PHI Affiliates, 2002-2013 17 60 55.6 51.8 52.9 50 43.5 40 31.1 30 27.2 21.8 20 15.7 11.4 10 5.3 0 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Number of Affiliates (Millions of Individuals) 80. The Project contributed to the PHI reaching a high number of affiliates within a shorter timeframe. While the World Bank loan only financed a portion of the cost of the PHI program, it did signal the World Bank’s support to the Program and helped protect its funding during the financial crisis. Without the Project, PHI coverage would most likely have progressed at a slower pace. 81. The expansion in PHI coverage disproportionately benefited vulnerable groups and fostered equity of health care coverage. Cultural and social factors in Mexico have put the poor, indigenous, and rural population at a disadvantage for having contributory social security and corresponding health insurance. The Project identified four vulnerable groups whose affiliation to PHI was more closely monitored: the poor (i.e. individuals in deciles 1 and 2), beneficiaries of the Oportunidades program, indigenous populations, and females. 18 According to the ENIGH survey, between 2008 and 2012, affiliation rates to PHI rose by approximately 30 percentage points in Quintiles I and II, while affiliate rates for higher quintiles did not rise as strongly. (Figure 2) As shown in Figure 3, the majority of PHI affiliates belong to socio- economic quintiles 1 and 2. On the basis of ENIGH, at the time of the ICR we estimate that the percentage of individuals in deciles 1 and 2 that were affiliated to PHI, as percentage of those in deciles 1 and 2 that were not affiliated to a contributory social security system, went from 17 Administrative data as reported in CPSS 2014 18 Section 2.3 discussed a number of issues related to the calculation of coverage rates for different populations. This ICR evaluates the project’s achievements based on absolute numbers and therefore it cannot make any judgements about population coverage rates. 24 42.30% (2008) to 72.32 % (2012), a 30 percentage point increase. (ICR alternative to intermediate indicator 3) According to Gutierrez (2013), the 2012 ENSANUT shows that 20.1% of the population in the lowest quintile (i.e. deciles 1 and 2) had no health coverage (contributory or non-contributory), down from 58.8% in 2006. The number of Oportunidades beneficiary families affiliated with the PHI (intermediate indicator 4) grew from 3.06 million to 5.7 million, compared to the original target of 4 million. The number of individuals affiliated with the PHI residing in areas where more than 40 percent of the population speaks an indigenous language (intermediate indicator 5) grew from 3.5 million to 6.9 million, against the original target of 4.8 million. Finally, the number of women and girls affiliated with the PHI (intermediate indicator 6) increased from 16.9 million to 30 million, much above the original target of 22 million. Figure 2: Percentage of Individuals affiliated to PHI, by Socioeconomic Quintile 19 80% 70% 68% 60% 58% 53% 50% 45% 43% 2008 38% 40% 31% 2010 29% 30% 30% 2012 22% 18% 16% 20% 13% 11% 10% 6% 0% Quintile I Quintile II Quintile III Quintile IV Quintile V 19 Source ENIGH (2008, 2010 and 2012) and authors’ own calculations. Socio-economic quintiles are based on trimestral current income. 25 Figure 3: Socio-economic Composition of PHI Affiliates (ENIGH 2012) 20 Quintile V, 8.5% Quintile IV, 15.4% Quintile I, 27.2% Quintile III, 22.2% Quintile II, 26.6% 82. The Project’s efficacy was substantial in its objective to improve the capacity of the Commission and State Health Systems to effectively administer the entitlements of the PHI. We use the following outcome measures to make this assessment: effectiveness at providing preventive screenings, affiliates awareness of the PHI program, and concrete actions to improve administration of entitlements. This assessment keeps in mind that the original objective of the Project in this area was a relatively modest one. 83. PDO indicator 3 can be viewed as a tracer indicator for effectiveness of coverage and measures the number of individuals affiliated with the PHI who have received a preventive health screening. Health prevention is particularly important in Mexico as the country experiences an epidemiological shift, with the population increasingly struggling with obesity and diabetes, and as health care costs rise. Over the longer term prevention strategies can enhance the health profile of the population and lower health care cost. During the implementation of the Project, the number of PHI affiliates that had received health screenings increased from zero in 2009 to 22.9 million, amply surpassing the 10 million target. 84. For PHI coverage to be effective, affiliates need to understand their rights and obligations. While the Project indicators reveal some progress in this area, there is still an outstanding agenda to ensure beneficiary awareness. According to the 2012 ENSANUT survey, 74.3 percent of PHI affiliates reported having received enough information to know their rights and obligations under the PHI. (PDO Indicator 4) This is above the baseline number of 71.2 percent but below the Project target of 80 percent. The Project’s intermediate indicators show that several interventions were undertaken to increase affiliates’ understanding of their rights, but that this did not directly translate into “sufficient” knowledge, according to beneficiaries. For example, the Program designed and distributed information materials on PHI rights, responsibilities and affiliation processes through the Oportunidades program (intermediate indicator 12). Also, in the PHI beneficiary satisfaction survey, 89 percent of PHI affiliates reported that they had received information at the time of affiliation with regard to their right to 20 Source: ENIGH (2012) and authors’ own calculations. Socio-economic quintiles are based on trimestral current income. 26 not pay service fees (intermediate indicator 11), higher than the targeted 80 percent. The percentage of PHI affiliates who reported having received a bill of rights and responsibilities at the time of affiliation (intermediate indicator 9) fell to 77.7 percent from the baseline of 81.9 percent, and missed the target of 88 percent. The percentage of PHI affiliates who reported having received a catalogue of their benefits package at the time of affiliation (intermediate indicator 10) fell to 74.3 percent from the baseline of 77 percent, and missed the target of 83 percent. The Technical Audit finds that the degree to which affiliates know their rights depends on the type of benefit, in the sense that basic rights are better known than rights related to catastrophic services. For example, over 97 percent of affiliates are aware of their right to not pay co-payments; however less than half of family units are aware of their right to enroll children under 5 in Siglo XXI, a catastrophic health insurance for children. (COLMEX 2014) 85. During the implementation of the Project, the Commission implemented a number of strategies to improve the administration of the PHI entitlements. 86. First, the Commission increased its supervision of PHI in the states in four core action areas (affiliation and operation, health service management, financing and Oportunidades program). With this strategy, the Commission aimed to strengthen the implementation of the health norms signaled by the administration, and in particular those related to health prevention and rights of the affiliates. Intermediate indicator 7 shows that the Commission supervised all 32 states in at least one of the four core action areas of during a calendar year. PDO indicator 2 shows that up to 90 percent of the recommended actions resulting from the Commission’s supervision were implemented by the states. This translated into successful expansion of health prevention activities and promotion of the rights of affiliates. 87. Second, the Commission took a number of concrete steps to strengthen the administration of the program in the states. The Commission strengthened the role of medical managers (gestores medicos) who support affiliates in accessing care; the Commission aimed to have one medical manager per 30,000 affiliates. An agreement with the State Ministries of Finance of all federal entities was signed in 2013 to work towards the timely transfer of PHI resources to service providers. The Commission also supported the development of financial monitoring indicators to ensure that resources are used for intended purposes and the development of effective and quality coverage indicators. 88. Third, the Commission invested in improving health information systems. It allocated increased amounts of funding for technical assistance to improve the collection and analysis of state health information. The corresponding intermediate indicator 8 surpassed its target. Spending increased to US$8 million as of end-2013 from a base of US$1.7 million, nearly US$1 million over the target. The Commission also rolled out an individualized health information system, SINOS, 21 to support the preventive health screenings in 31 of the 32 states 21 SINOS has been upgraded and now has a third module called Health Care, which is a primary health care medical record. The Health Care module was originally only compulsory for the beneficiaries of the Oportunidades program and used as a verification mechanism of responsibilities in health. The Commission has been piloting SINOS so that it can be used for the verification of 27 (intermediate indicator 14), against the target of 10 states, and 31 out of the 32 states started capturing biometric information of individuals affiliated to the PHI Program. (intermediate indicator 15) As of December 2013, had over 27.7 million biometric registries, approximately 64 percent of affiliates. Overall, the preventive health screening program and associated data should help the Commission and the states to manage better manage health risks and support the health sector’s fiscal sustainability. 89. The Commission’s work on the development of SINOS strengthens its own and REPSS capacity to manage health risks of PHI affiliates. SINOS aims to: (i) determine the risk profile of PHI affiliates and generate relevant information for the planning of the SPSS, (ii) optimize the use of the resources transferred to States, (iii) strengthen the prevention schemes for the population affiliated to PHI, (iv) track personalized health actions based on the National Health Card and (v) strengthen mechanisms for monitoring compliance mechanisms. Information generated through SINOS will gradually be consolidated at various levels (national, state, judicial, municipal and health unit). The system requires the coordinated effort of the REPSS and the State Health Services for integrating the information through a control panel. In addition to determining affiliates risk profile, the system aims to generate information for planning purposes, optimize resource targeting, strengthen prevention schemes, track personalized health actions based on the National Health Card and strengthen monitoring compliance mechanisms for Oportunidades. 90. Fourth, the Commission financed a wide range of studies. Approximately 11 studies were mapped to the Project though the Commission actually carried out more than 45 different studies in 2010-2012 valued over US$26 million (See Table 2). The studies focused on a number of the issues listed in the PAD, such as the knowledge of beneficiaries regarding their entitlements, the design of existing mechanisms, and the evaluation of the PHI in general. However it is difficult to link the wide array of studies with concrete improvements in the Commission and State Health Systems capacity to manage health risks, enhance performance management or institute reforms. The data used generally measured the period preceding the Project’s implementation and in some cases proved too short a period to demonstrate an impact, however the studies do provide ample evidence regarding the initial impact of PHI on utilization, access and efficiency, etc. The majority of the studies were carried out by CIDE, while some were carried out by the National Autonomous University of Mexico (Universidad Nacional Autónoma de México, UNAM), the Autonomous University of Chapingo, the University of Guadalajara and the College of Mexico (Colegio de México – COLMEX). Table 2: Mapping of Studies Carried Out by the Commission to Project Objectives Studies carried out by the Results and recommendations Commission (a) Support enhancements in performance management in the administration of the PHI co-respoonsibilities country-wide. Nevertheless the verification mechanism has been delayed because of technical issues related to the size of the database. 28 Studies carried out by the Results and recommendations Commission Analysis of the regulation The study completed a literature review of SPSS and the health mechanism, supervision and systems in Australia, Brazil y Colombia, as well as the operation performance evaluation of the manuals that establish and define the organism, structure, SPSS and identification of process and function of the CNPSS. Four strategies were best practices identified to improve supervision and performance evaluation in the health system relating to: regulation tools, coordination mechanisms, management tools, and the use of information and communication technology Study on the consistency and The study concluded that the affiliate registry was reasonably reliability of the SPSS affiliate reliable, that the financial transfers were calculated and census (2011, 2013) transferred correctly and that Bank funds were used according to the contract conditions. Effects of PHI on Health and The study found that affiliates between 20 and 60 years of age Affiliate Spending went more to the doctor than the non-affiliated. There was a 5 percent increase in affiliates’ perception that they had good health; and a 7 percent decrease in the probability that affiliates (20 and 60 years) suffer from cancer. There was a 56 percent fall in household health expenditure. Study on the Quality of PHI Operation (b) Support State Health systems in preparing and carrying out reforms in the administration of the PHI Study to strengthen the The analysis showed how the social and economic inequality coordination, effective and existing between federal entities in Mexico is also evident in efficient administration SPSS health conditions, the availability of interventions and the human in the Federal entities (2010) resources and infrastructure capacity. The study recommends the creation of a national office dedicated to the evaluation and planning of PHI, the establishment of result-based financing mechanisms, the design of a national information system and the establishment of an efficient supervision system and the professionalization of the REPSS directives. (c) Improve the knowledge of eligible beneficiaries about their entitlements under the PHI PHI Satisfaction Survey The survey found that most affiliates were satisfied with PHI. (2011, 2012 and 2013) Evaluation of the use and protection of SPSS affiliate rights (2011) Use and health service access The study confirmed the indigenous population’s lack of of the PHI affiliated knowledge regarding their rights and responsibilities regarding indigenous population (2011) the PHI, while documenting the group’s growing affiliation and coverage. The study also revealed how health centers in indigenous areas often did not have the necessary infrastructure, personnel and medicine, while geographic, economic, cultural and administrative factors hindered health care access. (d) Strengthen their capacity to manage health risks Benefit-cost evaluation of PHI The study finds that an increase in PHI coverage lowers OOP (2012) expenditure by 0.088 percentage points (90 percent confidence level). The study also found that for every peso saved in 29 Studies carried out by the Results and recommendations Commission affiliated households in 2004 PHI spent between 1.07 and 2.2 pesos on the program (95 percent confidence level). Characteristics and Potential The study found that SINOS is a unique database which has the of SINOS: Proposals to necessary information to indicate the likelihood of suffering improve the Quality of Health from a particular disease according to the federal entity, Service municipality or clinic, which can provide useful information for decision makers. Analysis of the design and The design and implementation of Consulta Segura vary widely implementation of Consulta among the states and must be homogenized. Segura and development of an instrument to measure the strengthening a preventive culture (2011) 91. Dissemination of study results. During the mid-term review the Commission jointly with CIDE organized a seminar where some results were presented. Some of the main conclusions were the following: • Effective coverage and the impact on health conditions appear positive though difficult to measure. • PHI reduced the financial burden imposed by health expenditures, particularly catastrophic ones, to the formerly uninsured households in Mexico. • Health care utilization has been positively affected by the PHI. • The gap between public and private health spending has narrowed. • Universal health coverage is a moving target therefore continuous monitoring and evaluation is essential to introduce necessary adjustments. 92. The Bank worked closely with the Commission on the dissemination of the studies. The Bank supported the documentation and dissemination of important results and lessons learned from the implementation of the SPSS at national and international workshops, seminars and conferences in and outside Mexico. In 2012 the World Bank organized an international conference presenting numerous papers drawing on different data sets and methodologies regarding the PHI’s effects on the formality of the workforce. The forum brought together policy-makers, researchers and experts from around the world to discuss research findings about the impact of Mexico’s SPSS in health on formal sector labor force participation and debated its relevance for health systems world-wide. The World Bank also supported the Commission through the promotion of a bilateral cooperation with the German Society of Pediatric Oncology and Hematology to reduce the variability of health outcomes and costs in the care of children with cancer. As part of the cooperation, the World Bank financed travel costs of two German experts to participate in a workshop jointly organized by the World Bank and the Commission in Mexico City on December 7-8, 2011. Finally, the World Bank financed an exchange with the Argentine Plan Nacer team on Consulta Segura. (b) Contribution to Higher Level Outcomes 30 93. As mentioned in Section 2.1, the Project aimed to support Mexico in achieving universal health coverage. Therefore it is useful to look beyond the Project’s original indicators (which mostly look at population coverage) to evaluate whether there were any impacts on financial protection and on the quantity and quality of health services, and on final health outcomes. 94. As the number of PHI beneficiaries increased, federal and state expenditure on PHI (intermediate indicator 1) increased from US$3.4 billion in 2009 to over US$5.2 billion in 2013, significantly above the target of US$4.5 billion. Public spending on health grew from 2.55 percent of GDP in 2003 to 3.1 percent in 2012. (Figure 3). While the trend is positive, total health expenditure per capita (US$977) and as a percentage of GDP (6.2 percent) remain significantly below the OECD averages (US$3,322 and 9.3 percent respectively). (OECD 2013) Table 3: Trends in of Health Sector Expenditure in Mexico, 2003-2011 22 Indicator 2003 2008 2011 Trend Health expenditure, private 3.2% 3.1% 3.11% ↔ (% of GDP) Health expenditure, public 2.55% 2.7% 3.1% ↑ (% of GDP) (2012) Health expenditure public 44.2% 47% 49.4% ↑ (% of total expenditure) Out-of pocket health expenditure 52.9% 49.2% 46.5% ↓ (% of total health expenditure) 95. Due to its focus on individuals lacking social security, the PHI program is highly progressive and has contributed to closing inequities in health financing. In fact, the PHI and Oportunidades are among the most progressive programs in Mexico. (Figure 4) The difference in per capita financing for health between those without social security and those with social security decreased from 145 percent in 2004 to 51 percent in 2011. 23 In 2000 the difference between the state receiving the greatest allocation of Federal resources per person and the state receiving the least was 6.1 to 1, by 2010 the difference was 3 to 1. (Knaul 2012) 22 Source: World Development Indicators, World Bank (2012) 23 Programa Sectorial de Salud (PROSESA). Available at http://portal.salud.gob.mx/contenidos/conoce_salud/prosesa/pdf/programa.pdf, p. 38 31 Figure 4: Concentration Coefficients for Selected Programs, Mexico 2010 24 96. The expansion of PHI improved beneficiary financial protection. A randomized study by King et al. (2009) estimated that the reduction in out-of-pocket expenditures was 915 pesos per year. Avila-Burgos et al (2012) find that PHI reduced the probability that a household would spend out-of-pocket for health by 2 percentage points, from a baseline probability of 52 percent. In addition, they find that PHI reduced the likelihood of catastrophic health expenditures by 36.8 percent. 25 Knaul et al. (2012) find that the gap in financial protection between households covered by social security, and those not covered reduced between 2004 and 2010. (Table 4) While the share of OOP expenditures in total health expenditure fell gradually between 2003 and 2011, (Table 3) it remains high and suggests that the population uses private sector services either because it perceives them to be of better quality or because they continue to encounter obstacles in accessing public services. 24 Scott, 2013 25 Avila et al (2012) define catastophic health expenditures those that exceed 30 percent of the total expenditures of the household. 32 Table 4: Evolution of Gaps in Financial Protection between Social Security Affiliates and Non Affiliates 26 No Social Indicator Security Social security Double 2004 2010 2004 2010 difference OOP spending/household income 4.4 3.2 3 2.6 -0.8 OOP spending/disposable income 5.9 4.6 3.7 3.3 -0.9 HHs with catastrophic health expenditure (%) 3.6 2.8 1.5 1.4 -0.7 Households with impoverishing health expenditure (%) 2.1 1.6 0.2 0.1 -0.4 97. The range of covered services rose considerably during the Project’s implementation. Between 2009 and 2013, the gap in the benefits packages offered by social security and PHI gradually reduced, as PHI started to cover an increasing number of health interventions and medicines. Currently, the PHI catalogue of health services (CAUSES) covers 285 health interventions and 609 drugs. PHI affiliates are also covered by the Fund against Catastrophic Expenditure, which itself covers 59 groups of diseases. (Figure 5) 98. Still, the availability of services continues to be an issue. According to CONEVAL (2014), the supply of services has not kept up with the number of affiliates. For example, at 1.7 the number of hospital beds per 1000 people remains significantly below the OECD average of 4.8. (OECD 2013) According to an Inter-American Development Bank study, the Government is still unable to guarantee a universal package of benefits as many clinics and hospitals cannot provide the services that are part of a universal package. (Bosch 2012) For example, though breast cancer treatment is financed through the catastrophic fund, only 25 percent of the needed radiologists are available to meet the demand. Some states have addressed this issue by improving their referral system so that small medical health centers have the tools to refer patients when lacking the necessary service. 99. There continues to be a large disparity between States in availability of services. (CONEVAL 2014) PHI reimbursements to the states from the catastrophic fund partially hinge on the availability of accredited hospital and centers – yet there is a wide disparity in how many accredited centers the states have. The poorest States tend to be the ones with the fewest accredited facilities 27, and the largest need of funding to bring their facilities up to the required level, yet PHI funding is generally not sufficient for the States to update their facilities. There is a need for PHI to more actively monitor accreditation levels and focus attention on those states that need to catch up. 100. On the utilization side, there is some evidence of increased utilization of services, and that the program has led to a shift in services from the private sector to the public sector. Utilization of public hospitals rose from 25.9 percent in 2000 to 38.3 percent in 2012, while private sector hospital use declined from 24 percent to 17 percent. (INSP 2012) The first 26 Source: Knaul et al. (2012) 27 This is based on interviews – at the time of the ICR, we could not obtain information on numbers and trends in accreditation of facilities in the states. 33 semester 2013 evaluation of the PHI found that in urban areas there was a 30 percent increase in the number of doctor visits by affiliated urban women between 20-60 years of age. (CNPSS 2013) For children under five there has been a shift in medical attention from private and social security-linked hospitals to public hospitals. (Figure 6: Hospitalization of Children under 5: Provider Share). However, further information would be needed on the quality of services in the public sector to evaluate whether this shift was a positive development. Figure 5: Number of Interventions and Medicines Covered under PHI, 2003-2013 Figure 6: Hospitalization of Children under 5: Provider Share 101. The evidence regarding the impact of PHI on health outcomes is mixed. A randomized study by King et al. (2009) found a 23 percent reduction from the baseline in catastrophic expenditures, but no effects on medication spending, utilization or health outcomes. The evaluation found that the Program’s resources reached the poor, but that the Program did not show other effects, possibly due to the short duration of the evaluation (10 months). All other evaluations of the program use non-experimental methods and results vary. Most studies do not find large effects on standard indicators of maternal and child care usage, possibly because the baseline levels were high to start with. By contrast, in a recent paper, Pfutze (2014) uses a weighted exogenous sampling maximum likelihood (WESML) estimator and finds that PHI can be expected to reduce Mexico’s infant mortality rate by close to 5 out of 1,000 births. A few evaluations look at the impact of PHI in those areas that now form 90 percent of the burden of disease, i.e. non-communicable diseases and injury. (Knaul et al.) Teruel et al (2014) use the 34 2002 and 2009-2011 rounds of the MxFLS and find positive impact of the program on health services, detection and diagnosis for urban women and rural men. They also find an improvement of health status for urban women, rural men, and children in areas related to chronic conditions. (See Annex 4) 102. Part of the explanation for the mixed evidence on a the link between PHI and MDG health outcomes may be that funding to the states for PHI ends up strengthening the entire health system in the state, which benefits not only PHI beneficiaries. The comparison between PHI affiliates to non-affiliates within states may be biased because non-affiliates may benefit from improvements in health services as well. Some states such as Puebla have made particularly impressive progress on the MDGs such as maternal mortality (Box 1) and have used PHI funds to finance their intervention strategies against Maternal Mortality. In practice, women who need emergency services and are not affiliated are affiliated at the time they need the service. This kind of strategy makes it challenging to compare affiliated and non-affiliated populations to extract the impact of the program. Still, overall the numbers seem promising. For Example, Knaul et al. (2012) find that the gaps in maternal and under-5 mortality have been reduced between those who have access to social security (i.e. the formally employed and their immediate family members), and those who don’t. (Table 5) Table 5: Evolution of Gaps in Health Outcomes between Social Security Affiliates and Non Affiliates 28 No Social Social Indicator Security security Double 2004 2010 2004 2010 difference Under 5 mortality (per 1000 births) 25 22.3 12.8 12.1 -2 Maternal Mortality (per 100,000 live births) 72.2 48.9 28.7 27.9 -22.5 103. The expansion of health insurance coverage took place alongside an improvement in Mexico’s Millennium Development Goals (MDG) indicators. Infant and under-five mortality rates have decreased significantly over the years, and the country is on track to achieve the MDG of reducing under-five mortality (Table 6), partly due to rise in the number of births attended by skilled physicians, currently approximately 95 percent of all births. (OECD 2013). The maternal mortality rate has also improved, though it is unlikely to reach the MDG target of 22 deaths per 1000 births by 2015. (PROSESA) Table 6: MDG and Other Indicators 1 Indicator 2003 2012 Infant mortality 18.2% 13.9% Under 5 mortality1 21.3% 16.2% 1 Maternal mortality 54 (2005) 43 (2012) Hospital Beds (per 1000 people) 1 1.7 (2010) Physicians (per 1,000 people) 1.5 1.96 (2010) 28 Source: Knaul et al. (2012) 35 (c) Indigenous People’s Plan 104. The Indigenous Peoples Plan had three main objectives. The first objective was to remove health service barriers identified in the social assessment in order to (i) promote indigenous peoples enrollment taking into account specific socio-economic and cultural characteristics; and (ii) raise awareness about the importance of health insurance as well as PHI beneficiary entitlements. The second objective was to strengthen the capacity of the Commission to: (i) monitor the affiliation of indigenous peoples to the PHI as it was scaled up; (ii) monitor the delivery and accreditation of health services in priority indigenous localities as defined by the CDI and rural areas; (iii) monitor indigenous peoples’ satisfaction with the PHI, incorporating indigenous localities in its semi-annual user satisfaction surveys; and (iv) improve the SPSS’s information system by disaggregating data by gender and ethnicity. The third objective was to strengthen the capacity of the State Health Systems to support the implementation of IPP activities. 105. The Project benefited the indigenous population through increased affiliation and specific actions as specified in the Indigenous Peoples Plan (IPP). This is important because the health status of indigenous peoples has historically been poor in Mexico, with access to health services constrained due to remote and rural residence and lack of contributory social security enrollment. The social assessment prepared during the Project’s design identified many of the issues hindering PHI enrollment of indigenous peoples. Based on these finding, the Project prepared and implemented a strategy that went beyond the Government’s initial cultural and communication plans for targeting the population. In order to support the affiliation and re- affiliation of Indigenous people to the Program, the Commission worked with the Indigenous Peoples Development Commission (CDI) to implement a radio and visual information campaign in numerous indigenous languages. The PHI program also supported sensitivity training for health personnel and hired bilingual staff as health promoters, gestores (health information managers that deal with complaints and promote affiliates rights and knowledge) and affiliation staff in areas with significant indigenous populations. The significant increase in enrollment among this marginalized population reflects the success of the framework. Between 2006 and 2012, the gap in insurance coverage rates between indigenous and non-indigenous populations was virtually eliminated, and significant advances were made in narrowing service coverage gaps (eg. hospital based births) and even health outcomes (eg. stunting), even though there remain significant service utilization differences. (Leyva et al. 2012 and Figure 7). Despite the IIP objectives, during the Project the Commission did not regularly publish information on the delivery and accreditation of health services in priority indigenous localities, though this would be possible on the basis of information available at the Commission. 36 Figure 7: Health Insurance Coverage among Indigenous and Non-Indigenous Populations, 2006- 2012 29 70% 65% 60% 51% 50% 40% 30% No coverage 22% 21% 20% PHI 10% 0% 2006 2006 2012 2012 Indigenous Non Indigenous Non Indigenous Indigenous 29 Source: ENSANUT (2012) and Leyva et al (2012) 37 Box 1: The Case of Puebla The state of Puebla is one of the states with the largest reported indigenous population and with a hard to reach rural mountainous population. The state reflects a positive example of innovation and coordination between the different health providers in the area of outreach to indigenous populations. In an effort to affiliate the hard to reach indigenous population, Puebla used PHI resources to establish traditional medicine centers alongside mainstream hospitals. The eight traditional centers include traditional midwives, healers and hueseros (chiropractor or masseuse) and provide their services in coordination with the neighboring hospital. For example, while pregnant women can choose to receive prenatal care and give birth with a traditional midwife in the traditional medicine center, they must also attend five pre-natal visits with the obstetrician/gynecologist in the adjoining hospital. Traditional midwives are trained to detect risks during prenatal care and deliveries, and the medical hospital staff are on hand to attend any complications that occur. Puebla has also been successful in reaching the indigenous population through a successful coordination with the CDI, and in particular through regularly scheduled indigenous language radio stations. The state has also hired health promoters and medical managers (gestores) who are fluent in the local indigenous language. Puebla used PHI funding to implement a multi-step program to address the main causes of maternal mortality. This program includes close monitoring of high-risk hospitalized obstetric cases, an agreement with the Red Cross to attend uncomplicated normal deliveries, the establishment of maternal homes where women who live far away from medical centers can stay before giving birth, and the establishment of a triage and escalation system for emergency obstetric cases. In addition, a maternal defunction triggers an immediate administrative investigation as to its causes. As a result of the program, maternal mortality in Puebla dropped from 63 cases in 2012 to 37 in 2013. Death of mothers affiliated to PHI, remained the highest in 2013 (15) but fell 53 percent year-on-year. (Source: Presentation on “Reducción Muerte Materna Puebla Sana”). (d) Monitoring and Evaluation Framework 106. During the implementation of the Project, the Commission invested heavily in improving its M&E capabilities. These included the development of indicators for monitoring at the state level, technical assistance to improve the collection and analysis of information in the States, investments in health information systems such as an individualized health information system to support preventive health screenings, and the rollout of a biometric identification system for affiliates. The Commission carried out a wide range of studies that included various evaluations of the PHI program. (Annex 2) 107. PHI was evaluated using a randomized evaluation methodology before the approval of the Project. (King et al. 2009) At the time of the Project’s design, it was not deemed possible to carry out a second impact evaluation of the Program, despite the fact that the first evaluation did not demonstrate impacts on service utilization or health outcomes, possibly due to a rather short evaluation timeframe. During the implementation of the Project, the 38 Commission contracted several external evaluations of the PHI program. (Eg. Lazcano 2012 and Lozano 2013) 108. The Project M&E Framework used a multi-pronged approach that included existing administrative data and regular survey data. At Project closing, the combination of these approaches allowed for measurement of PDOs. At the same time, the M&E framework could have made fuller use of regular survey data, and there were weaknesses in the indicators’ design and definition, which resulted in unclear and imprecise measurement of certain outcomes during the Project’s implementation. 109. Better definition of indicators would have enhanced the Project M&E Framework. First, The Project Appraisal Document (PAD) and the Operational Manual (OM) lacked detailed information about indicator calculation. For a number of indicators, including PDO indicator 1 and intermediate indicators 3, 4 and 5, the monitoring framework was ambivalent about whether the indicator’s target was the percentage or the numerator and did not give sufficient specification of the denominator. The monitoring framework seemed to suggest that the baseline value of the denominators would be used for all subsequent years, in which case using absolute numbers would have been a better choice: the use of percentages does not convey more information about project achievement since they were not adjusted for population shifts or changes in contributory insurance affiliation. 30 Second, a number of indicators were rather complex, yet the design documents were incomplete as to the source of data and methods of calculation. For example, PDO indicator 1 cannot be calculated on the basis of program indicators from the Commission only; intermediate indicator 3 was calculated on the basis of the ENSANUT (instead of ENIGH according to the PAD), yet this was not reported during supervision. PDO indicator 2 was revised in the first restructuring partly to add baseline and target values. However, the unit of measure does not correspond to the indicator definition (percentage of actions) and therefore the target values in the restructured M&E framework are not appropriate. 110. The Project’s M&E Framework would also have benefited from having fewer and less complex indicators, but including indicators to measure effective access. While the Project’s monitoring framework included a wide array of indicators measuring affiliation, PDO indicator 3 is the only one that captures utilization of services by PHI affiliates. The Program’s success towards increasing effective access to health services could have been measured through: (i) public health spending per beneficiary, (ii) composition of PHI affiliates by income quintiles based on household surveys (iii) service availability and utilization. 111. During implementation, the lack of clarity in the design of the indicators resulted in imprecise measurement of a number of outcomes. A number of indicators that were 30 At the time of the Project’s design, the most recent population numbers were those from the 2005 Population Counting (Conteo de Población). The 2010 Population and Household Census data was carried out in June 2010, however, the final data only became available in early 2012, and therefore they were not used in adjusting the project indicators. 39 expressed as percentages reached over 100 percent. Other indicators reached high levels but were not confirmed by alternative data sources. For example, intermediate indicator 3 reported that coverage of PHI among the non-insured in deciles 1 and 2 reached 97 percent, using the 2012 National Health and Nutrition Survey (Encuesta Nacional de Salud y Nutrición – ENSANUT). Yet a study by Gutierrez (2013) using the same dataset finds that approximately 20 percent of individuals in those deciles were not covered by any health insurance, which gives a rather different picture of the situation. As a result, this indicator does not convey accurate information about the reach of the program. In this context, it is important to note that the different household surveys in Mexico have different sampling and calculation methodologies, and therefore, indicators can be different depending on the source of data. While this is not under the control of the implementing agency, it is critical to spell out the source of data and methods of calculation in the project design documents to ensure replicability and comparability of the indicators that are used to monitor the project. 112. Despite the issues with the M&E Framework, at the time of the ICR there were sufficient data available to estimate the efficacy of the Project. The existence of regular household surveys allows to use alternate measures where needed. Table 7 presents the key issues that were encountered with the indicators, and the ICR’s approach in evaluating the Project. 40 Table 7: Comments on Key Project Indicators and ICR Approach Indicator Comments PDO Indicator 1: Number of The source or method of calculation of the denominator is individuals affiliated with not specified in the Project documents. Popular Health Insurance as a At the time of the ICR, the World Bank calculated on the percentage of the total number of basis of the ENIGH that the baseline number of people not individuals that are not affiliated affiliated with a contributory Social Security System was of with a contributory Social the order of 70M. Security System Based on the 2005 Conteo de Población, the number of individuals not affiliated with a contributory social security system is 62.1M. The ICR bases project achievement on the numerator. PDO Indicator 2: Percentage of This indicator was revised in the first restructuring partly to recommended actions add baseline and target values. However, the unit of measure implemented by federal entities does not correspond to the indicator definition (percentage resulting from the supervision of actions) and therefore the target values in the restructured action plan carried out by the M&E framework are not appropriate. Commission in the four action The ICR uses the target of 71 percent which was used by the areas (Affiliation and Operation, GOM in the May 16, 2011 request for restructuring. Health services management, Financing and Oportunidades program) Intermediate Indicator 3: While the PAD and the 2011 restructuring paper mention Number of individuals in deciles that the source of data is the ENIGH, the results reported by 1 and 2 affiliated with Popular the Government appear to have been calculated on the basis Health Insurance’s subsidized of ENSANUT. However, triangulation with other analyses regime as a percentage of total yield different results: First, according to Gutierrez (2013), number of individuals in deciles the 2012 ENSANUT shows that 20.1% of the population in 1 and 2 that are not affiliated the lowest quintile (i.e. deciles 1 and 2) had no health with a contributory social coverage (contributory or non-contributory), down from security system. 58.8% in 2006. Second, on the basis of ENIGH, at the time of the ICR we estimate that the percentage of individuals in deciles 1 and 2 that were affiliated to PHI, as percentage of those in deciles 1 and 2 that were not affiliated to a contributory social security system, went from 42.30% (2008) to 72.32% (2012). While the numbers based on ENIGH and ENSANUT are somewhat different due to different calculation methods, they are not compatible with the numbers reported on intermediate indicator 3, whether at baseline or endline. The ICR does not use this indicator to evaluate the efficacy of the Project. Intermediate Indicator 4: The numerator uses PHI’s definition of a family in the Number of Oportunidades nuclear sense, while the denominator uses Oportunidades’s beneficiary families affiliated more extended family definition. As a result, a family unit in with the Popular Health Oportunidades (denominator) might be counted as two Insurance as a percentage of the family units in PHI (numerator). The denominator also does total number of Oportunidades not take into account the expansion of the Oportunidades beneficiary families. program that took place in 2010. For these two reasons, the 41 Indicator Comments indicator can go above 100 percent. To measure the achievement of PHI in expanding access to Oportunidades families, the ICR uses the numerator only. Intermediate Indicator 5: Source: CNPSS (numerator) and INEGI (denominator) Number of individuals affiliated The indicator was adjusted since there was a downward re- with the Popular Health vision of the municipalities where more than 40% of the Insurance residing in areas where population speaks an indigenous language. more than 40% of the population The denominator underestimates the number of individuals speaks an indigenous language not affiliated, for the following reasons: (i) The denominator as a percentage of total number uses population estimates from the 2005 Conteo, which are of individuals that are not about 8.8 percent below those of the 2010 Census. The latter affiliated with a contributory are a more accurate reflection of the population numbers social security system that are during the Project. (ii) The denominator counted the residing in these areas. affiliates of Seguro Popular as affiliated to a contributory social security system, whereas they should be counted as unaffiliated. As a result of (i) and (ii), the denominator significantly underestimates the population that is unaffiliated to a contributory social security system in the said areas. For this reason, the ICR uses the numerator information only. Intermediate Indicator 6: Source: CNPSS/SINAIS (numerator). Number of women and girls The source of data and method of calculation of the affiliated with the Popular Health denominator is not specified in the Project documents. Insurance as a percentage of the Triangulation with other sources of data yields different total number of women and girls estimates. According to the 2005 Conteo, in 2005 there were who are not affiliated with a 29.9 million women not affiliated with a contributory social contributory social security security system (against 24.8 million in the PAD). system. Given the inability to replicate the baseline denominator, the ICR uses the numerator to evaluate the success of the project in reaching women and girls. 42 Annex 3. Economic and Financial Analysis 113. In this Annex, we estimate whether the Project was cost effective, using a break- even analysis. To do so, we first estimate the costs of the PHI Program. We then estimate the required reduction in the burden of disease for the Program to break even between costs and benefits. 114. The gross per capita cost of the PHI program is 2,735 Pesos per person per year. This amount includes the Social Contribution from the Federal Government and the Federal and State solidarity contributions from both the Federal and State Governments. (See Table 8) While the World Bank project only directly financed the federal Social Contribution, the cost of the program should include all funding sources. It is important to note that this does not necessarily correspond to the amounts that are transferred to the States, as those are adjusted for other programs that are financed by the Federal Government and benefit PHI beneficiaries, such as the health investments financed by the Oportunidades program. 115. To obtain the net cost of the PHI program, one needs to take into account the possible reduction in out-of-pocket spending by beneficiaries. Reductions in out-of-pocket spending are equivalent to a transfer from the Government to beneficiaries and should not be counted as part of the economic cost of the program. King et al. (2009) estimate that the transfer value of the program is 915 pesos per year (Table 8). Table 8: Cost of the PHI Program Government spending Social Contribution 912 Federal Solidarity Contribution 1,367 State Solidarity Contribution 456 Sub-total (a) 2,735 Transfer value (Reduction in OOP spending) (b) 915 Net cost (a)-(b) 1,820 116. Since PHI has beneficiaries across different age ranges and includes a wide variety of health services and conditions, it could potentially have impacts on multiple health outcomes. For the economic analysis, we can convert the multiple metrics into saved Disability Adjusted Life Years (DALYs) and compare the saved DALYs to the cost of the program to obtain a measure of cost-effectiveness. (Saban et al, 2012) We then benchmark the cost per DALY saved against national per capita GDP to assess whether the intervention is cost-effective in the Mexican context. However, since we do not have comprehensive estimates of the impact of PHI, we take the inverse approach: we estimate the percentage reduction in DALY burden of disease that would be required for the PHI program to break-even against the benchmark of GDP per capita. 117. The 2004 Burden of Disease estimates that Mexico had a burden of disease of 15,430 DALYs per 100,000 inhabitants. (Table 9) There are two potential problems with using this burden of disease estimates. First, Mexico has experienced increases in life expectancy and decreases in infant mortality since 2004 - in that sense the 2004 burden of disease estimates may 43 overestimate the situation in 2010, at the start of the Project. On the other hand, the PHI program excludes individuals who have alternative health insurance schemes, which tend to be those that are better-off – in that sense the burden of disease estimates may underestimate the burden of disease for PHI beneficiaries. However, we do not have either more updated numbers, or estimates that are specific to the PHI intended beneficiaries. Since it is likely that the two effects somewhat compensate each other, we will use the 2004 burden of disease estimate for the general population as our best estimate of the 2010 burden of disease for the PHI intended beneficiaries. Table 9: Parameters used in the Economic Analysis Indicator Value Source of information DALY lost per 100,000 population 15,430 2004 Burden of Disease estimates Net cost PHI (Pesos per year per person) 1,820 2013 PHI results report GDP per capita (current LCU), 2010 112,215 World Development Indicators PPP conversion factor, private consumption 8.7522 World Development Indicators (LCU per international $), 2010 118. In 2010, GDP per capita was estimated to be MXPs 112,215 (World Development Indicators). In Table 10, we simulate the cost per DALY saved for different percentage reductions in the overall burden of disease. We find that an overall reduction in the burden of disease of 11 percent for PHI beneficiaries would result in a break-even situation where the cost per DALY saved is equal to GDP per capita. Table 10: Break-even Analysis % Reduction in the burden of disease 5% 10% 11% 20% DALY saved (per 100,000 population) 772 1,543 1,622 3,086 Net cost PHI 1,820 1,820 1,820 1,820 Cost per DALY saved 235,882 117,941 112,215 58,971 Cost per DALY saved (USD) 26,951 13,476 12,821 6,738 119. We then analyze available results to decide whether an 11 percent reduction in the burden of disease is a likely impact of PHI. A number of studies have looked into the impact of PHI on health outcomes. A randomized study by King et al. (2009) assessed the impact of rolling out PHI, and finds a 23 percent reduction from the baseline in catastrophic expenditures, but no effects on medication spending, health outcomes or utilization. The evaluation found that the Program’s resources reached the poor, but that the program did not show other effects, possibly due to the short duration of treatment (10 months). All other evaluations of the program use non-experimental methods and results vary. For example, Gakidou et al (2006) and Scott (2006) find that SP beneficiaries have higher utilization rates than the uninsured. Bernal et al (2010) use the 2006 and 2009 ENADID 31 to estimate utilization of pre-natal services and health services during the first year of life, and find no impact of PHI. However, they argue that already in 2002, the first year of operation of the program, 97 percent of the eligible women already 31 Encuesta Nacional de la Dinámica Demográfica 44 received prenatal care, and 96 percent of children under one year of age already received medical care, which makes it unlikely to find a significant effect of any program on those indicators. At the same time, they do find a 3 percentage point increase in the probability that deliveries in the rural areas were attended by a medical doctor, comparing to an 83 percent baseline. The authors do not find any evidence of an impact on low birth weight in public or social-security maternities. However, the treatment variable used in the evaluation is the state-level status of PHI enrollment, which is a noisy indicator of individual enrollment in the program. Therefore the estimates will suffer from attenuation bias and are likely to underestimate the impact of the Program. In a recent paper, Pfutze (2014) uses a weighted exogenous sampling maximum likelihood (WESML) estimator and finds that PHI can be expected to reduce Mexico’s infant mortality rate by close to 5 out of 1,000 births. 120. Part of the explanation for the mixed evidence on the link between PHI and MDG health outcomes may be that funding to the states for PHI ends up strengthening the entire health system in the state. The comparison between PHI affiliates to non-affiliates within states may be biased because non-affiliates may benefit from improvements in health services as well. For example, some states such as Puebla have made particularly impressive progress on the MDGs such as maternal mortality (Annex 2, Box 1) and have used PHI funds to finance their intervention strategies. 121. A few evaluations look at the impact of PHI in those areas that now form 90 percent of the burden of disease, i.e. non-communicable diseases and injury. (Knaul et al.) Parker and Rubalcava (2010) use the MxFLS 2002 and 2005 and find no impact in diabetes, arthritis, migraine, obesity or heart disease; however, they find a reduction in cholesterol levels and days off work due to illness. Teruel et al (2014) use the 2002 and 2009-2011 rounds of the MxFLS and find positive impact of the Program on health services, detection and diagnosis for urban women and rural men. They also find an improvement of health status for urban women, rural men, and children in areas related to chronic conditions. 122. Overall, we estimate that it is modestly likely that the PHI achieved an 11 percent reduction in the overall burden of disease, thereby rendering it moderately likely that the Program was cost-effective. However, the lack of a rigorous evaluation strategy that encompasses more than 1 year of program exposure is unfortunate from the policy point of view, especially given the magnitude of the investment. 45 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Maria E. Castro-Munoz Consultant LCSHH Alejandra Gonzalez Program Assistant LCSFM Dmitri Gourfinkel Financial Management Specialist LCSFM Jose C. Janeiro Senior Finance Officer CTRLA Veronica Yolanda Jarrin Operations Analyst AES Christoph Kurowski Lead Health Specialist ECSH1 Claudia Macias Senior Operations Officer LCSHH Christina Novinskey E T Consultant LCSHH Gabriel Penaloza Procurement Specialist LCSPT Mariangeles Sabella Senior Counsel LEGES Manuel Antonio Vargas Madrigal Lead Financial Management Specialist MNAFM Adam Wagstaff Research Manager DECHD Yasuhiko Matsuda Sr. Public Sector Spec. EASPR Xiomara A. Morel Sr. Financial Management Specialist LCSFM Manuela Villar Uribe Consultant AFTHD Supervision/ICR Maria E. Castro-Munoz Consultant LCSHH Alejandra Gonzalez Program Assistant LCSFM Dmitri Gourfinkel Financial Management Specialist LCSFM Jose C. Janeiro Senior Finance Officer CTRLA Veronica Yolanda Jarrin Operations Analyst AES Claudia Macias Senior Operations Officer LCSHH Xiomara A. Morel Sr Financial Management Specialist LCSFM Christina Novinskey E T Consultant LCSHH Luis Adrian Ortiz Blas Junior Professional Associate LCSHH Gabriel Penaloza Procurement Specialist LCSPT Gunars H. Platais Senior Environmental Economist LCSEN Mariangeles Sabella Senior Counsel LEGES Tomas Socias Senior Procurement Specialist LCSPT Manuela Villar Uribe Consultant AFTHD Juan Carlos Serrano-Machorro Senior Financial Management Specialist LCSFM Maria Eugenia Bonilla-Chacin Senior Economist LCSHH Christel Vermeersch Senior Economist LCSHH Natasha Zamecnik Consultant LCSHH 46 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands Stage of Project Cycle No. of staff weeks (including travel and consultant costs) Lending 2009 3.40 46,436 2010 84.86 269,834 Sub-total: 88.26 316,270 Supervision/ICR 2010 18.18 97,086 2011 104.15 244,000 2012 71.75 216,112 2013 28.35 105,375 2014 20.53 84,688 Sub-total: 242.96 747,261 TOTAL 331.22 1,063,531 47 Annex 5. Beneficiary Survey Results 123. The Commission carried out three beneficiary studies (Encuesta Nacional de Afiliados al Seguro Popular) during Project implementation, in 2010, 2011 and 2012. The panel surveys obtained information on the personal characteristics of affiliates and their households, interactions with the PHI, and health expenditure. 124. The 2012 satisfaction survey of PHI affiliates 32 found a 97 percent satisfaction with PHI, with a high satisfaction for all service components (affiliation process, medical attention, etc.). Ninety-nine percent of those surveyed planned to re-affiliate, with most providing the fact that service is free, that they receive good attention and that they don’t have access to another health insurance as the main reasons for re-affiliation. Most interviewed felt they received a detailed explanation of what they were suffering. The average wait time for receiving attention was 104 minutes, with variations between states from 69 minutes to 157 minutes. Most affiliates say they would return to the same health center. However, less than half of the affiliates received information concerning their rights at the time of affiliation. Indigenous affiliates rated satisfaction with PHI in the satisfaction survey between 8.5 and 9.7 (total of 10). 125. While the surveys show high satisfaction levels, the survey does not fully represent the population of PHI affiliates because it selected PHI affiliates at the point of service exit. Hence the survey implicitly excluded PHI affiliates who did not choose to attend PHI health centers or those that went to alternative providers; arguably these could be the PHI affiliates that are less satisfied or have given up on receiving services from PHI providers. 126. In terms of socio-demographics, the surveys showed the following: • Demographics: Most affiliates live in households with less than five members. Most PHI affiliates are women and their family members: 54.5 percent, 56.4 percent and 57.8 percent in 2010, 2011 and 2012 respectively. The age of affiliates has increased over the last few years, from 26 for males and 28 for females in 2010 to 27 and 30 respectively in 2012. Nine out of 10 affiliates speaks Spanish. According to the 2012 survey 37.8 percent of affiliates had only finished primary school, 31.7 percent had finished secondary and 12.2 percent had completed high school. • Housing: Four out of every five households having a separate cooking space. Most households have electricity, but only slightly over half have running water. Approximately 60 percent of households are connected to drainage systems. Over 75 percent of households are tenants. • Employment: Fifty-eight percent of the affiliates were employed in urban jobs. Ninety- three percent of affiliates (2012) counted on only one type of health service. 127. Approximately a third of the surveyed affiliates experienced an illness in 2010 and 2011, though this number fell to a fourth in 2012. During the three surveys the number of 32 The 2012 survey is the latest available. The Ministry of Health carries out a satisfaction survey for PHI affiliates on a biannual basis. 48 affiliates that required hospitalization due to an illness or lesion increased from 9.9 percent in 2010, to 13.3 percent in 2012, while those that sought attention but did not receive it fell from 3.2 percent to 1.2 percent between 2011 and 2012. The likelihood of experiencing a respiratory illness fell from 52.2 percent to 31 percent between 2010 and 2012, while treatment for diabetes increased to 6.7 percent of affiliates in 2012 from 4.3 percent in 2010. Most affiliates (94.7percent in 2011 and 93.2 percent in 2012) that sought attention only realized one visit to the health center. 128. More than four-fifths of PHI affiliates who received attention were prescribed medicine. Of these 79 percent obtained the prescribed medicine in 2010, 74 percent in 2011 and 75.4 percent in 2012. The most frequent reason for which affiliates did not obtain medicine was due to unavailability. Medicine costs were covered through own income (54.6 percent in 2011 and 44 percent in 2012), savings (26.2 and 21.2 percent, respectively), and loans (21.9 and 15.3 percent, respectively). 129. The beneficiary surveys show that coverage of health care costs by PHI increased between 2010 and 2012. In 2010, the PHI partially covered medical costs for 28.8 percent of affiliates and fully covered costs for 41.5 percent of all affiliates. In 2011 it partially covered costs 27.5 percent and total 51.7 percent of affiliates and 20.9 percent and 56 percent, respectively, in 2012. In 2010, 29.7 of the affiliates surveyed stated that the PHI did not cover their costs; 20.8 percent in 2011 and 23.2 percent in 2012. For those that required hospitalization, in 2010 28.8 percent said the PHI covered part of the costs, 38.1 percent in 2011 and 28.6 percent in 2012. The PHI totally covered hospitalization costs in 41.5 percent, 41.7 percent and 50 percent of the cases, respectively. The reasons given for non-coverage included: affiliates were not attended to or the illness was not covered; service was deficient or took too long; affiliates were seen by a private doctor; and administrative issues. Over time, the surveys show a decrease in the issues that affiliates encountered in the search for health services. 130. The number of affiliates that were aware that they could make complaints increased from a little less than half in 2010 to 61.6 percent in 2012. The proportion of affiliates that actually made complaints increased from 1 to 2 percent over the time period covered. 49 Annex 6. Stakeholder Workshop Report and Results Not Applicable 50 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 51 52 53 54 55 56 57 58 Unofficial translation of the Borrower’s Comments on the Draft ICR • [Intermediate Indicator 2]: At the end of 2013, the number of entities that contributed with their State Solidarity Contribution was all 32 entities in the country; none of them failed to comply with this requirement of participation in the Social Protección System in Health. • [Intermediate Indicator 4]: The correct number of Oportunidades families at the closing [of the Project] is 5.7 million and not 6.3 million; for this reason the calculations and comments about this indicator should be corrected in the document. On page 25, there is a reference to the correct number, as follows: “The number of Oportunidades beneficiary families affiliated with the PHI (intermediate indicator 4) grew from 3.06 million to 5.7 million, compared to the original target of 4 million.” • [Intermediate Indicator 6]: The denominator was established in the Project Appraisal Document and was used as the basis for the calculation of the indicator. However, the growth of the affiliation led to an increase in the number of affiliated women, to a larger extent than for men because women tend to have more precarious and vulnerable labor conditions. • [Intermediate Indicators 9, 10 and 11]: The indicators about the receipt of the letter of rights and obligations and about the benefits exhibit a large variation nationally between the different federal entities. This fact is outlined in various parts of the document; however, it should be emphasized so as to demonstrate that the knowledge and dissemination [of information] depend on how the Regímenes Estatales del Sistema de Protección Social en Salud (REPSS) are organized. • The national sources of data that are external to the Commission (CNPSS) present variability in their results as well as in their [methods] of processing results; this leads to limited consistency with the presented indicators. This problem is transversal to the national sources of statistical information which are being generated by various national institutions; their design and elaboration is not the competency of the Commission. The construction of the instruments used to collect the variables on income is different between ENIGH and ENSANUT; the methods used by those surveys to calculate income are also different in the sense that they are constructed using different concepts. • There has been a change in the General Health Law’s articles 77 bis, bis 6, bis 11, and bis 16. These changes were submitted through the Legislative chambers in 2013 and approved, and they will become effective with the signing of the executive in 2014. Those reforms aim to improve the efficiency of the use of resources of Seguro Popular destined to the affiliates in the federal entities through the REPPS, while improving the capacity of the CNPSS to supervise and reinforce the rules, so that the resources are used for health services in the federal entities. • The CIDE is the institution that has the most qualified personnel in the country in health economics and in areas related to social protection systems. In addition, [the institution] 59 has the technical knowledge to use the pertinent analysis tools. This is the reason for which this institution assumed the largest part of the responsibilities for the studies. On the other hand, we note that there were other academic institutions involved in various projects; for example the National Autonomous University of Mexico, the Autonomous University of Chapingo, the University of Guadalajara, and the College of Mexico were involved in studies carried out by the CNPSS. • [Intermediate Indicator 5]: When the Project indicators were established in February 2010, the most recent data available were those from the 2005 Population Counting which clearly did not correspond to the real number of inhabitants at the time [2010]. The [new] Population and Household Census was carried out in July of 2010, the first data were released early 2011, and the final results were released early 2012. For this reason they were not used and the denominator that was used was the number that was registered at the beginning. • [Table 2 (c): Study on the use of health service access of the PHI affiliated indigenous populations]: The study confirmed the indigenous population’s lack of knowledge regarding their rights and responsibilities regarding the PHI, while documenting the group’s growing affiliation and coverage. The study also revealed how health centers in indigenous areas often did not have the necessary infrastructure, personnel and medicine, while geographic, economic, cultural and administrative factors hindered health care access. • [Indigenous People’s Plan]: In the Social Protection System in Health Results Report which is presented every semester by CNPSS to the [Mexican] Congress, there is information on the number of accredited health units that can offer services [listed] in CAUSES, FPGC and SMSXXI. The report does not present a breakdown of those units but it can be done, because the General Direction of Affiliation and Operations makes trimestral tabulations of the population that is affiliated to the System and of the health units that give health services, using information from the General Direction of Health Information. Using those [data], it is possible to elaborate tables such as the one presented below, with breakdown by jurisdiction and municipality, although this is not being regularly published. [Table follows.] 60 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable 61 Annex 9. List of Supporting Documents Ávila Burgos, L, E Serván Mori, Verónica Wirtz, Sergio Bautista Arredondo, Sandra Sosa-Rubí, A. Salinas Rodríguez (2012), “Seguro Popular y Gasto en Salud en Hogares Mexicanos: Consolidar el Efecto Protector,” Instituto Nacional de Salud Pública, Cuernavaca. Bonilla-Chacín, Maria Eugenia and Nelly Aguilera (2013), The Mexican Social Protection System in Health, Unico Studies Series 1, The World Bank, Washington, DC. Bosch, M., M.Belen Cobacho y C. Pages (2012), “Taking Stock of Nine Years of Implementation of Seguro Popular in Mexico – Lessons for Developing Countries,” Technical notes Nr. IDB-TN-442, Inter-American Development Bank, Washington, DC. Presidencia de la República (2012), El Presidente Calderón Agradece a Méxio en Mensaje”. Available at: http://calderon.presidencia.gob.mx/2012/11/el-presidente-calderon-agradece-a-mexico/ Colegio de México (Colmex) (2014), Auditoría del Padrón de Afiliados del Sistema de Protección Social en Salud, Mexico D.F. Comissión de Protección Social en Salud (2014), Sistema de Protección Social en Salud: Informe de Resultados 2013, Mexico D.F. Comissión de Protección Social en Salud (2014), Nota Técnica Sobre las Fuentes de Documentación de los Indicadores del Proyecto 7890-MX, Mexico D.F. Comissión de Protección Social en Salud (2013), Sistema de Protección Social en Salud: Informe de Avances Semestral – Proyecto del Sistema de Protección Social en Salud, Prestmo No. 7860- MX, Cierre del Ejercicio 2012 y Primer Semestre 2013 Mexico D.F. CONEVAL (2014), “Indicadores de Acceso y Uso Efectivo de los Servicios de Salud de Afiliados al Seguro Popular,” Mexico D.F. Consejo Nacional de Evaluación de la Política de Desarrollo Social (CONEVAL). Available at http://www.coneval.gob.mx/Medicion/Paginas/Evolucion-de-las-dimensiones-de-la-pobreza- 1990-2010-.aspx. Duval Hernández, R. and R. Smith Ramírez (2011), “Informality and Seguro Popular under Segmented Labor Markets,” preliminary draft, Centro de Investigación y Docencia Económicas (CIDE), Mexico, D.F. Gakidou, Emmanuela, Rafael Lozano, Eduardo Gonzalez-Pier, Jesse Abbott-Klafter, Jeremy Barofsky, Chloe Bryson-Cahn, Dennis Feehan, Diana Lee, Hector Hernandez-Llamas, and Christopher Murray (2006), “Assessing the Effect of the 2001-2006 Mexican Health Reform: An Interim Report Card,” The Lancet, Vol. 368(9550), pp. 1920-1935. 62 Galárraga, Omar, Sandra G. Sosa-Rubí, Aarón Salinas-Rodríguez and Sergio Sesma-Vázquez (2010), “Health Insurance for the Poor: Impact on Catastrophic and Out-of-pocket Health Expenditures in Mexico,” European Journal of Health Economics, Vol. 11(5), pp. 437-447. Gobierno de la Republica, Plan Nacional de Desarrollo 2013-2018: Programa Sectorial de Salud, Mexico, D.F. Available at http://portal.salud.gob.mx/contenidos/conoce_salud/prosesa/pdf/programa.pdf. Gutiérrez, Juan Pablo and Mauricio Hernández-Ávila (2013), “Cobertura de Protección en Salud y Perfil de la Población sin Protección en México, 2000-2-12,” Salud Publica Mex, Vol 55, Supl 2, PP. S83-S90. Instituto Nacional de Salud Publica (2012), Encuesta Nacional de Salud y Nutrición (ENSANUT) - Resultados Nacionales, p.51. King, Gary, Emmanuela Gakidou et al. (2009), “Public Insurance for the Poor ? A Randomized Assessment of the Mexican Universal Health Insurance Program”, The Lancet, Vol. 373 (9673), pp. 1447-1454. Knaul Felicia, Eduardo González-Pier, Octavio Gómez-Dantés, David García-Junco, Héctor Arreola-Ornelas, Mariana Barraza-Lloréns, Rosa Sandoval, Francisco Caballero, Mauricio Hernández-Avila, Mercedes Juan, David Kershenobich, Gustavo Nigenda, Enrique Ruelas, Jaime Sepúlveda, Roberto Tapia, Guillermo Soberón, Salomón Chertorivski, Julio Frenk (2010), “The Quest for Universal Health Coverage: Achieving Social Protection for All in Mexico,” The Lancet, Vol. 380, Issue 9849, pp. 1259-1279. Kurowski Christoph and Luis Ortiz (2012). “Mexico’s Social Protection System in Health and the Financial Protection of Citizens without Social Security.” World Bank, Washington, D.C. Lazcano Ponce, Eduardo, Héctor Gómez Dantés, Rosalba Rojas, Francisco Garrido Latorre (Eds) (2012), Sistema de Prtoección Social en Salud: Evaluación Externa 2012, Instituto Nacional de Salud Pública, Cuernavaca, Morelos. Lozano Ascencio, Rafael, Belkis Aracena Genao, Emanuel Orozco Núñez, Francisco Franco Marina, and Nicéforo Farnelo Bibiano (2013), Evaluació Externa del Fondo de Protección Contra Gastos Catastróficos del Sistema de Protección Social en Salud 2013, Instituto nacional de Salud Pública, Cuernavaca, Morelos. Leyva, René, César Infante Xibille, Edson Serván-Mori, Juan Pablo Gutiérrez (2012), “Inequidad Persistente en Salud en los Pueblos Indígenas: Retos para el Sistema de Protección Social,” Instituto Nacional de Salud Pública, Cuernavaca. Observatorio de Mortalidad Materna en México (OMM) (2014), online statistics at http://public.tableausoftware.com/profile/#!/vizhome/OPSindicadores/TMMDES, last accessed on 04/16/2014. 63 OECD (2013) Health at a Glance 2013: OECD Indicators, OECD Publishing. Parker S. and L. Rubalcava (2010), “Identificación y Análisis de los Efectos en las Condiciones de Salud de los Afiliados al Seguro Popular,” Centro de Investigación y Docencia Económicas (CIDE), Mexico D.F.. Pueblita, Jose Carlos.R., (2013) Screening Seguro Popular: The Political Economy of Universal Health Coverage in Mexico, Working paper, Center for International Development, Harvard University. Pfutze, Tobias (2014), “The Effects of Mexico’s Seguro Popular Health Insurance on Infant Mortality: An Estimation with Selection on the Outcome Variable,” World Development, Vol. 59, pp. 475-486. Reduccion Muerte Materna Puebla Sana – Estrategia Estatal para la Reducción de la Muerte Materna. Presentation 2014. Secretaría de Salud (2013), Plan Nacional Desarollo 2013-2018 - Programa Sectorial de Salud, Phttp://portal.salud.gob.mx/contenidos/conoce_salud/prosesa/pdf/programa.pdf Secretaría de Salud (2004), Fair Financing and Universal Social Protection: The Structural Reform of the Mexican Health System, Mexico, D.F. Scott, John (2006), Seguro Popular Incidence Analysis, in Decentralized Service Delivery for the Poor, Vol. II, Chapter 3, pp. 147-466, The World Bank, Washington DC. Scott, John (2010), Gasto Público para la Equidad: del Estado Truncado al Estado de Bienestar Universal, Trabajo para México Evalúa, Mexico D.F. Scott, John (2013), Redistributive Impact and Efficiency of Mexico’s fiscal System: Comittment to Equity, Working Paper No. 8, Tulane University and CIPR. Sos-Rubí, Sandra, Aarón Salinas-Rodríguez, Omar Galárraga (2011), “Impact del Seguro Popular en el Gasto Catastrófico y de Bolsillo en el México Rural y Urbano, 2005-2008,” Salud Publica Mex, Vol. 55, Supl 4, pp. S425-S435. Teruel Graciela, Erika Arenas, Susan Parker and Luis Rubalcava (2014), “Impact Evaluation of Seguro Popular on the Use of Health Services, Health Status and Out-of-pocket Health Expenditures,” PPT presentation available at http://www.sedesol.gob.mx/work/models/SEDESOL/Resource/139/1/images/JT_EvSegPopular2 014.pdf. The Lancet Commission on Investing in Health (2013), “Global Health 2035: a World Converging Within a Generation,” The Lancet, Vol. 382, Issue 9908, PP. 1898-1955. 64 Wirtz, Veronika, Edson Serván Mori, Anahí Dreser, Ileana Hereia Pi, and Leticia Ávila Burgos (2012), “Surtimiento y Gasto en el Acceso a Medicamentos en Instituciones Públicas: Asignaturas Pendientes,” Instituto Nacional de Salud Pública, Cuernavaca. World Health Organization (2010), The World Health Report: Health Systems Financing: the Path to Universal Coverage, Geneva. Available at: http://www.who.int/whr/en/index.html World Health Organization and World Bank Group (2013), “Monitoring Progress towards Universal health Coverage at Country and Global Levels: A Framework.” Discussion Paper available at http://www.who.int/healthinfo/country_monitoring_evaluation/UHC_WBG_DiscussionPaper_D ec2013.pdf 65 Annex 10. Map IBRD 33447R 66