PERU Juntos Results for Nutrition Project Report No. 135232 JUNE 7, 2019 © 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2019. Peru—Juntos Results for accuracy of the data included in this work. The Any queries on rights and licenses, including Nutrition Project. Independent Evaluation boundaries, colors, denominations, and other subsidiary rights, should be addressed to Group, Project Performance Assessment Report information shown on any map in this work do World Bank Publications, The World Bank 135232. Washington, DC: World Bank. not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 135232 PROJECT PERFORMANCE ASSESSMENT REPORT PERU JUNTOS RESULTS FOR NUTRITION PROJECT (IBRD LOAN NO. 79610) June 7, 2019 Human Development and Economic Management Independent Evaluation Group Currency Equivalents (annual averages) Currency Unit = nuevo sol (S/.) 2009 $1.00 S/. 3.01 2014 $1.00 S/. 3.18 2010 $1.00 S/. 2.82 2015 $1.00 S/. 3.37 2011 $1.00 S/. 2.75 2016 $1.00 S/. 3.37 2012 $1.00 S/. 2.63 2017 $1.00 S/. 3.26 2013 $1.00 S/. 2.70 Abbreviations CCT conditional cash transfer CRED Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo) ENCRED Encuesta a Establecimientos de Salud (Health Centers Survey) IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IEG Independent Evaluation Group MEF Ministry of Economy and Finance (Ministerio de Economía y Finanzas) MIDIS Ministry for Development and Social Inclusion (Ministerio de Desarrollo e Inclusión Social) MINSA Ministry of Health (Ministerio de Salud) PAN National Nutrition Program (Programa Articulado de Nutrición) PPAR Project Performance Assessment Report RENIEC National Civil Registry (Registro Nacional de Identidad y Estado Civil) SIGA Integrated Administrative Management System (Sistema Integrado de Gestión Administrativa) SIS Integral Health Insurance Program (Seguro Integral de Salud) SWAp sectorwide approach All dollar amounts are US dollars unless otherwise indicated. Fiscal Year Government: January 1–December 31 Director-General, Independent Evaluation Ms. Alison Evans Acting Director, Human Development and Economic Management Ms. Sophie Sirtaine Acting Manager, Corporate and Human Development Mr. Rasmus Heltberg Task Manager Ms. Ann Elizabeth Flanagan ii Contents Preface .............................................................................................................................................................. viii Summary .............................................................................................................................................................. x 1. Background and Context ..........................................................................................................................1 Project Context ............................................................................................................................................. 1 2. Objectives and Design ...............................................................................................................................5 Objectives .......................................................................................................................................................5 Relevance of the Objectives .....................................................................................................................5 Project Design ............................................................................................................................................... 7 Components ................................................................................................................................................................ 7 Theory of Change ...................................................................................................................................................... 8 3. Implementation ......................................................................................................................................... 10 Project Restructuring ................................................................................................................................ 10 Implementation Arrangements ............................................................................................................. 10 Implementation Experience .....................................................................................................................11 Implementation of Monitoring and Evaluation .............................................................................................. 11 Safeguards Compliance ......................................................................................................................................... 12 Financial Management and Procurement ....................................................................................................... 12 4. Achievement of the Objectives ............................................................................................................ 13 Objective 1: Increase Demand for Nutrition Services by Strengthening the Operational Effectiveness of Juntos ............................................................................................................................. 14 Outputs and Intermediate Outcomes ............................................................................................................... 14 Did Demand for Nutrition Services Increase? ................................................................................................ 15 Objective 2: Improve Coverage of the Basic Preventive Health and Nutrition Services In Communities Covered under PAN Including Juntos...................................................................... 19 Outputs and Intermediate Outcomes ............................................................................................................... 19 Did Coverage of Basic Preventive Health and Nutrition Services Improve?....................................... 20 Objective 3: Improve the Quality of Basic Preventive Health and Nutrition Services in Communities Covered under PAN Including Juntos...................................................................... 21 Outputs and Intermediate Outcomes ............................................................................................................... 21 Did the Quality of Basic Preventive Health and Nutrition Services Improve? .................................... 22 Why Did Malnutrition Decline? ............................................................................................................ 23 iii 5. Efficiency ...................................................................................................................................................... 24 6. Ratings .......................................................................................................................................................... 27 Outcome ...................................................................................................................................................... 27 Risk to Development Outcome ............................................................................................................ 27 Bank Performance .................................................................................................................................... 28 Quality at Entry ........................................................................................................................................................ 28 Quality of Supervision ........................................................................................................................................... 29 Monitoring and Evaluation .................................................................................................................... 30 Design ......................................................................................................................................................................... 30 Implementation ....................................................................................................................................................... 30 Use................................................................................................................................................................................. 31 Lessons .......................................................................................................................................................... 31 Bibliography..................................................................................................................................................... 35 Box Box 1.1. Types of Malnutrition ........................................................................................................................3 Tables Table 4.1. Juntos Children Younger than 12 Months Who Received the Complete CRED Scheme........................................................................................................................................... 16 Table 4.2. Children Younger than 36 Months Who Received the Complete CRED Scheme .. 17 Table 4.3. Stunting Rates in Children Younger than Five Years in Peru, 2009–18 (percent) .. 24 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 38 Appendix B. Nutrition Strategies and Programs................................................................................ 41 Appendix C. Project Components ........................................................................................................... 44 Appendix D. Implementation Details ..................................................................................................... 47 Appendix E. Key Performance Indicators .............................................................................................. 49 Appendix F. Main Characteristics of Juntos ......................................................................................... 52 Appendix G. Performance-Based Budgeting and the Articulated Nutrition Program ........ 54 Appendix H. Summary of Field Visits ..................................................................................................... 56 Appendix I. Relevant Nutrition and Health Support to Peru, 2009–17 ..................................... 61 iv Appendix J. List of Persons Met ............................................................................................................... 65 This report was prepared by Ann Elizabeth Flanagan with support from Rosa Puech, who assessed the project in November 2018. The report was peer-reviewed by Denise Vaillancourt and panel reviewed by Judyth Twigg. Aline Dukuze provided administrative support. v Principal Ratings Indicator ICR ICR Review PPAR Outcome Satisfactory Moderately satisfactory Satisfactory Bank performance Satisfactory Satisfactory Moderately satisfactory M&E quality Substantial Substantial Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. M&E = monitoring and evaluation; PPAR = Project Performance Assessment Report. Key Staff Responsible Management Appraisal Completion Project team leader Alessandra Marini Nelson Gutierrez Sector manager or practice manager Helena G. Ribe Pablo Gottret Sector director or senior Global Practice Keith Hansen Michal J. Rutkowski director Country director Carlos Felipe Jaramillo Alberto Rodriguez vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: (i) to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and (ii) to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website (http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vii Preface This is the Project Performance Assessment Report (PPAR) for the Juntos Results for Nutrition Project in Peru (P117310). An International Bank for Reconstruction and Development loan (IBRD 7961-PE) of $25 million for the Juntos Results for Nutrition Project was approved on March 8, 2011. Total project cost at appraisal was $54 million, which included a $29 million contribution from the government of Peru. The loan became effective on January 12, 2012, and closed on June 30, 2017, 15 months after the original closing date. Total project cost at closing was $25 million. The government of Peru did not contribute directly to project activities as planned at appraisal. Between 2012 and 2017, the government provided about $230 million in financing for its National Nutrition Program (Programa Articulado de Nutrición; PAN)—a program that the project also supported. All loan resources were disbursed. This report serves an accountability purpose by evaluating the extent to which the operation achieved its intended outcomes. It also seeks to draw lessons to inform and guide future investments in nutrition. The project was chosen for a field evaluation as part of a cluster of PPARs on nutrition and its relevance to a planned IEG study on nutrition. This assessment complements the Implementation and Completion Results Report (ICR) prepared by the World Bank with borrower contributions and the ICR Review (ICRR) performed by the Independent Evaluation Group (IEG) by providing an independent, field-based assessment more than two years after the project closed and a review of project documents, the ICR, aide-mémoire, supervision reports, and other relevant data, research, and material. On a mission to Peru in November 2018, IEG conducted interviews with officials and technical staff, service delivery personnel, local and municipal authorities, civil society organizations, beneficiaries, relevant development partners, and other involved persons. The IEG mission visited the districts of Namora, La Encañada, Baños del Inca, and Jesús in Cajamarca. In each location, IEG visited health centers and met with health personnel, beneficiaries, and local authorities. Interviews were also conducted with relevant World Bank staff in Lima, Peru, and Washington, DC. IEG gratefully acknowledges all those who made time for interviews and generously provided documents and information, especially those in the municipalities visited. IEG also expresses its gratitude to the World Bank office in Lima for the technical, logistical, and administrative support provided to the mission. Appendix I provides a list of persons consulted. viii Following standard IEG procedures, a copy of the draft PPAR was sent to the relevant government officials and organizations for their review and feedback. No comments were received. ix Summary In 2011, approximately 20 percent of Peru’s children younger than five years were chronically malnourished. In rural areas, chronic malnutrition was roughly three times higher than in urban areas—8 percent in urban areas compared with 26 percent in rural areas. Tackling chronic malnutrition became one of the government’s top priorities in the country’s efforts to achieve sustainable, equitable economic growth. At the time of project preparation, the government had taken major steps to tackle chronic malnutrition. Previous attempts by the government to reduce chronic malnutrition, such as the Juntos conditional cash transfer program introduced in 2005 and several food distribution programs, had not put Peru on the path to achieving the nutrition-related Millennium Development Goal. In 2007, the government introduced an integrated, multisectoral national nutrition strategy, Crecer (To Grow), to reduce poverty and child malnutrition. In 2008, the government introduced performance-based budgeting, addressing governance issues in nutrition through the National Nutrition Program (Programa Articulado Nutricional; PAN). The Juntos Results for Nutrition Project targeted both supply and demand improvements in basic preventive health and nutrition services. It supported PAN, a guaranteed package of health and nutrition services including the Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo; CRED) program, the Integral Health Insurance (Seguro Integral de Salud; SIS), a revamped Juntos conditional cash transfer program, and community-based demonstration sessions to promote behavior change. Its objectives were to (i) increase demand for nutrition services by strengthening the operational effectiveness of Juntos and (ii) improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the program, including Juntos. The project targeted 3 of the 14 poorest regions of Peru: Amazonas, Cajamarca, and Huánuco. The overall achievement of outcomes is rated satisfactory. The relevance of objectives is rated high. The project’s development objectives were responsive to and aligned with the country’s conditions at appraisal and completion and remain relevant today. Stunting rates in Peru had declined but were still relatively high despite a decade of economic growth, poverty reduction, and increased government spending. The objectives reflected the country’s priorities and national sector policies, embodied in Crecer and PAN, and were also aligned with the World Bank country strategies at project appraisal and completion. The Country Partnership Framework FY17–21 focuses on improving services for all citizens and targets improvements in both x nutrition-specific and nutrition-sensitive sectors: water and sanitation, health, and nutrition. The theory of change was well thought out, with adequate causal pathways linking the inputs, outputs, intermediate results, and outcomes. It was based on existing evidence and research on the relationship between early childhood development, nutrition, and stunting, and focused on high-impact interventions to tackle chronic malnutrition. Incentives, accountability, and institutional capacity were used strategically to improve the demand for and supply of basic preventive health and nutrition services. Incentives, accountability, and training were relied on to improve the quality of basic preventive health and nutrition services. The project made explicit assumptions about how inputs would translate into changes in behaviors and processes that were needed to achieve its objectives. A weakness in the theory of change was linking performance-based budgeting, social monitoring, and training to improved quality when quality had not been defined. Achievement of the objective to increase demand for nutrition services by strengthening the operational effectiveness of Juntos is rated substantial. Preparation and implementation of an affiliation strategy helped ensure children younger than 12 months old received Juntos benefits, were covered by SIS, and could receive the guaranteed package of basic health services. The percentage of Juntos children younger than 12 months receiving the full CRED scheme increased between 2011 and 2018. Evidence from the field suggests the mothers’ knowledge and awareness regarding nutrition and its effect on development also played a role in increasing the demand for nutrition services. But there is also evidence of constraints that the project had not anticipated or did not address, which continue to reduce mothers’ nutritional practices in the home, such as access to diversified foods, clean drinking water, remoteness, and social and cultural factors. Achievement of the objective to improve coverage of basic preventive health and nutrition services in communities covered under PAN including Juntos is rated substantial. By supporting early enrollment in SIS and guaranteeing basic preventive health and nutrition services, the project helped beneficiaries access services. The package of services was covered under SIS and delivered in eligible health facilities in health networks in the targeted regions. Coverage of the complete CRED scheme for children younger than 36 months improved in Amazonas and Huánuco, meeting the target set by the project. Only Cajamarca, the poorest region, did not meet the target. The objective to improve the quality of basic preventive health and nutrition services in the communities covered under PAN, including Juntos, is rated modest. The main outputs to support the achievement of this outcome were the strengthening of the xi government’s performance-based financing approach, the social monitoring strategy, provision of medical equipment, enhanced information systems, and training to ensure quality in the delivery of the basic package of health and nutrition services. Quality was never defined, and no outcome indicators were monitored. Fieldwork suggests that social monitoring was constructive and user-friendly. Overall, beneficiaries believed the technical quality of health and nutrition services had improved, and they appreciated health staff visiting communities, but some found the attitude of health staff lacking. Efficiency is rated as substantial. Evidence from impact evaluations of the Juntos program suggests use of health services increased, as did expenditure on nutritious foods. The evidence of achievement of the objectives and evidence from across disciplines that investments in young children’s health and nutrition—children younger than three years, when neural connections are rapidly forming—have high private and social rates of returns and are highly cost-effective, which suggests the project was cost- effective. Performance-based budgeting and PAN’s targeted programs increased productive and allocative efficiency. Implementation efficiency was generally satisfactory. The project laid a strong foundation for sustainability, but persistent challenges pose risks to outcomes. It is difficult to increase the supply of basic health and nutrition services in smaller and more remote health posts. Social norms in remote locations affect nutritional outcomes, and behaviors are hard to change. These social and economic challenges will need attention to sustain achievements. Political and institutional changes may affect institutional outcomes. Both the social monitoring strategy Padron Nominal and the Health Centers Survey, which were instrumental for project success, will require financial support from the government. Enrollment in the Juntos program may falter if families are not provided with incentives to enroll additional children. Currently, the program does not provide additional money for additional children. The World Bank’s performance is rated moderately satisfactory. Quality at entry is rated moderately satisfactory. The World Bank worked with the government of Peru (with different administrations) on nutrition for over a decade, building a relationship and supporting national policies. The long-standing dialogue and collaboration were critical in ensuring a project design that supported evidence-based solutions for improving childhood nutrition. The World Bank worked closely with the government to ensure the strategic relevance of the project design. The World Bank and the government opted to include components and activities in the project’s design that complemented PAN. However, there were shortcomings in design in terms of the monitoring of quality and the role envisioned for the Ministry of Health. Quality of supervision is also rated satisfactory. The World Bank was proactive during implementation, working with the government to improve performance and make xii course corrections. It carried out its fiduciary responsibilities satisfactorily. The project underwent minor restructurings to address challenges and keep it on track. Lessons Long-term engagement is critical to a deep understanding of complex malnutrition challenges. The World Bank’s long engagement in Peru was critical to the deep understanding of the country’s malnutrition challenges: long-term support, dialogue, and collaborative working relationships enabled an innovative project design that required coordination among various implementing agencies at multiple levels of service delivery. Attitudes toward nutrition need to change at all levels to make a difference. Peru’s commitment to reducing malnutrition started at the highest level of government. It required interinstitutional coordination from the Ministry of Economy and Finance, the Ministry of Health, the Ministry for Development and Social Inclusion, SIS, the National Civil Registry, and regional and local governments. Evidence from the field suggests attitudes are changing. For example, red, green, and yellow color-coded cards are being used to monitor and guide local-level goals and outcomes. And, in remote areas, health staff have begun giving priority to women and children who come to the health centers on market days. It is important to address both the supply and demand for health and nutrition services. Addressing the many factors that influence nutrition is complex and challenging. Increased demand must be met with an increase in the availability of services. In the case of Peru, financial incentives and accountability measures were used to increase both the supply and demand for services through performance-based budgets, conditional cash transfers, a guaranteed package of services, health insurance, and improvements in the identification of eligible households. Understanding the causes, consequences, and corrective actions required to reduce malnutrition can lead to changes in behaviors. At approval, mothers had little knowledge of a child’s normal growth process and little awareness of the link between a child’s growth and development and nutrition and health. Poor understanding of the problem contributed to malnutrition. Communication, CRED counseling, and community engagement helped increase awareness and understanding among mothers. Evidence from the field suggests the mothers’ knowledge and awareness regarding nutrition and its effect on development played a role in increasing the demand for nutrition services. Changes in beneficiaries’ behaviors cannot be assumed; they must be monitored. Given the complexity of the project and the causal chain, there were unexpected xiii implementation challenges or barriers to changing behavior. For example, remoteness was an issue that hindered mothers’ ability to attend CRED sessions. Other examples include the following: not boiling water because traditional containers are now made differently and a bad taste leaches into the water, many families do not have electricity and refrigeration to keep healthy foods, and people in higher areas cannot realistically eat meat more than once per week. In such cases, monitoring behaviors is critical to address unexpected constraints to behavior change. Sophie Sirtaine Acting Director Human Development and Economic Management Independent Evaluation Group xiv 1. Background and Context 1.1 Peru is an upper-middle-income country with a population of 32.2 million people (2017) and entrenched inequalities. The country’s population is concentrated on the coast around Lima, where almost a third of the population lives. In 2017, gross national income per capita was $11,773, yet 24 percent of the population lives in poverty (using the poverty line for an upper-middle-income country of $5.50 per day), 10 percent lives on $3.20 per day, and 4 percent lives on $1.90 per day.1 1.2 The Juntos Results for Nutrition Project aimed to address malnutrition to reduce inequality, reduce poverty, and help grow Peru’s economy sustainably and inclusively. Evidence shows nutritional deficiencies between the ages of zero (in utero) to three years impact brain development, resulting in cognitive and noncognitive developmental delays, which are in turn associated with the intergenerational transmission of poverty. In Peru’s poorest communities—the Andean and jungle regions of the country and in rural areas—access to health services remains limited, and rates of malnutrition and anemia are high. Pockets of poverty and entrenched inequality continue to affect long- term economic and social development. Project Context 1.3 In the decade before the project, Peru experienced strong economic growth accompanied by broad-based poverty reduction. Between 2000 and 2010, poverty was cut by more than half—from 31 percent to 15 percent. Extreme poverty fell by nearly a third, from 16 percent to 6 percent. 1.4 Human development outcomes lagged behind Peru’s sustained economic growth and poverty reduction. Health inequalities were large and persistent across socioeconomic groups and regions, particularly between rural areas and indigenous communities compared with national averages. At the time of project preparation, half a million children younger than five years suffered from chronic malnutrition (box 1.1). Rates of chronic malnutrition varied, with the poorest regions, at high altitude, having the highest rates (above 40 percent). Rural children were three times more likely to be chronically malnourished than urban children. In 2004, 28 percent of the poor and 44 percent of the extreme poor were chronically malnourished compared with 5 percent of the non-poor (INEI 2006). In 2001, 69 percent of children younger than two years were anemic (Encuesta Demográfica y de Salud Familiar 2001).2 1.5 Limited access to quality health services and health insurance resulted in low coverage of health services. At the time of preparation, approximately 25 percent of births occurred outside of health facilities, representing a missed opportunity to reduce maternal 1 and perinatal mortality and to improve the nutrition of mothers and children through mothers’ contact with health centers. Even with the creation of the Integral Health Insurance (Seguro Integral de Salud; SIS) and substantial budget increases, in real terms, to finance the provision of health services, most Peruvians in the lowest income quintiles lacked access to health insurance, including SIS. A significant barrier to accessing SIS was the inability to obtain the required National Identity Card for enrollment. 1.6 Lack of accountability and incentives, and poor understanding of the problem, also led to malnutrition. Malnutrition is the result of both supply- and demand-side factors. The health sector lacked clearly defined standards and operational frameworks, and government financing was not well targeted or linked to results (World Bank 2011c, 32–33). Parents, especially mothers, had little knowledge of a child’s growth process and what was considered normal at a given age, or how their child compared with that norm. Awareness of the adequacy of their children’s growth and the link between nutrition, health, and hygiene was low. Inadequate feeding practices, low rates of exclusive breastfeeding, inadequate supplementary feeding practices, and inadequate childhood health care were common. According to a Juntos impact evaluation, health services use was low—only 43 percent of children younger than five years who experienced illness received medical attention, only 36 percent were vaccinated, and only 46 percent underwent health checks (Perova and Vakis 2009). Inadequate hygiene practices also contributed to high malnutrition rates. A Cooperative for Assistance and Relief Everywhere study found that in several rural areas 85°percent to 100 percent of respondents reported inadequate hygiene (CARE 2000). 2 Box 1.1. Types of Malnutrition Chronic malnutrition is associated with low height-for-age, or stunting. It is the result of chronic or recurrent undernutrition, usually associated with poor socioeconomic conditions, poor maternal health and nutrition, frequent illness, or inappropriate feeding and care in early childhood. Growth is considered stunted if a child is more than two standard deviations below the mean based on age and sex. Acute malnutrition is associated with low weight-for-height or wasting. It usually indicates recent and severe weight loss, often for lack of sufficient food or due to infectious diseases. Underweight is associated with low weight-for-age and can include stunting, wasting, or both. Micronutrient malnutrition is caused by inadequate intake of vitamins and minerals (such as iodine, vitamin A, and iron), which enable the body to produce enzymes, hormones, and other substances. Micronutrient deficiency represents a major threat to the health and development of children and pregnant women in low-income countries. Source: https://www.who.int/news-room/fact-sheets/detail/malnutrition (dated February 26, 2018). 1.7 At the time of project preparation, the government had taken major steps to tackle chronic malnutrition concentrating on the poorest regions with the highest malnutrition rates. In 2007, the government introduced an integrated, multisectoral, national nutrition strategy, Crecer (To Grow), to reduce poverty and child malnutrition by integrating its health and nutrition programs. In 2008, the government introduced the National Nutrition Program (Programa Articulado Nutricional; PAN), which provided an incentive to reduce chronic malnutrition through performance-based budgeting and required coordination of government agencies and stakeholders at various levels. PAN was the first of five strategic programs promoted by the performance-based budgeting policy of the Ministry of Economy and Finance (Ministerio de Economía y Finanzas; MEF).3 PAN concentrated efforts not only on regions with high malnutrition but also on actions that had been tested and proven to be cost-effective. In line with international evidence, the government had implemented child growth monitoring associated with individualized counseling to mothers through the Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo; CRED) program. In addition to supply-side interventions, the government implemented the Juntos conditional cash transfer (CCT) program. International evidence suggests that chronic malnutrition can be significantly reduced when demand-side interventions like Juntos are accompanied by adequate coverage and quality of health and nutrition services to complement the income effects of the transfers. (For more on strategies and programs to address malnutrition see appendix B.) 1.8 Juntos’ targeting mechanism prioritized poor and extremely poor households, but there were challenges. At project approval, the efficiency of the CCT was being 3 challenged by issues with the verification of compliance with coresponsibilities, limited availability of basic health and nutrition services, challenges in the affiliation process to include newborns, and errors related to eligible households. The Juntos Results for Nutrition Project provided technical and financial support to address the challenges. 1.9 At the time of project approval, the World Bank was supporting the government’s nutrition efforts through an array of instruments. The project complemented ongoing World Bank projects and advisory and analytic services and was well-coordinated with the efforts of the European Commission, the Inter-American Development Bank, KfW (formerly Kreditanstalt für Wiederaufbau Bankengruppe), the United Nations Children’s Fund, and Cooperative for Assistance and Relief Everywhere (see appendix H). 1.10 The European Community was also working to reduce malnutrition in Peru. The Europan program (a €60 million grant) targeted Peru’s three poorest regions. It was the first program to introduce convenio de apoyo presupuestal—budget support agreements between local governments and the MEF. These convenios were designed to allocate resources more efficiently in the fight against malnutrition by focusing on results. The program supported the goal of PAN and tracked progress in selected indicators, such as increased coverage of CRED child immunizations. 1.11 The Juntos Results for Nutrition Project intended to address the key causes of malnutrition. Its design built on evidence from impact evaluations that demonstrated what worked to change behaviors. An impact evaluation of the Juntos CCT program results (2005–08) highlighted its achievements related to poverty reduction, use of health services, and increased expenditure on more nutritious foods. However, consistent with international evidence at the time, no evidence of the program’s effect on chronic malnutrition was found. The probable causes of this were inadequate quantity and quality in the supply of basic health and nutrition services and inattention to activities to ensure attitudes and behavioral changes (Perova and Vakis 2009, 2012). Further research and experimentation by the World Bank led to recommendations to improve the design and implementation of the Juntos program. 1.12 The government’s program supported under the proposed operation aimed to close PAN’s coverage gaps and solve its implementation challenges where Juntos operates. A key operational challenge was to ensure that PAN was built on an accountability framework that set mechanisms and incentives that tie the budget formulation with the performance of fragmented executing units. Rigidities in the allocation of budget resources along the chain of service providers prevented the flexibility required to effectively assign funds at each level to meet these goals. 4 2. Objectives and Design 2.1 The Juntos Results for Nutrition Project was designed to support the demand, supply, and governance of nutrition services provided by the government of Peru. The project was aligned with PAN: the Juntos CCT program and early affiliation (or enrollment of newborns) into the SIS are key features of PAN’s framework to reduce chronic malnutrition. Objectives 2.2 The project’s development objectives as laid out in the loan agreement were not materially different from those in the project appraisal document. The development objectives in the loan agreement were to increase demand for nutrition services by strengthening the operational effectiveness of Juntos and to improve coverage and quality of the supply of basic preventive health and nutrition services in the communities covered under the program, including Juntos (World Bank 2011b, 5). 2.3 The statement of objectives contains three distinct objectives. The objectives used to evaluate project effectiveness separate coverage from quality. Therefore, the project’s development objectives were to (i) increase demand for nutrition services by strengthening the operational effectiveness of Juntos, (ii) improve coverage of basic preventive health and nutrition services in the communities covered under PAN including Juntos, and (iii) improve the quality of the supply of basic preventive health and nutrition services in the communities covered under PAN including Juntos (World Bank 2011c). Objective (i) concentrates on Juntos beneficiaries and objectives (ii) and (iii) target the entire community covered by PAN whether a Juntos beneficiary or not. 2.4 The project targeted three of Peru’s poorest regions—Amazonas, Cajamarca, and Huánuco.4 The regions are rural, with an estimated population of 370,262 families, representing 5.8 percent of Peru’s population. Of these, approximately 15 percent (58,076 families) were beneficiaries of Juntos. Relevance of the Objectives 2.5 The relevance of objectives is rated high. 2.6 The project’s development objectives responded to one of Peru’s most urgent development challenges. They were responsive to and aligned with country conditions at the time of project appraisal. The government had committed to reduce the national chronic malnutrition rate by 9 percentage points between 2008 and 2011. The government’s national plan for 2016–21 aims to reduce both stunting and anemia in children. It has set a target of 6 percent for stunting and 19 percent for anemia (from 5 44 percent). It also targeted opportunities in social investments in access to water and sanitation services, health sector reforms, malnutrition, anemia, and food security and rural development, all of which are correlated with nutrition (World Bank Group 2017). Chronic malnutrition causes permanent damage to a child’s intellectual development and irreversible losses of human capital formation, affecting future productivity and economic growth. It will be difficult for Peru to make substantial progress in reducing poverty and inequality and sustain growth in the long term unless access to quality health services improves and stunting rates decline. 2.7 The objectives were highly relevant to the World Bank Group Country Partnership Strategy at project approval in fiscal year (FY)07–11. The overarching goals of the strategy were higher and sustained economic growth and poverty reduction (World Bank 2006, 7). Inadequate human capital and the poor’s limited access to services to accumulate human capital were core causes of poverty, particularly in education, health, nutrition, and social assistance (World Bank 2006, 11). The objectives to improve the demand for nutrition services, access to health services, and the quality of health services were necessary conditions for two of the Country Partnership Strategy’s three pillars: accelerating economic growth and enhancing social development (the third being modernizing the state). The shared vision of the government and the Bank Group recognized the need for policies to promote a new social contract in education, health, and nutrition that addressed the basic needs of the poor and disadvantaged to reduce poverty and inequality. These had stagnated during a period of rapid growth and were unsustainable in the long term. 2.8 The objectives were highly relevant to the Bank Group’s Country Partnership Framework for FY17–21 at project closure. The strategy focused on improved productivity for growth (pillar I) and improved services for all citizens (pillar II). The Systematic Country Diagnostic identified the need to improve human capital to sustainable economic growth and inclusion of the poor (World Bank 2017). The Bank Group’s strategy prioritized improving public services, through modernizing service delivery in health and nutrition, to increase access and quality and provide a variety of nutrition-related activities such as deworming and micronutrient supplements, which have been linked to higher learning and in turn have been linked to higher economic growth (Hanushek 2013). Human capital development—increasing cognitive abilities— is directly relevant to economic growth and opportunities for all Peruvians to enhance their livelihoods. The Country Partnership Framework (FY17–21) specifically targets improvements in three public services necessary for improved health and nutrition: water and sanitation, health, and nutrition (World Bank 2017, 23). 6 Project Design 2.9 The $25 million Specific Investment Loan supported PAN during 2011–16. The proposed sectorwide approach (SWAp), the first implemented in Peru’s social sector, supported PAN’s goal to reduce chronic malnutrition. Its SWAp characteristics included country leadership and ownership and emphasis on coherence—a budget process linked to results and improved information systems to monitor results. The Juntos Results for Nutrition Project provided project support. The SWAp was consistent with the need for interinstitutional coordination, use and strengthening of government systems, and the ability to link disbursement to sector results (versus general reforms), which was more consistent with PAN (World Bank 2011c, 13).5 PAN combined different programs and interventions and used performance-based budgeting to achieve its goal. Project design relied heavily on incentives to change behaviors to increase use of health (and education) services and to improve the coverage and quality of health services provided at the municipal and health care provider levels. It included performance-based disbursements in the form of CCTs for basic health and nutrition services, and performance-based budgets and disbursements for the supply of basic health and nutrition services. The project was well-coordinated with World Bank and other donors’ nutrition activities in Peru. (For details on the government’s nutrition strategies, Juntos, PAN, and the nutrition landscape, see appendixes B, E, F, and H.) Components 2.10 The Juntos Results for Nutrition Project financed three components with specific activities intended to achieve its development objectives. The components sought to (i) strengthen and consolidate the design and operational capacity of the Juntos CCT program for families with children younger than three years, (ii) improve the coverage and quality of basic preventive health and nutrition services in Juntos areas, and (iii) strengthen the government’s capacity to influence nutritional outcomes through improved budgetary planning and better monitoring of results for selected activities of PAN (appendix C, table C.1). 2.11 Component 1. Strengthen and consolidate the design and operational capacity of the Juntos CCT program (appraisal, $5.5 million; actual, $5.5 million). The component supported (i) readjustment of the Juntos program’s implementation strategy to strengthen the communication strategy and information system to encourage families to enroll young children in Juntos; (ii) incentives to enroll newborns and children younger than one year in the Juntos program including differentiating the amount of the cash transfer based on number of eligible children, harmonizing the Juntos program and health information systems, and helping mothers understand the objectives of Juntos and the health and 7 nutrition coresponsibilities; and (iii) incentives to improve the verification of the health coresponsibilities for children younger than three years affiliated with the Juntos program. 2.12 Component 2. Improve the coverage and quality of the provision of basic preventive health and nutrition services in Juntos areas (appraisal, $5.5 million; actual, $5.5 million). The component provided additional funding for PAN, under the Health Facility Capitation Payment, to ensure provision of a guaranteed package of basic health and nutrition services to a defined population and allow beneficiaries of the Juntos program to comply with their health and nutrition coresponsibilities. 2.13 Component 3: Strengthen the government’s capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results for selected activities of PAN (appraisal, $14 million; actual, $14 million). The component supported the monitoring capacity of performance-based budgeting for PAN. Theory of Change 2.14 Project design was evidence based. The project’s theory of change is based on existing evidence and research on the relationship between early childhood development, nutrition, and stunting. Analytical work conducted by the World Bank highlighted factors that facilitated and hindered previous efforts to influence childhood malnutrition in Peru. An impact evaluation on the restructured Juntos CCT program in the Apurímac region of Peru provided the evidence base for the Juntos Results for Nutrition Project.6 The findings also resonated with the broader literature. 1.1 Project design focused on outputs and behavior changes required to reduce stunting. Successful accomplishment of the project’s development objectives relied on achieving changes in processes and behaviors (for example, of local governments, service providers, and beneficiaries) that are necessary to reduce malnutrition. Project design assumed that changes in behaviors and processes would contribute to reducing chronic malnutrition. 2.15 The project supported PAN’s overarching objective to reduce chronic malnutrition focusing on several causal pathways. The project’s design assumed increased enrollment in Juntos would lead to increased demand for nutrition services. Increased funding for a guaranteed package of health and nutrition services, covered under SIS, would also increase the coverage and use of basic preventive health and nutrition services. The causal pathway between activities, outputs, and improvement in the quality of basic preventive health and nutrition services was weaker. Quality was not defined, and improvements relied primarily on accountability mechanisms, such as the social monitoring strategy (Monitoreo Social), performance-based budgeting, and having the right equipment and trained staff. 8 2.16 Project design assumed demand for basic health and nutrition services would increase through incentives, accountability, and knowledge. The theory of change assumed that by readjusting Juntos’ implementation strategy, strengthening its communication strategy, and strengthening its information system, families would be encouraged to enroll young children. The financial incentive provided—conditioned on regular health visits for pregnant women and children younger than five years and enhanced verification of beneficiaries’ coresponsibilities—should increase the demand for basic preventive health and nutrition services and motivate compliance. Stronger incentives and ease of compliance, for example, differentiating the amount of the cash transfer based on the number of eligible beneficiaries, and harmonizing Juntos and health information systems, would strengthen the link between the financial incentive and behavior changes (using basic health and nutrition services). 2.17 Knowledge and communication activities had been highlighted as bottlenecks and included in project design. Demand would increase, for example, if mothers understood the objectives of Juntos and the health and nutrition coresponsibilities, information on CRED was made available and easy to understand, and information on key nutrition services and nutrition outcomes was user-friendly. To reach the poorest families, these communication strategies were to be made available to non–Spanish- speaking and indigenous peoples to increase their demand for basic preventive health and nutrition services. 2.18 The project was designed to meet increased demand with an increased availability of services. Increased availability allowed improvement in coverage of basic preventive health and nutrition services. The theory of change assumed an incentive—in the form of an increased budget—would increase the availability or supply of basic preventive health and nutrition services, including full coverage of a basic package of essential services for mothers and children. Again, the project assumed that coverage would increase if it was easier to identify beneficiaries and for beneficiaries to obtain the legal and administrative documents required to receive Juntos and the basic health and nutrition services. 2.19 Incentives and accountability measures were used to improve the quality of basic preventive health and nutrition services. The social monitoring strategy is a participatory approach to monitoring (including local government, local civil society organizations, media, users, and service providers) that oversees nutrition results and the quality of basic health and nutrition services at the district level. Performance-based budgets provided the incentive to produce results, disbursing against measures that were assumed would lead to improvements in quality, such as the number of municipalities with a social monitoring system. Improvements in quality were linked to improvements in the quality of the budget process. The design assumed that improving 9 information systems for better and updated information, monitoring indicators for the achievement of targets, and using data and indicators to drive decisions would improve the quality of basic preventive health and nutrition services. 2.20 Increasing institutional capacity brought all the pieces together. Project activities to improve the synergies between information systems, to promote birth registration, and to obtain identity cards for women and children were assumed would lead to overall project effectiveness, including by improving verification of coresponsibilities, such as CRED, to remain enrolled in Juntos. The project supported a strategy to get newborns registered and covered by SIS and created a roster of children—the Padron Nominal—that highlights gaps in coverage of identity documents. The project also got mayors of small towns involved.7 3. Implementation Project Restructuring 3.1 The project was restructured five times, but the development objectives remained unchanged.8 The restructurings were minor, added critical activities to address gaps and facilitate achieving objectives, addressed issues related to the delay in effectiveness, aligned project indicators with government program indicators to facilitate monitoring, and helped keep the project on track. (For a summary, see appendix D, table D.1.) Implementation Arrangements 3.2 Implementation was multi-institutional and built on existing institutional arrangements. Multiple institutions were responsible for different components of the project (see appendix D, table D.2). Although this introduced complexity, it ultimately paid off. The project used existing institutional structures for performance-based budgeting, nutrition, and health to strengthen the effectiveness of the institutions to improve nutritional outcomes. MEF’s National Directorate for Public Budget was responsible for the implementation of all aspects of the performance-based budget.9 3.3 Implementation was multileveled—the central level facilitated engagement at the local level. MEF effectively pulled together all elements of the theory of change. MEF was the lead institution for technical aspects of project implementation. The convenios were critical. They established an incentive-based mechanism between MEF and local governments supported and enabled local activities, which were key to achieving the project’s intermediate outcomes and development objectives.10 10 Implementation Experience 3.4 Effectiveness took longer than expected. The project took close to a year to become effective, delaying implementation. The project was expected to start simultaneously with the Europan project, which targeted the poorest regions; the World Bank’s project targeted the three next poorest. Instead, the project built on the lessons from the Europan project. In the end, this required an extension of the project to complete activities and foster sustainability. 3.5 The project was well-coordinated and consistent with the government’s policy on nutrition. There was a winning combination: a good theory of change, a good causal path, and good technical teams and data. Well-defined national public policies, supported by budget programs (such as PAN and Programa Materno Infantil), established overarching objectives. Created in 2011, the Ministry for Development and Social Inclusion (Ministerio de Desarrollo e Inclusión Social; MIDIS) together with MEF and the Ministry of Health (Ministerio de Salud; MINSA) established indicators to monitor results. Incentives existed for local governments and other key actors to both perform and institutionalize the process. 3.6 The role of MINSA was larger than anticipated at approval. Although the project appraisal document did not foresee a role for MINSA, the ministry played a crucial role from the very beginning of implementation, notably by providing data to MEF and SIS, upgrading information systems (such as certificado de nacido vivo and Padron Nominal), and preparing norms and guidelines for the provision of health and nutrition services and quality improvements. This information and the support were critical for the performance-based budget arrangements. It is worth mentioning that MINSA also established a strategic alliance with the National Civil Registry (Registro Nacional de Identidad y Estado Civil; RENIEC) that was critical to support the achievements in nutrition outcomes. Implementation of Monitoring and Evaluation 3.7 Monitoring and evaluation activities were a key element of the design. The project supported the government’s efforts to strengthen its existing information systems. The project successfully strengthened the Juntos information system to improve monitoring of the program’s performance, including verification of compliance with coresponsibilities by beneficiaries. 3.8 Upgrading the Padron Nominal and the social monitoring strategy were two of the most successful activities. These two monitoring and evaluation activities were foundational components in breaking institutional and behavioral bottlenecks, respectively, to better nutrition. The Padron Nominal, which had previously existed as 11 Excel files, was transformed into a census of children younger than five years (helping provide identity to every newborn). At the end of 2018, approximately 1,700 municipalities and 300 health centers were registered and using the Padron Nominal. Today, the Padron Nominal is an essential tool. The social monitoring strategy improved awareness and accountability at the local level. Safeguards Compliance 3.9 The project triggered the Indigenous People Policy, OP4.10. No environmental safeguard policy was triggered. Two documents guided the social safeguards of the project: the Indigenous Populations Strategy Framework carried out in 2009 and a social assessment carried out in 2010 in the three departments covered by the project. The framework provided guidance to overcome barriers to accessing basic health and nutrition services by indigenous households, where chronic malnutrition was higher than among the general population. Design features supported by the Indigenous People Strategic Framework and the social assessment included a disaggregation of the project information (indicators) for indigenous and nonindigenous populations to allow purposeful monitoring, and the design of a communication campaign and the social monitoring mechanism to ensure culturally adequate participation by indigenous households. An update to the social assessment was completed in March 2017. Financial Management and Procurement 3.10 Procurement was rated moderately satisfactory or better throughout project implementation. The Sectoral Projects Coordinating Unit was responsible for the fiduciary aspects of the project. No major procurement issues occurred during implementation. However, there were weaknesses and bottlenecks in carrying out procurement activities, in part because of the need to coordinate between several technical teams but also because procurement processes needed improvement. The midterm review highlighted the main delays in procurement and the steps to be taken to overcome them. Training on procurement rules was carried out for the staff in charge of procurement to improve the project’s performance. These trained staff then trained the technical staff in the different ministries involved in project implementation. 3.11 Financial management was satisfactory with some shortcomings during implementation. The initial institutional financial management arrangements seemed adequate, but there were delays in disbursements because of Peru’s legal and regulatory framework, which allowed disbursement against results, not budget line items. After resolution of this issue, funds flowed smoothly. However, the technical audit for performance-based disbursements was delayed due to bureaucratic procedures. The technical audit was split into a financial audit and a unit cost audit—fulfilling the government’s requirement for financial audits and the World Bank’s requirement for an 12 expert in unit cost methodology to conduct the technical audit of the performance-based disbursements. Otherwise, the government carried out timely and clean financial audits for the loan resources managed by the Sectoral Projects Coordinating Unit. 4. Achievement of the Objectives 4.1 The project’s scope was limited, but its support was key to the implementation of PAN and the goal of reducing chronic malnutrition. The project supported demand, supply, and institutional capacity building and governance of the government’s system to reduce chronic malnutrition. The development objectives reflected these three factors associated with reducing childhood malnutrition rates. First, to reduce stunting, parents require financial support and knowledge. Financial support increases income and, ultimately, demand for normal goods, like nutritious foods. Knowledge has the potential to increase demand through behavior change if parents understand their children’s growth process and the role of nutritious foods in the growth process and change their behavior at home (child care behavior) and by enrolling in and utilizing services available to them. Second, the supply of services must be able to support households’ demand for services, implying a need for improved coverage of basic preventive health and nutrition services. Third, the government and providers must be held accountable for nutrition results, implying a need to improve capacity and the quality of basic preventive health and nutrition services. 4.2 The key performance indicators provided a combined picture of demand for and supply of basic preventive health and nutrition services. Coverage of CRED by Juntos children younger than 12 months is a proxy for the demand for nutrition services, predicated on the assumption that the financial incentive and coresponsibilities would increase demand for these services. Since CRED includes health and nutrition services for all children younger than 36 months (not just Juntos children), it is a proxy for demand for basic preventive health and nutrition services for both age groups. Provision of a guaranteed package of basic preventive health and nutrition services is an indicator of the increase in the availability of services for children younger than 12 months as well. The two indicators measure use of services, which captures both demand and supply. 4.3 The quality dimension was poorly defined. No indicator was provided to measure improvement in the quality of basic preventive health and nutrition services, such as a patient-centered measure of service quality (see appendix E for the performance indicators). 13 Objective 1: Increase Demand for Nutrition Services by Strengthening the Operational Effectiveness of Juntos 4.4 The achievement of this objective is rated substantial. Outputs and Intermediate Outcomes 4.5 A key output of the project was a strategy to ensure newborns were enrolled in Juntos and SIS. The project supported both the preparation and implementation of an affiliation strategy focused on removing obstacles to enrolling children younger than 12 months in both programs, thus ensuring they receive both Juntos benefits and the guaranteed package of basic health services (see appendix E). Another key output was the involvement and support of the municipalities in the cycle of documentation and affiliation of children younger than 12 months in SIS and Juntos. The project supported the preparation of educational materials, in Spanish and indigenous languages, to disseminate information on Juntos to beneficiary households, including the health coresponsibilities of the program. The project also supported the improvement and expansion of the Juntos CCT Information System, which improves the program’s ability to monitor beneficiary households. The CCT Information System has been a web-based platform since 2014, with interoperability between SIS and the Ministry of Education, and connected to the Padron Nominal. 4.6 The project also contributed to transforming the children’s Padron Nominal into a stronger census.11 The Padron Nominal now identifies and monitors children at the individual level and provides real-time data to MINSA, MEF, MIDIS, and others. It has allowed MINSA’s health information system and SIS to work together and also contributed to the definition of protocols and simplification of processes at all levels. When the project started, the Padron Nominal already existed but in a very basic format. The project transformed it into a system that consolidated its data with data from Juntos, SIS, and health centers—creating a database of critical information on all Peruvian children younger than five years. Starting with the three departments of the project, the Padron Nominal was expanded to all other departments in Peru. By completion, MINSA had prepared a directive on the Padron Nominal at the district level, a sign of further institutionalization, contributing to making the database permanent in the district.12 The Padron Nominal is critical for the articulation of PAN; it has contributed to change the mentality of the health centers, is being used as part of the country’s strategy on anemia, and has been used to develop a registry of pregnant mothers. Overall, the Padron Nominal data have allowed activities and resources to be based and allocated on real demand and evidence. It has significantly contributed to efficiently and effectively achieving health outcomes. 14 4.7 The project met or exceeded most of its intermediate outcome targets, although outcomes varied by region. The share of children registered in the CCT Information System before they turned a month old surpassed its target of 40 percent (appendix E), but it varied considerably among Amazonas (28 percent), Cajamarca (41 percent), and Huánuco (64 percent). There is less variation in the share of children younger than two years registered in CCT Information System, with a birth certificate, identification document, and enrolled in SIS in the project regions.13 These variations suggest the project had mixed success in achieving some of its targets in some regions. 4.8 The project supported training on nutritious and healthy food preparation, interpersonal counseling, and technical demonstration sessions for Juntos parents. The target for mothers attending demonstration sessions was not met; however, nutrition training happened at CRED visits, nutrition-related CRED checkups for children included encouraging exclusive breastfeeding and adequate complementary feeding, counseling parents on feeding practices, and monitoring children’s growth. Trained health center personnel counseled parents who visited the health centers. IEG fieldwork confirmed that health personnel provided substantial interpersonal counseling on how to feed children and how to prepare and preserve foods. These sessions were organized by the field staff of Juntos with the help of women community leaders who help foster participation and follow up after the sessions. Fieldwork also found that the use of the PAN indicators with red, green, and yellow color coding to guide goals and outcomes and the incentives for improving management and performance in municipalities helped motivate mothers, communities, health staff, and local governments. 4.9 The project also supported better information on coverage of CRED checkups by mother tongue and ethnicity. As further detailed in the Did the Demand for Nutrition Services Increase section, these data allowed the government to estimate the demand for services and access gaps in nutrition services for indigenous households. The data also provided evidence of improved access to nutrition services among Peru’s diverse population. Did Demand for Nutrition Services Increase? 4.10 The demand for nutrition services increased in Amazonas, Cajamarca, and Huánuco (combined). Removing income constraints and providing nutrition-related CRED services likely contributed to increased demand for nutrition. The increase in coverage of Juntos children younger than 12 months surpassed the original and revised targets by project closure and had continued to increase at the time of the Project Performance Assessment Report (PPAR; Table 4.1). In 2018, the percentage of Juntos children younger than 12 months receiving their complete CRED program in Amazonas, Cajamarca, and Huánuco (combined) was 89 percent. These findings are aligned with 15 evidence from an impact evaluation conducted on the Apurímac pilot. Evidence suggested that restructuring the Juntos program would increase expenditure on more nutritious foods (Perova and Vakis 2009). Table 4.1. Juntos Children Younger than 12 Months Who Received the Complete CRED Scheme (percent) Baseline Actual PPAR Indicator or Target (2011) Original Revised (2017) (2018) Juntos children (no.) 64 80 80 86 89 Source: Instituto Nacional de Estadística (National Institute for Statistics)—Encuesta Demográfica y de Salud Familiar (2011–17) data (https://proyectos.inei.gob.pe/endes/resultados.asp); Ministry for Development and Social Inclusion 2018 data. Note: CRED = Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo). 4.11 The demand for nutrition services for children younger than three years increased across the country but did not achieve its target in all project regions. Coverage of the complete CRED scheme for children younger than 36 months also improved (table 4.2); although at project closure Amazonas and Huánuco had met the revised target (73 percent, see appendix E), Cajamarca, the poorest region, had not met the target. At the time of the PPAR, the demand for nutrition services for this age group had fallen significantly in Huánuco, leaving Amazonas the only district meeting the target. 16 Table 4.2. Children Younger than 36 Months Who Received the Complete CRED Scheme (percent) Area 2012 2013 2017 2018 National 50.8 50.5 60.1 60.8 Urban, national 46.9 47.0 58.0 58.5 Rural, national 58.5 58.1 65.9 67.5 Amazonas 60.2 59.8 73.0 73.5 Cajamarca 68.6 68.3 65.0 65.3 Huánuco 68.9 69.3 72.7 67.4 Juntos nationally 62.0 60.8 65.5 65.4 Juntos districts in 67.2 66.9 69.9 — Amazonas, Cajamarca, and Huánuco Juntos districts in 65.4 56.6 60.3 — Amazonas, Cajamarca, and Huánuco; mothers of indigenous language Source: Instituto Nacional de Estadística (National Institute for Statistics)—Encuesta Demográfica y de Salud Familiar (https://proyectos.inei.gob.pe/endes/resultados.asp). Note: The definition of the variable is slightly different from the indicator reported during implementation. Disaggregated data for Juntos districts are not available in 2018. CRED = Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo). 4.12 The demand for nutrition services among indigenous families in Juntos districts did not increase. The project did not set targets for indigenous Peruvians. However, progress was supposed to be tracked (World Bank 2011c, 47). In Juntos districts in Amazonas, Cajamarca, and Huánuco, for mothers who speak indigenous languages, the coverage of complete CRED for children younger than 36 months ranged from 57 percent to 65 percent (table 4.2). As a comparison, the percentage of all children in Juntos districts receiving their complete CRED scheme was substantially higher in 2013 and at project completion in 2017 suggesting efforts to target women speaking indigenous languages were either not sufficient or could not remove other barriers to bringing children to health centers on time for the CRED visits (for example, social, cultural, or remoteness). 4.13 Participation in nutrition demonstration sessions signals increased demand for nutrition services and behavioral changes. During the PPAR mission, focus groups with Juntos staff and Juntos beneficiary mothers confirmed mothers’ active and frequent participation in demonstration sessions on food preparation (and other content) carried out by Juntos field staff or mother leaders and that mothers were engaged and active in their communities (checking on their neighbors). The women considered the sessions very informative and helpful. Staff relied on mother leaders to pass along information and organize activities and monitor changes in their communities. 17 4.14 The project’s information, education, and communication activities, affiliation strategy, and counseling sessions during CRED visits were critical to improving demand. The PPAR mission observed that mothers understood the relationship between nutrition and growth and the adequacy of their children’s growth—an issue identified before the restructuring of Juntos. Mothers were bringing their children to the health centers to comply with their coresponsibilities regarding health, so children received growth controls and immunizations and mothers received nutrition counseling (pointing to much higher awareness levels). Field staff report parents’ behaviors are changing. For example, they observed healthy practices in the households during unannounced visits, husbands had begun to attend demonstration sessions, mothers were actively engaged in the demonstration sessions, and young first-time pregnant women were already attending health centers for pregnancy controls. Interviews with health center staff showed that they believed more time spent in counseling support would strengthen these changes in behavior. Nutritional education has been proven to have reduced stunting in Peru if the availability of food is not a constraint (Penny et al. 2005). 4.15 The project helped make it easier to obtain a birth certificate and a Código Único de Identificación (Unique Identification Number). With an understanding of the importance of nutrition on child development, the project changed the way parents perceive the importance of obtaining identification for their children. Interviews with mothers in Cajamarca highlighted that the women were aware and fully invested in the necessary processes to ensure they can receive basic preventive health and nutrition services, which are required for the normal, healthy growth of their children. 4.16 World Bank support contributed to the efficacy of the project in the targeted regions, but other donors were also active in these areas and nationally. Other support to increase the demand for basic health and nutrition services in these areas came from the Japanese in Cajamarca, with a focus on educating beneficiaries through informational kits related to nutrition and care of children. The European Union, the Cooperative for Assistance and Relief Everywhere, the United Nations Children’s Fund, and the Inter-American Development Bank were part of these efforts as well. 4.17 The PPAR mission found that some challenges remain. Some families are too remote, and the costs of enrolling in the program exceed the (private) benefits of the program. A stronger financial incentive may be effective in overcoming the barriers for this group. Additionally, not all health centers have offices of the RENIEC, which is critical for early enrollment in SIS and Juntos (making registration simple for institutional births). 18 4.18 Monitoring for anemia during regular growth monitoring was a missed opportunity, which is now being addressed. Chronic malnutrition rates have decreased, but the incidence of anemia is still high and unchanged. The government has recently placed the eradication of anemia at the center of its maternal and early childhood development policy. The PPAR mission observed that during regular checkups, children are now being checked for anemia and mothers are aware of their children’s iron levels. Screening for anemia is now part of the regular assessment of health status and a health outcome linked to the child’s well-being and growth. Objective 2: Improve Coverage of the Basic Preventive Health and Nutrition Services In Communities Covered under PAN Including Juntos 4.19 The achievement of this objective is rated as substantial. Outputs and Intermediate Outcomes 4.20 Technical assistance conducted by the World Bank identified limited supply as a bottleneck to improvement in nutrition. The project supported a basic package of health and nutrition services covered by SIS, activities to ensure beneficiaries enrolled in SIS, and other activities designed to close access gaps through improved data and better monitoring, supervision, evaluation, and control. 4.21 The project supported a guaranteed set of services for basic preventive health and nutrition through a health facility capitation payment. Aggregated capitation payment amounts were based on the number of health facilities, health facilities’ unit costs, and estimated use of the guaranteed package of health and nutrition services by the defined population (fixed payment). Capitation amounts were also allocated based on the coverage of complete CRED and immunization schemes for children younger than six months in the targeted Juntos areas and attendance at a minimum of one community demonstration session for mothers with children younger than 36 months (variable payment). 4.22 By supporting early enrollment in SIS and guaranteeing basic preventive health and nutrition services, the project helped beneficiaries access services. The package of services was covered under SIS and delivered in eligible health facilities in health networks in the targeted regions. The package included services related to immunizations, CRED, and counseling for parents. Eligible facilities ranged from regional hospitals to health posts—part of the microhealth network—which regularly provided health services to the targeted population in the three departments. MINSA and SIS provided the technical norms for the content of the basic services (including 19 CRED) and costed them as well. Health centers at all levels in the project’s three regions were adequately equipped to provide the basic CRED package and vaccines. Ninety- one percent of health facilities had the basic equipment to give CRED services and vaccines in Juntos districts, exceeding the target (40 percent). The percentage of children younger than 36 months affiliated with SIS and with complete and timely CRED scheme in the areas of intervention of Juntos increased from 43 percent in December 2011 to 88.5 percent in June 2017, exceeding the revised target of 65 percent. 4.23 The Padron Nominal highlighted gaps in access to services, especially for indigenous populations. The project supported preparation and dissemination of data on CRED for children up to 36 months old, by ethnicity and language. This disaggregation and reporting by mothers whose first language is indigenous was expected to contribute to greater awareness of the services being provided and health status of these children, and the ability to target gaps in the provision of health services for these populations. From the start, there was evidence of a gap in access and adequacy of the services for indigenous populations. The project supported training materials and worked to devise a strategy to promote a culturally sensitive way to deliver health services to overcome the inequality with indigenous households, as described in the safeguards section. 4.24 Support for personal identification (official documentation) was instrumental in helping beneficiaries access services. The project also supported activities that created the infrastructure to ensure children received a Certificate of Live Birth, which is required for newborns to access services. The project supported the strengthening of the Registry Office for Civil Status; and the establishment of Auxiliary Registry Offices in hospitals and other facilities where women gave birth (in coordination with the MINSA and RENIEC). Did Coverage of Basic Preventive Health and Nutrition Services Improve? 4.25 Coverage of basic preventive health and nutrition services improved across the country and in project regions. The demand for nutrition services was matched by a guaranteed package of basic preventive health and nutrition services (including CRED) and increased use of these services by children younger than 12 months and 36 months (tables 4.1 and 4.2). The increase in coverage or use of CRED for Juntos children younger than 36 months in the project’s districts was not significantly different from the national Juntos increase. For all children younger than 36 months, the trends in Amazonas and Huánuco are also comparable to the national trend. These data suggest that the project’s support to Juntos, PAN, and SIS, among other activities, addressed barriers to service use for many poor families (given the target districts), but possibly not for indigenous families. 20 4.26 Beyond the increased coverage of the CRED scheme, immunization rates also increased in two of three targeted regions.14 Immunization rates in Amazonas increased from 60 to 90 percent between 2012 and 2017, and from 80 to 90 percent in Huánuco. Cajamarca’s rates remained stable. These are substantial improvements compared with the change at the national level: the percentage of children younger than 12 months of age who received full immunizations increased by roughly 2 percentage points—from 77 percent in 2012 to 79 percent in 2017 and 81 percent in 2018. Although the overall immunization rates improved, the uptake of some vaccines remained flat, possibly because of a lack of follow-up of families on the recommended immunization calendar. Objective 3: Improve the Quality of Basic Preventive Health and Nutrition Services in Communities Covered under PAN Including Juntos 4.27 The achievement of this objective is rated as modest. Outputs and Intermediate Outcomes 4.28 The project equipped health centers, trained staff, and focused expenditures on results. The main outputs to support the achievement of this outcome were the strengthening of the government of Peru’s performance-based financing approach, the social monitoring strategy, the provision of medical equipment, enhanced information systems, and training to ensure quality in the delivery of the basic package of health and nutrition services. 4.29 The social monitoring strategy contributed to improvement in the quality of services provided. The project directly supported the articulation of the social monitoring strategy and consultants to implement it. The project used existing community meeting “infrastructure” or created new ones to implement social monitoring at the local level and provided technical assistance to monitor key social indicators, the Padron Nominal, and other social programs (such as the vaso de leche program in schools). IEG fieldwork witnessed the social monitoring conducted in meetings; using colors (green, yellow, and red) to indicate children’s nutritional status, deficiencies, and gaps; and planning activities to improve outcomes, including what could be done at the local level. This process fostered the identification of bottlenecks to decreasing malnutrition. The social monitoring work started in the poorest districts in the three targeted regions. With technical assistance, social monitoring was expanded to more districts. By June 2017, 156 districts in the first quintile had discussion forums about CRED coverage and vaccines and were registered on the MEF’s web page, more than doubling the formally revised target of 75. 21 4.30 Performance-based agreements at the municipal level have contributed to improved quality. By the end of 2018, districts and municipalities had strong commitments and agreements to achieve nutritional health outcomes. Improvement in service quality was supported through the provision of incentives and resources. Municipalities established clear expectations and agreements related to the results to be achieved with the Dirección Regional de Salud (Regional Health Directorate). The social assessment conducted in 2017 found that implementation of a social monitoring strategy was positively correlated with nutrition results at the local level (World Bank 2017). 4.31 Most health centers in the areas covered by the project were equipped to provide basic health and nutrition services in Juntos areas. Resources were allocated to equip and staff health facilities. To achieve improvements in health and nutrition in the target population, Dirección Regional de Salud allocated resources to ensure health facilities had the basic health equipment to administrate the CRED services and immunizations, achieving almost full coverage of health facilities by June 2017 (91 percent). Microhealth networks in Amazonas, Cajamarca, and Huánuco were verified annually to ensure they met the basic criteria for certification. The three regions were allocated budgetary resources to enable them to deliver services, and efforts were made to monitor their delivery. Overall, however, the target for this indicator was not achieved, with 44.1 percent of microhealth networks annually verified versus the revised target of 60 percent. Ongoing difficulties in reaching distant health centers may have explained this gap. 4.32 Training increased the awareness of health staff and key stakeholders of what is required to improve the quality of health and nutrition services. During the PPAR mission, it was observed that health staff were changing their approach to deliver higher-quality health and nutrition services. However, the PPAR interviews in the field suggested health personnel still require training and, in some cases, attitudes can deter mothers from visiting facilities (appendix H). 4.33 The project was also instrumental in supporting enhanced information systems. The project supported the development and scaling up of the use of data on key health and nutrition outcomes to manage interventions, supporting the alignment and coordination of data and information systems by different key institutions (Juntos, SIS, MINSA, MEF). The project also supported the preparation of the Health Centers Survey (Encuesta a Establecimientos de Salud; ENCRED) survey (carried out in 2013– 15). Did the Quality of Basic Preventive Health and Nutrition Services Improve? 4.34 It is difficult to determine whether the quality of basic preventive health and nutrition services improved. The project did not include specific outcome indicators to 22 measure improvement in the quality of services. The IEG mission collected some evidence from the field on provider behavior changes, which combined with adequate access to facilities and well-equipped facilities, enabled higher-quality service provision. However, the evidence is limited to IEG’s field visits. (For a detailed summary of data collected during IEG’s field visits see appendix H.) 4.35 Significant training contributed to improvements in quality. The central Ministry of Health prepared standards and new directives on the content and delivery of basic health and nutrition services and health staff received training on it. In the field, the IEG team observed that health personnel were implementing these new practices in their interactions with patients. During visits to health centers, it was observed in how the rooms were organized, in the information and explanations conveyed to mothers regarding growth charts, and in the interaction between mothers and health providers.15 4.36 Additionally, the PPAR mission observed a real desire for change in the nutrition space, even beyond Juntos. The PPAR mission observed16 that there was a change in mentality across all institutions and actors working on nutrition (not only the Juntos program), focusing on results, taking ownership for achieving results, and going out to the field, reaching out to the population, instead of waiting in the health centers for them to come. Health staff worked with mothers to accommodate them and help them to overcome the obstacles they face accessing and receiving health and nutrition services. 4.37 Mothers’ perceptions of quality were mixed. In interviews during the field visits, mothers were asked questions about the quality of service they received at health centers. Overall, beneficiaries believed the technical quality of health and nutrition services had improved over time. Beneficiaries also reported improvement through health staff visiting the communities. However, feedback was mixed on the attitudes of health staff toward beneficiaries. The mission observed that health staff tried to work with mothers and the constraints they face in getting to health centers. 4.38 Part of the improvement in the quality of the supply was related to quality of budgeting. The project supported institutional capacity building and governance. One of the key achievements during the 2011–17 period (as part of strengthened institutional capacity) was the improvement in the quality of budgeting, planning, and public spending at the regional and local levels. Why Did Malnutrition Decline? 4.39 Peru has made a significant improvement in reducing chronic malnutrition, although inequality persists. The share of children younger than five years who are stunted has fallen from nearly 25 percent to 10 percent of children (table 4.3). In July 2018, the share was 8.7 percent at the national level. There was progress in the reduction 23 of chronic malnutrition in children younger than five years in the project’s three regions. However, chronic malnutrition in rural areas was 19 percent, whereas only 5 percent in urban areas. Additionally, anemia levels in children younger than five years remained at 47 percent at the national level but reached 52 percent in rural areas and 45 percent in urban areas. Between 2013 and 2018, the share of children younger than five years with anemia fell in Cajamarca and Huánuco, but not in Amazonas. Finally, certain behaviors associated with better nutrition, such as adequate feeding practices, have not improved: the share of children younger than six months of age who are exclusively breastfed has decreased significantly in the three project regions between 2013 and 2018 (appendix E, table E.2). Table 4.3. Stunting Rates in Children Younger than Five Years in Peru, 2009–18 (percent) Indicator or Year 2009 2010 2011 2012 2013 2014 2015 2016 2018 Children with low 24 23 20 18 18 15 14 13 9 height-for-age Source: World Development Indicators and INEI 2018. 4.40 Although challenges remain, the project likely contributed to reductions in stunting. There is no rigorous evidence linking the Juntos CCT to reductions in chronic malnutrition. However, a review of experimental and quasi-experimental results of evaluations of the Juntos CCT program, along with some indication of a secular trend in reducing malnutrition, suggests that the government’s comprehensive nutrition strategy and the coordinated efforts to support the government, such as the Juntos Results for Nutrition Project, likely contributed to the success in the increasing rate of stunting reduction in the period of 2009–11 (Marini Rokx, and Gallagher 2017).17 The causal chain in CCT interventions are long and complex, and it is important to monitor implementation process along the causal chain to disentangle the project implementation and impacts from other factors that could influence outcomes that are beyond the project. Further, CCTs rely on changes in behavior for impact, which are rarely monitored. However, evidence from the field suggests changes in behaviors and attitudes surrounding nutrition. There is also evidence to suggest PAN has had an impact on stunting in Peru, but the impact for the three regions is not known (Marini, Rokx, and Gallagher 2017). 5. Efficiency 5.1 The efficiency of the project is considered substantial. 5.2 Impact evaluations show that CCTs demonstrate consistent short-term program effects. The evidence from systematic reviews of impact evaluations suggests 24 that CCTs produce positive short-term effects on investments in human capital, such as health. But the evidence of long-term impacts on health and nutritional status is mixed. In the short term, use of health services and growth monitoring generally increase. In the long term, the evidence suggests what works to improve the health and nutritional status of children in one context may not work in another (IEG 2010, 2011). 5.3 Evidence suggests the Juntos program produced short-term effects. An impact evaluation provided evidence that Juntos increased the use of health services and increased expenditure on more nutritious foods. Consistent with the international evidence at the time, the program showed no effect on chronic malnutrition (Perova and Vakis 2009, 2012). 5.4 Investments in child health and nutrition are high impact. A consensus has emerged from evidence across disciplines that investments in young children’s health and nutrition—children younger than three years, when neural connections are rapidly forming—are cost-effective, due in part to the high private and social rates of return (Young 1996, 2002; Young and Richardson 2007). Early nutritional deficiencies have been linked to delayed cognitive and noncognitive development, which has translated into delayed school progress (Georgieff 2007; Glewwe, Jacoby, and King 2001; Grantham- McGregor et al. 2007; Walker et al. 2007). Readiness to learn directly affects school attainment and earnings (Belfield et al. 2006; Gertler et al. 2013). 5.5 The project supported the efficient allocation of resources to areas where malnutrition rates were the highest, with spillover effects to all regions in Peru. The project directly supported defined activities critical to close gaps set by PAN in three of the poorest regions, the project’s effects extended beyond the three geographic areas, spreading to the rest of the country. This was notably the case for the achievements related to the Padron Nominal, the social monitoring mechanism and its arrangements, and the institutional strengthening of Juntos. 5.6 Improvement in the quality of public spending contributed to greater productive and allocative efficiency. The project contributed to improving the quality of information systems and their interoperability. This contributed to better evidence to support better planning and allocation of resources, better monitoring, and achievement of outcomes. Additionally, support to performance-based budgeting contributed to a more efficient allocation of resources at regional and local levels, as did the use of the Integrated Administrative Management System (Sistema Integrado de Gestión Administrativa; SIGA) and the Padron Nominal at the local level. Although the Padron Nominal provided the target number of children expected to be covered by a health center and the staff and inputs that would be required to assist them, SIGA (which includes a planning and execution module with logistics and inventory of goods and equipment) 25 calculated the required budget, helping identify costs that were not previously explicit (such as those for health personnel), and the budget was allocated based on the plans, targets, and outcomes to be achieved. This helped keep costs in check while improving allocative efficiency. By the end of the project, 100 percent of facilities had updated and completed asset information in SIGA (achieving its target). 5.7 Activities supported by the Juntos project are cost-effective, although a limitation to determining the project’s efficiency is the lack of cost data on the project itself. Activities supported by the Juntos project have been shown to be cost-effective. For example, World Bank research suggests cost-effectiveness of immunization programs given the increasing returns to scale in vaccination coverage (Brenzel 2005). Evidence on the economic returns to investments in nutrition-related interventions is growing (Alderman et al. 2016; Copenhagen Consensus Center 2015; Hoddinott et al. 2013) and new evidence is emerging for different scenarios and countries. For example, estimates of the economic returns to investments in stunting, anemia in women, exclusive breastfeeding, and treatment of severe wasting among young children (to attain the World Health Assembly’s global nutrition targets) place nutrition interventions among “the best value-for-money development actions” (Shekar et al. 2017, 6). However, although activities supported by the Juntos project are cost-effective, without project- or program-level cost data, it remains unclear how efficient this set of interventions really was, especially at a time when stunting rates had already begun to fall. 5.8 The implementation arrangements contributed to effective and efficient implementation of the project. The project used government systems. Several institutions were involved in implementation, but under MEF’s leadership, and with clearly defined outcomes, goals, and indicators, implementation was efficient. The implementation of the social monitoring mechanism, with MEF technical consultants in each department, contributed to coordination, supported the use of the management information systems (SIGA and the Integrated Financial Management System), the use of the Padron Nominal in these areas, and helped ensure collaboration among levels of government and institutions. Improved coordination among agencies brought about efficiency gains in the delivery of services. 5.9 Implementation took longer than expected because of a delay in effectiveness. Initial delays were due to required language changes in project documents to avoid ambiguity and ensure compliance with all government legal and regulatory frameworks. The project was the first of its kind in a social sector and adjustments needed to be made along the way. Because of these delays, the implementation period of the project was extended twice to an extra 15 months—for an implementation period of 26 approximately five years as initially envisioned. All funds were disbursed by project closure. 6. Ratings Outcome 6.1 Overall achievement of outcomes is rated satisfactory. The relevance of objectives is rated high. The objectives were responsive to country conditions at appraisal, completion, and currently. The systems put in place under the project were well suited to support the government’s continued efforts to combat stunting as it tackled anemia. The objective to increase demand for nutrition services by strengthening the operational effectiveness of Juntos is rated substantial. The objective to improve coverage of the basic preventive health and nutrition services in communities covered under PAN including Juntos is rated substantial. Only the objective to improve the quality of basic preventive health and nutrition services in communities covered under PAN including Juntos is rated modest. Efficiency is rated substantial. There is mounting evidence that suggests nutrition interventions are cost-effective, the project was well targeted, its results orientation improved the efficiency of public spending, and efficiency gains also resulted from using government systems (as a SWAp). Risk to Development Outcome 6.2 The project laid a strong foundation for sustainability. The main factors that support outcome sustainability are the (i) long-term political and social consensus on Peru’s national nutrition policy, (ii) achievements made in building institutional capacity, and (iii) strong information systems created and staff trained to use data to target and drive interventions. 6.3 Persistent challenges pose risks to outcomes. It is difficult to increase the supply of basic health and nutrition services in smaller and more remote health posts (which are harder to staff, have staff that are less frequently trained, and are inaccessible to mothers). The more remote locations are also poorer and have social norms that affect nutritional outcomes. These social and economic challenges will need attention to sustain achievements and continue to encourage mothers and family members to change their behaviors toward health, childcare, and feeding practices. The PPAR found clear demonstrated benefits to educating mothers. The most recent social assessment (2017) highlighted areas where changing behaviors of the indigenous populations will require more attention to specific barriers. Other challenges may explain the fall in demand for nutrition services and use of basic preventive health and nutrition services for children 27 younger than 36 months in Huánuco (from 69 percent to 67 percent between 2013 and 2018). 6.4 Institutional challenges and one design flaw in Juntos need to be addressed. Political and institutional changes may affect institutional outcomes. Continued government support is needed, including new financial support to maintain the Padron Nominal, the social monitoring strategy, and the ENCRED, which were instrumental for project success. Interviews during the PPAR mission found that social monitoring had continued after project closure and in one district—Namora (Cajamarca)—the current municipal government had prepared a handover memorandum to help a new team provide continuity of support. Enrollment in the Juntos program may falter if families are not provided with incentives to enroll additional children. Currently, the program does not provide additional money for additional children. Bank Performance 6.5 Overall Bank performance is moderately satisfactory. Quality at Entry 6.6 Quality at entry is rated moderately satisfactory. 6.7 The World Bank worked closely with the government to ensure strategic relevance of project design. The project’s development objectives, components, and activities were aligned with and supported national sector policies and priorities for reducing chronic malnutrition. Project preparation benefited from the World Bank’s long-term engagement with the government of Peru—consistent engagement over several different administrations—to reduce malnutrition. The long-standing dialogue and collaboration were critical to ensuring a project design that supported evidence- based solutions for improving childhood nutrition. The World Bank and the government opted to include components and activities in the project’s design that complemented PAN. 6.8 The project was part of a broader collaboration between the government, donors, NGOs, and international financial organizations in support of the government’s national nutrition policy. The World Bank supported the multisectoral approach to tackle malnutrition in Peru, which involved key ministries (MEF, MINSA, MIDIS), institutions (SIS, RENIEC), and local and regional governments in technical aspects of implementation. The implementation arrangements designed for the project supported this approach. 28 6.9 The project incorporated the main lessons from previous technical assistance and projects in Peru and the Latin American region. The World Bank’s nutrition nonlending technical assistance identified weaknesses in the design and implementation of the Juntos program (for example, lack of clarity surrounding the health coresponsibilities, inadequate verification of coresponsibilities, and inadequate institutional coordination). It piloted a restructured Juntos program in the Apurímac region. The design focused on activities to improve the supply and quality of health and nutrition services, based on previous findings that identified demand outpaced supply. Finally, the design focused on strengthening capacity of government agencies to foster sustainability, supporting activities that strengthened management capacity to implement the results-based budgets at the local level, as well as strengthening data and information systems, and supporting a social monitoring mechanism as a tool to empower and support the local level in the achievement of nutritional outcomes. 6.10 There were two main design shortcomings. First, the project lacked outcome indicators to measure quality improvements in the supply of basic health and nutrition services in the results framework. Although many outputs were related to the quality of supply, this shortcoming made it difficult to quantify actual quality improvements. Second, the project did not include MINSA as a key implementing agency, although its interventions were critical to the achievement of various outputs and critical for the quality dimension of outcomes. Quality of Supervision 6.11 The quality of the supervision is considered satisfactory. 6.12 The World Bank worked with the government to overcome the issues causing delays in project implementation. The World Bank was aware of the challenges inherent to an innovative operation that required interinstitutional coordination across several technical institutions. It carried out a thorough assessment of the potential risks for the operation and collaborated with and continuously supported, through the project, the implementing institutions. 6.13 The World Bank was proactive in tackling fiduciary and implementation challenges. It executed its fiduciary responsibilities satisfactorily and kept the project on track by proactively restructuring the project to address challenges. The World Bank also hired a specialized consultant to audit activities under components 2 and 3 to ensure the achievement of the project indicators. 6.14 The World Bank responded well to national requests to consider additional activities that supported PAN. This was for support for the pilot activities related to the 29 reduction of chronic anemia, a condition that had been identified as a critical challenge across Peru for children younger than five years. Monitoring and Evaluation 6.15 The monitoring and evaluation of the project is considered substantial. Design 6.16 The monitoring and evaluation arrangements under the project were part of the broader efforts by the government and MEF to strengthen several information systems. Monitoring was crucial to the project as disbursements were output based, requiring reliable, timely, and quality data to show achievement. The project included information systems strengthening within each of the project components. It included strengthening the Juntos information system and those within SIS. Upgrading the Padron Nominal was not originally part of project design. 6.17 The project’s results framework was adequate, although with some shortcomings. The indicators were designed to capture enrollment in Juntos and completion of basic preventive health and nutrition services; both of these capture use, which includes access and coverage (supply and demand). There was no outcome indicator defined to monitor the quality of health and nutrition services. The ENCRED survey monitored health facilities to verify their capacity and adequacy of equipment to provide health and nutrition services, but it was a poor indicator of the quality of services provided. However, a patient-centered indicator of the perception of quality could have been included. There were also no indicators to monitor changes in health and nutrition behaviors and practices in the home. Implementation 6.18 The project supported and implemented three tools to provide critical information for the project’s implementation and carried out monitoring activities. These tools were the Padron Nominal, the social monitoring strategy, and the ENCRED survey. The Padron Nominal became a key system to monitor and support the implementation and achievement of the project goals related to access to basic health services and nutrition. The Padron Nominal and the social monitoring strategy were also key tools to support MEF’s efforts to implement the performance-based budget at the local level. 6.19 Information on indigenous populations was not collected. All indicators were designed to be disaggregated by indigenous and nonindigenous beneficiaries, using mother tongue and rural location as identifiers for indigenous beneficiaries. These data were not collected (nor were baseline data collected). 30 Use 6.20 The information generated by the improved monitoring and evaluation systems guided decision making and activities. The data supported the implementation of activities that helped overcome bottlenecks in project design. The Padron Nominal was instrumental in resolving issues with identity papers, which were required for enrollment in both Juntos and SIS. Social monitoring, as implemented in Cajamarca and observed during the PPAR field visit, was an effective tool for creating knowledge and awareness surrounding nutrition. Lessons 6.21 Long-term engagement is critical to a deep understanding of complex malnutrition challenges. The World Bank’s long engagement in Peru was critical to the deep understanding of the country’s malnutrition challenges: long-term support, dialogue, and a collaborative working relationship enabled an innovative project design that required coordination among various implementing agencies at multiple levels of service delivery. 6.22 Attitudes toward nutrition need to change at all levels to make a difference. Peru’s commitment to reducing malnutrition started at the highest level of government. It required interinstitutional coordination from MEF, MINSA, MIDIS, SIS, RENIEC, and regional and local governments. Evidence from the field suggests attitudes are changing. For example, red, green, and yellow color-coded cards are being used to monitor and guide local-level goals and outcomes. And, in remote areas, health staff have begun to give priority to women and children who come to the health centers on market days. 6.23 It is important to address both the supply and demand for health and nutrition services. Addressing the many factors that influence nutrition is complex and challenging. Increased demand must be met with an increase in the availability of services. In the case of Peru, financial incentives and accountability measures were used to increase both the supply and demand for services through performance-based budgets, CCTs, a guaranteed package of services, health insurance, and improvements in the identification of eligible households. 6.24 Understanding the causes, consequences, and corrective actions required to reduce malnutrition can lead to changes in behavior. At approval, mothers had little knowledge of a child’s normal growth process and little awareness of the link between a child’s growth, development, and nutrition and health. Communication, CRED counseling, and community engagement helped increase awareness and understanding among mothers. Evidence from the field suggests the mothers’ knowledge and 31 awareness regarding nutrition and its effect on development played a role in increasing the demand for nutrition services. 6.25 Changes in beneficiaries’ behaviors cannot be assumed; they must be monitored. Given the complexity of the project and the causal chain, there were unexpected implementation challenges or barriers to changing behavior. For example, remoteness was an issue that hindered mothers’ ability to attend CRED sessions. Other examples include the following: not boiling water because traditional containers are now made differently and a bad taste leaches into the water, many families do not have electricity and refrigeration to keep healthy foods, and people in higher areas cannot realistically eat meat more than once per week. In such cases, monitoring behaviors is critical to address unexpected constraints to behavior change. 1 All figures are based on purchasing price parity, constant international dollars, 2011. 2Instituto Nacional de Estadística (National Institute for Statistics) —Encuesta Demográfica y de Salud Familiar (https://proyectos.inei.gob.pe/endes/resultados.asp). 3Around 2008–09, Peru adopted and implemented performance-based budgets for planning. These were geared toward articulating interventions and improving the efficiency and quality of public expenditure, with emphasis on results and accountability for social outcomes. The Ministry of Economy and Finance used performance-based budgeting to formalize the government’s commitment to results and accountability in key sectors, starting with nutrition. The performance-based budgeting aimed to break institutional budgetary inertia and assigning budgetary resources linked to performance to achieve better results in nutrition rather than on historical allocation. 4 Peruhas 25 regions and one province (Lima). The National Nutrition Program targeted the 14 poorest regions: Amazonas, Ancash, Apurímac, Ayacucho, Cajamarca, Cusco, Huancavelica, Huánuco, Junín, La Libertad, Loreto, Pasco, Piura, and Puno. The Europan Project was already operating in 54 districts in the three poorest regions: Apurímac, Ayacucho, and Huancavelica. The World Bank supported activities in Amazonas, Cajamarca, and Huánuco, the next three poorest regions. 5A sectorwide approach is a funding process that supports a single sector policy and expenditure program. For more information on sectorwide approaches, see Hutton and Tanner 2004; Vaillancourt 2009; and Peters, Paina, and Schleimann 2013. 6The restructuring focused on removing identified bottlenecks to Juntos program implementation including undercoverage of the target population, overestimation of compliance with coresponsibilities, cash transfers made without providing beneficiaries with information about compliance or coresponsibilities, limited supply capacity of health services, no managerial 32 monitoring system, inadequate institutional structure, no operational rules, and no recertification strategy (World Bank 2009, 5). 7 Use of authority figures is a well-known behavioral insight technique. 8The July 10, 2013, restructuring changed the wording of one of the two key performance indicators from “percent of Juntos children under 12 months that have received the complete Control of Growth and Development for Infants and Children in Peru (CRED) scheme according to their age in the areas of intervention of the Juntos Conditional Cash Transfer Program targeted by this operation” to “percent of children under 12 months that have received the complete CRED scheme, according to their age in the areas of intervention of Juntos targeted by this operation and in other similar regions.” In a subsequent restructuring, the original wording was maintained. 9The role of the National Directorate for Public Budget was to (i) monitor progress in the operation development objectives by the main implementing agencies (Juntos, SIS), especially with respect to the disbursement and results indicators defined for each component; (ii) maintain the dialogue and coordination among implementing agencies to identify and whenever possible remove implementation bottlenecks; (iii) ensure quality and timely preparation and updates to the Operations Manual; (iv) establish and chair the interinstitutional committee of the operation to monitor progress in implementation; and (v) consolidate progress and financial reports and check for compliance with World Bank fiduciary requirements as established in the loan agreement to initiate the processing of disbursements. 10The convenios allowed the Ministry of Economy and Finance to allocate additional resources to doctors or local health clinics based on results. They were a tool used to incentivize local level actors to provide more and better health and nutrition services (World Bank 2017, 6). 11The Padron Nominal is a registry or census of children including their name and the last names of the mother and father. 12As of 2019, data on each child in the Padron Nominal were crossed-checked and validated based on data from the National Civil Registry. This work is done by the health personnel at the health centers and the municipality. 13 In Peru, an identity document can be either the National Identity Card or a Single Identification Code. 14The immunization rate refers to the rate for “full immunizations” or todas las vacunas defined as vaccines for tuberculosis (BCG), rotavirus, pneumococcus, influenza, hepatitis B, and polio (World Bank 2011c, 60). 15 One of the practices observed in the field was of Juntos field officers and health staff reviewing a log of the types of food mothers had fed their children since their last CRED control visit. The information was recorded by mothers on a card given to them at the previous visit. The card provides a list of foods they can provide to their children, with space to mark what mothers agree to feed their children. Mothers take the card home and record actual feeding against planned 33 feeding (or other healthy activities). If targets are not met, barriers are identified, such as the constraints to serving meat multiple times per week. 16Although the Project Peformance Assessment Report mission was limited to two weeks in Peru and a field visit to three districts in Cajamarca, the main findings confirmed other evidence and observations in the study “Standing Tall” completed in 2017, which included extensive field findings in Huánuco (Marini, Rokx, and Gallagher 2017). 17At the time, demographic and health survey data were available up to 2012. There was a reduction in the percentage of children stunted pre- and postintroduction of the Juntos CCT program from 1992–2006 and 2008–12. See https://blogs.worldbank.org/impactevaluations/did- peru-s-cct-program-halve-its-stunting-rate. 34 Bibliography Belfield, Clive R., Milagros Nores, Steve Barnett, and Lawrence Schweinhart. 2006. “The High/Scope Perry Preschool Program: Cost–Benefit Analysis Using Data from the Age-40 Follow-up.” Journal of Human Resources 41 (1): 162–90. Brenzel, Logan. 2005. Note on Returns to Scale for Immunization Services. Washington, DC: World Bank. CARE (Cooperative for Assistance and Relief Everywhere). 2000. La Educación en Salud e Higiene en los Proyectos de Agua y Saneamiento. Lima, Peru: CARE. ENCRED (Encuesta a Establecimientos de Salud). n.d. “Quality and INEI.” ENCRED. Georgieff, Michael. 2007. “Nutrition and the Developing Brain: Nutrient Priorities and Measurement.”American Journal for Clinical Nutrition 85 (2): 614s–620s. Gertler, Paul J., Rodrigo Pinto, Arianna Zanolini, Christel Vermeerch, Susan Walker, Susan M. Chang, and Sally Grantham-McGregor. 2013. “Labor Market Returns to Early Childhood Stimulation: A 20-Year Follow-Up to the Jamaica Study.” Policy Research Working Paper 6529, World Bank, Washington, DC. Glewwe, Paul, Hanan G. Jacoby, and Elizabeth M. King. 2001. “Early Childhood Nutrition and Academic Achievement: A Longitudinal Analysis.” Journal of Public Economics 81 (3): 345– 68. Grantham-McGregor, S., Y.B. Cheung, S. Cueto, P. Glewwe, L. Richer, B. Strupp, and International Child Development Steering Group. 2007. “Developmental Potential in t he First 5 Years for Children in Developing Countries.” The Lancet 369 (9555): 60–70. Hanushek, Erik A. 2013. “Economic Growth in Developing Countries: The Role of Human Capital.” Economics of Education Review 37: 204–12. Hutton, Guy, and Marcel Tanner. 2004. “The Sector-Wide Approach: A Blessing for Public Health?” Bulletin of the World Health Organization 82 (12): 891–970. INEI (Instituto Nacional de Estadística e Informática). 2012. Informe Técnico Evolución de la Pobreza 2007–2011. Lima, Peru: INEI. ———. 2015b. Encuesta a Establecimientos de Salud en la Atención del Control de Crecimiento . Lima, Peru: INEI. INEI, MEF (Ministerio de Economía y Finanzas), and DNPP (La Dirección Nacional del Presupuesto Público). 2018. Indicadores de Resultados de las Proyecciones Presupuestarias del primer semestre de 2018. Encuesta Demográfica y de Salud Familiar. Resultados Preliminares al 50% de la muestra. Lima, Peru: INEI. 35 Jaramillo, M., and A. Sánchez. 2011. Impacto del Programa Juntos Sobre Nutrición Temprana. Lima, Peru: GRADE. Lescano, Andres. 2017. Evaluación de Impacto del FED sobre la Gestión del Sector Salud. Lima, Peru: Ministerio de Desarrollo e Inclusión Social. Marini, Alessandra, Claudia Rokx, and Paul Gallagher. 2017. Standing Tall: Peru’s Success in Overcoming its Stunting Crisis. Washington, DC: World Bank. MINSA (Ministerio de Salud, Peru). 2011. Guía técnica para la evaluación de la satisfacción del usuario externo en los establecimientos y servicios médicos de apoyo. Lima, Peru: MINSA. http://bvs.minsa.gob.pe/local/minsa/2252.pdf. ———. 2011. Norma Técnica de Salud Para El Control Del Crecimiento y Desarrollo de la Nina y el Nino Menor de Cinco Años. R. M.–Nº 990 – 2010/MINSA. Dirección General de Salud de las Personas. Lima, Peru: MINSA. Penny, Mary E., Hilary M. Creed-Kanashiro, Rebecca C. Robert, M. Rocio Narro, Laura E. Caulfield, and Robert E. Black. 2005. “Effectiveness of an Educational Intervention Delivered through the Health Services to Improve Nutrition in Young Children: A Cluster-Randomised Controlled Trial.” The Lancet 365: 1863–72. Perova, E., and Renos Vakis. 2009. Welfare Impacts of the “Juntos” Program in Peru: Evidence from a Non-Experimental Evaluation. Washington, DC: World Bank. Perova, E., and Renos Vakis. 2012, “5 Years in Juntos: New Evidence on the Program’s Short and Long-Term Impacts.” Economía 35 (69): 53–82. Peters, David H., Ligia Paina, and Finn Schleimann. 2013. “Sector -Wide Approaches (SWAps) in Health: What Have We Learned?” Health Policy and Planning 28 (8): 884–90. https://doi.org/10.1093/heapol/czs128. Pontificia Universidad Católica de Perú. 2017. Actualización del diagnóstico social y de las recomendaciones del marco estratégico de planeamiento indígena para el proyecto SWAP. Entregas 3 y 4. Lima: Pontificia Universidad Católica de Perú. Shekar, Meera, Jakub Kakietek, Julia Dayton Eberwein, and Dylan Walters. 2017. An Investment Framework for Nutrition: Reaching the Global Targets for Stunting, Anemia, Breastfeeding, and Wasting. Washington, DC: World Bank. Vaillancourt, Denise. 2009. “Do Health Sector-Wide Approaches Achieve Results? Emerging Evidence and Lessons from Six Countries.” Independent Evaluation Group Working Paper 2009/4, World Bank, Washington, DC. Walker, Susan, Theodore D. Wachs, Julie Meeks Gardner, Betsy Lozoff, Gail A. Wasserman, Ernesto Pollitt, Julie A. Carter, and the International Child Development Steering Group. 36 2007. “Child Development: Risk Factors for Adverse Outcomes in Developing Countries.” The Lancet 369 (9556): 145–57. Walters, Dylan, Julia Dayton Eberwein, Linda Brooke Schultz, Jakub Kakietek, Habibullah Ahmadzai, Piyali Mustaphi, Khwaja Mir Ahad Saeed, Mohammad Yonus Zawoli, and Meera Shekara. 2018. An Investment Framework for Nutrition in Afghanistan: Estimating the Costs, Impacts, and Cost-Effectiveness of Expanding High-Impact Nutrition Interventions to Reduce Stunting. Washington, DC: World Bank. World Bank. 2006. Peru—Country Partnership Strategy for FY07–FY11. Washington, DC: World Bank. ———. 2010. What Can We Learn from Nutrition Impact Evaluations? Lessons from a Review of Interventions to Reduce Child Malnutrition in Developing Countries. Independent Evaluation Group. Washington, DC: World Bank. ———. 2011a. Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets. Independent Evaluation Group. Washington, DC: World Bank. ———. 2011b. Peru—Loan 7961-PE Conformed Loan Agreement forthe Juntos Results for Nutrition Project. Washington, DC: World Bank. ———. 2011c. “Peru—Juntos Results for Nutrition Project.” Project Appraisal Document 51149 - PE, World Bank, Washington, DC. ———. 2012. Peru—Country Partnership Strategy for FY12–16. Washington, DC: World Bank. ———. 2017a. “Peru—Results in Nutrition for Juntos SWAp.” Implementation Completion and Results Report ICR00004175, World Bank, Washington, DC. ———. 2017b. Peru—Country Partnership Framework for FY17–FY21. Washington, DC: World Bank. ———. 2017c. Peru—Systematic Country Diagnostic. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/919181490109288624/pdf/Peru-SCD-final-3- 16-17-03162017.pdf. Young, Mary Eming. 1996. Early Child Development: Investing in the Future. Washington, DC: World Bank. Young, Mary Eming, ed. 2002. From Early Child Development to Human Development Investing in Our Children’s Future. Washington, DC: World Bank. Young, Mary Eming, and Linda M. Richardson, eds. 2007. Early Child Development from Measurement to Action: A Priority for Growth and Equity. Washington, DC: World Bank. 37 Appendix A. Basic Data Sheet Juntos Results for Nutrition (Loan No. 7961-PE; Project P117310) Table A.1. Key Project Data Actual or current Financing Appraisal estimate estimate Actual as percent of ($, millions) ($, millions) appraisal estimate Total project costs 54.00 25.00 46 Loan amount 25.00 25.00 100 Table A.2. Cumulative Estimated and Actual Disbursements Disbursements FY11 FY12 FY13 FY14 FY15 FY16 FY17 FY18 Appraisal estimate ($, millions) 6.60 13.40 18.80 22.20 25.00 25.00 25.00 25.00 Actual ($, millions) 0.00 0.00 0.00 7.28 11.72 18.02 23.26 25.00 Actual as percent of appraisal 0 0 0 33 47 72 93 100 Date of final disbursement October 30, 2017 Table A.3. Project Dates Event Original Actual Concept review 09/10/2009 09/10/2009 Negotiations 01/21/2011 01/21/2011 Board approval 03/08/2011 03/08/2011 Signing 07/05/2011 07/05/2011 Effectiveness 01/12/2012 01/12/2012 Closing date 03/31/2016 06/30/2017 38 Table A.4. Staff Time and Cost World Bank budget only Stage of project cycle Staff time Costa (no. weeks) ($, thousands) Lending FY10 0 369,519.55 FY11 0 157,878.74 FY12 0 −210.08 FY13 0 0.00 Total 0.00 527,188.21 Supervision or ICR FY11 0 43,310.76 FY12 0 183,350.57 FY13 0 178,174.48 FY14 45.860 146,799.22 FY15 46.766 191,468.11 FY16 31.700 146,894.31 FY17 25.485 242,313.19 FY17 1.375 6,268.51 Total 151.19 1,138,579.15 Note: ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. Table A.5. Task Team Members Name Titlea Lending Alessandra Marini Task Team Leader Nelson Gutierrez Human Development Operations Officer Fernando Lavadenz Senior Health Sector Specialist William Reuben Senior Social Development Consultant Omar Arias Human Development Sector Leader Maria Dolores Arribas-Baños Senior Country Officer Francisco Rodriguez Procurement Specialist Patricia McKenzie Manager, Financial Management Lourdes Linares Senior Financial Management Specialist Patricia Hoyes Pilar Larreamendy Senior Finance Officer Amparo Gordillo Senior Health Sector Specialist Isabella Micali Senior Counsel Renos Vakis Senior Economist Silvana Vargas Consultant 39 Name Titlea Rory Narvaez Consultant Mario Arrospide Consultant Carlos Ricse Consultant Suzana de Campos Abbott Lead Operations Officer/Consultant Elisa Seguin Junior Professional Associate Claudia Sanchez Junior Professional Associate Margaret Grosh Peer Reviewer Claudia Rokx Peer Reviewer Marcelo Bortman Peer Reviewer Patricia Orna Team Assistant Sara Burga Team Assistant Supervision or ICR Nelson Gutierrez Task Team Leader Selene del Rocio La Vera Procurement Specialist Juan Carlos Martell Rivera Procurement Specialist Patricia De la Fuente Hoyes Financial Management Specialist Claudia Rokx Team Member Patricia O. Orna Team Member William Reuben Social Safeguards Specialist Alessandra Marini Team Member Rory Narvaez Team Member Sara Burga Team Member Hugo Martin Brousset Chaman Team Member Javier Soriano Tabares de Nava Team Member Note: ICR = Implementation Completion and Results Report. a. At time of appraisal and closure, respectively. 40 Appendix B. Nutrition Strategies and Programs Box B.1. Government of Peru’s Strategies and Programs to Improve Nutrition as of 2011 At the time of project preparation, the government of Peru had taken major steps to tackle chronic malnutrition. • The Integral Health Insurance (Seguro Integral de Salud; SIS) was established in 2001 to address the fragmentation of service delivery and inequality in access to health services. In 2007, SIS was extended (Decreto Supremo 2004–07) to vulnerable populations living in poverty. • Juntos, or the National Program of Support to the Poorest, was established in 2005. The Juntos conditional cash transfer program was established in 2005. It played an important role in unveiling coverage gaps in the provision of health and nutrition services. The Juntos conditional cash transfer uses a financial incentive to promote regular health visits for pregnant women and children younger than five years and an 85 percent (or higher) school attendance rate for children between the ages of 6 and 14 years who have not completed elementary education. The program is premised on the theory that the accumulation of human capital can break the intergenerational cycle of poverty. In 2007, Juntos became part of the national nutrition strategy, Crecer (To Grow). Juntos conditions, payments, and processes have changed over time (in part due to World Bank technical assistance). Juntos and the Ministry of Health collaborated on a tool to improve parents’ understanding of how children grow. Juntos and SIS worked closely with the Ministry of Health to promote the importance of basic health and nutrition, to reinforce incentives to monitor the height and weight of children, and introduce community-level nutrition counseling programs. Since 2012, the Ministry of Social Development and Inclusion has managed the program in coordination with other relevant ministries, such as the Ministry of Health, Ministry of Education, and the Ministry of Women and Vulnerable Populations by linking Juntos beneficiaries to other public benefits. Juntos beneficiaries also have access to the Food Supplement Program for Higher-Risk Groups, among other programs. • The national nutrition strategy, Crecer, was launched in 2007. Crecer’s objective is to reduce poverty and child malnutrition. Crecer integrated nutrition strategies and provided a coordination platform for agencies addressing malnutrition under the leadership of the Interministerial Commission for Social Affairs. It strengthened interinstitutional management at the national, regional, and local levels to improve social inclusion and reduce chronic malnutrition in children through active participation of regional and local governments and beneficiary families. Crecer included other demand-side activities to help reduce childhood malnutrition through simple communication tools, video, television, and radio messages (for example, Mi futuro in mis primeros centímetros). In 2013, Crecer evolved into the National Strategy for Development and Social Inclusion—Incluir para Crecer (Include for Growth). Incluir para 41 Crecer is a multisectoral development strategy with nutrition components. The relevant nutrition objective is to reduce chronic child malnutrition through interventions for pregnant mothers and children from birth through age three years. Incluir para Crecer takes a lifecycle approach to social inclusion. It provides a coordination framework to organize interventions focusing on common objectives, is coordinated with relevant regional and local government initiatives, and is implemented through a results-based management system. • The National Nutrition Program (Programa Articulado Nutricional; PAN)—began in 2008. PAN is the Ministry of Health’s comprehensive national nutrition policy and one of the Ministry of Finance’s first strategic programs to introduce performance -based budgeting, an approach that aimed to improve budget preparation and execution in support of key national policies. PAN relies not only on performance-based budgets but also on (i) shared responsibility for reducing malnutrition in a multisectoral and interinstitutional framework and (ii) specialized training in malnutrition and management of nutrition interventions for public servants, including in the local offices of the Ministry of Finance. PAN’s objective is to reduce chronic malnutrition of infants through reduced incidence of low birth weight, better feeding and nutrition of children younger than 36 months, and reduced morbidity due to acute respiratory infections and acute diarrheal diseases for children younger than 25 months, concentrating on the poorest regions with the highest malnutrition rates. Specialized training is provided to public servants, including the Ministry of Finance, to understand the causes, consequences, and strategies to reduce malnutrition and to implement and manage nutrition interventions. PAN includes interventions to improve operational capacity to address diarrhea and respiratory infections in children; increase coverage of SIS through additional subsidies for uninsured children; improve incentives to join Juntos through increased cash transfers; implement information, education, and communication services to prevent malnutrition and promote nutrition; improve food preparation hygiene practices; create healthy municipalities, communities, schools, and families; and train mothers through Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo; CRED). CRED is a package of basic health and nutrition services provided to mothers and children through a set of activities (at set times) to adequately monitor the growth and development of children; detect early signs of risks, disorders, or disease, and diagnose and intervene to reduce control risks and address illnesses. CRED includes counseling, information and education, and demonstration sessions related to nutrition (for example, on exclusive breastfeeding, adequate child feeding practices, and child care) and the provision of safe drinking water (chlorination), safe management of solid waste, provision of nutritional supplements, particularly micronutrients (ferrous sulfate and vitamin A). Source: World Bank 2011b, https://extranet.who.int/nutrition/gina/en/policies/1520 and https://dds.cepal.org/bpsnc/programme?id=29. Note: The agencies and programs included in the Interministerial Commission for Social Affairs are Juntos, National Food Aid Program (Programa Nacional de Asistencia Alimentaria), National Program for the Mobilization for Literacy (Programa Nacional para la Movilización para la Alfabetización), National Watershed Management Program (Programa Nacional de Manejo de Cuencas Hidrográficas y Conservación de Suelos), National Water and Sanitation Program 42 (Programa Nacional de Agua y Saneamiento), National Registry of Identification and Civil Status (Registro Nacional de Identificación y Estado Civil), and Integral Health Insurance (Seguro Integral de Salud). 43 Appendix C. Project Components Table C.1. Juntos Results for Nutrition Project Detailed Component Description Component Activity Component 1. Strengthen and consolidate the • Activity 1.1 Community-Based Self-Enrollment design and operational capacity of the Juntos Programs. The Juntos program will organize conditional cash transfer program (appraisal community meetings to increase the coverage of $5.5 million; actual $5.5 million). Juntos beneficiaries and improve eligibility Readjust the Juntos program’s implementation verification and validation using registry forms. strategy to strengthen the communication • Activity 1.2. Management Information Systems strategy and information system to encourage (MIS). The capacity of the Juntos’ Area of families to enroll young children in Juntos. Technological Information will be expanded to Support incentives to enroll newborns and develop, enhance, and implement the necessary children younger than one year in the Juntos information systems to support the Juntos program’s program, including differentiating the amount of operational processes. A company specializing in the the cash transfer based on the number of eligible development and maintenance of MIS will be hired. children, harmonizing the Juntos program and • Activity 1.3. Communication Strategy. Support the health information systems, and helping mothers content development and mechanisms to understand the objectives of the Juntos program disseminate information to beneficiaries about the and the health and nutrition coresponsibilities. Juntos program and its restructuring, including Support incentives to improve the verification of detailed information on the program’s objectives and the health coresponsibilities for children younger coresponsibilities. Design, develop, and distribute than three years affiliated with the Juntos communication material using culturally appropriate program. formats. Develop an information, education, and communication strategy to encourage participation in the Juntos program and to explain how the Juntos program coresponsibilities help human capital development. Component 2. Improve the coverage and • Activity 2.1. Additional Funding for PAN. Provide quality of the provision of basic preventive additional support to PAN through the Health Facility health and nutrition services in Juntos areas Capitation Payment as a financial transfer, analogous (appraisal $5.5 million; actual $5.5 million). to a health insurance premium that finances a set of Provide additional funding for National Nutrition guaranteed and targeted results related to health and Program (Programa Articulado de Nutrición; nutrition, to be delivered to the population in the PAN), under the Health Facility Capitation Juntos areas in eligible health facilities. The World Payment, to ensure provision of a guaranteed Bank would transfer the Health Facility Capitation package of basic health and nutrition services to a Payment to the Ministry of Finance based on a fixed defined population and allow to beneficiaries of and a variable amount every year. the Juntos program to comply with their health • Activity 2.2. Guaranteed Package of Health and and nutrition coresponsibilities. Nutrition Services. Provide a guaranteed package of health and nutrition services for a defined population (adjusted for the probability of the demand for these services). The package of health and nutrition services refers to (i) complete Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo; CRED) checkups (CREDs for children younger than 12 months not attended yet in the Juntos areas, CREDs for children 13–24 months not attended yet in the Juntos areas, CREDs for children 15–36 months not attended yet in 44 the Juntos areas); (ii) complete immunization scheme (children younger than one year have a complete immunization scheme: including vaccines for tuberculosis (BCG), rotavirus, pneumococcus, influenza, hepatitis B, and polio; (iii) community- based interventions for behavioral change (sesiones demostrativas) (community-based behavioral change methodologies and tools on feeding practices, hand washing, and breastfeeding); and (iv) recurrent training of personnel on CRED program. • Activity 2.3. Eligible Health Facilities. Maintain a list of eligible health facilities (or a registry of enrolled facilities) serving beneficiaries of Juntos and located in one of the Juntos areas that regularly provides basic health and nutrition services. Component 3: Strengthen the government of • Activity 3.1. Strengthen Planning and Monitoring Peru’s capacity to influence nutritional Capacity of Health Facilities (appraisal $5.3 million). outcomes by improving budgetary planning Output payment disbursed against “the number of and monitoring of results for selected health micronetworks with accurate and timely activities of PAN (appraisal $14 million; actual information in [Sistema Integrado de Gestión $14 million). Administrativa (Integrated Administrative This component is expected to support the Management System)].” SIGA aims at improving the monitoring capacity of performance-based quality of the national public budget by enhancing budgeting for PAN. the consistency between planned and executed budgets (highly relevant for performance-based budgeting). • Activity 3.2. Social Monitoring (appraisal $4.4 million). Supports the establishment of a social monitoring mechanism run by local governments with the participation of local civil society organizations, media, users, and service providers. The social monitoring mechanism will oversee nutrition results and the quality of primary health and nutrition services at the district level. Output payment would disburse against “the number of municipalities with an installed social monitoring system.” Technical support includes (i) the provision of user-friendly information to local actors on coverage of key nutrition services and outcomes, and the capacity of health facilities to meet minimum conditions of effectiveness to provide the guaranteed package of services; and (ii) the use of this information by multistakeholder forums involving municipal authorities, service providers, and local civil society representatives organized by the local governments to oversee progress in the achievement of local nutrition goals. Monitoring results will be linked to the award of nonmonetary incentives. • Activity 3.3. Technical Verification (appraisal $2 million). Supports of activities aimed at (i) improving local planning capacity and increasing the transparency in the flow of resources from the 45 budget executing units to local health facilities and (ii) verifying the technical capacity of the health facilities to provide the guaranteed package of services, including the use of intercultural formats. Technical verification will be made by Instituto Nacional de Estadística (National Institute for Statistics) through an additional module of the Encuesta de Demográfica y de Salud Familiar (Household Health and Demographic Survey) continuous survey. The technical verification mechanism will be used to support the disbursement of the Health Facility Capitation Payment and aims to serve an equivalent function to technical or concurrent audits used in other performance-based World Bank–supported operations. This output payment would disburse against “the number of health micronetworks that have been technically verified.” • Activity 3.4. Early affiliation of newborns to SIS (appraisal $1.2 million). Supports an incentive to affiliate newborns to SIS. Specifically, it would finance “per affiliate output payments” for “affiliation of children younger than 12 months to SIS.” The information would be used by Juntos to verify health and nutrition coresponsibilities and act as an input to social monitoring and technical verification. • Activity 3.5. Verification of CRED Information (appraisal $1.1 million). Supports increasing the reliability of CRED data through (i) internal control as a managing tool and (ii) software acquisition. Source: World Bank 2011b, 53–64. Note: CRED = Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo); PAN = National Nutrition Program (Programa Articulado de Nutrición); SIGA = Integrated Administrative Management System (Sistema Integrado de Gestión Administrativa); SIS = Integral Health Insurance Program (Seguro Integral de Salud). 46 Appendix D. Implementation Details Table D.1. Details on the Restructurings of the Juntos Results for Nutrition Project Restructuring Date Change to Project July 10, 2011 • Strengthening the Padron Nominal was added to project design, ensuring children receive identity documentation and are eligible to enroll in Juntos and the Integral Health Insurance Program (Seguro Integral de Salud; SIS). • Clarifying language was added to facilitate disbursements, which had been delayed. • Adding, revising, and deleting intermediate outcome indicators to better align indicators with the project’s development objectives, and ensuring each indicator was monitorable with existing government data. • Amending the loan agreement to include a covenant related to a required technical audit of performance-based disbursements. February 26, 2014 • Revising indicators to align them with both the project’s and the Juntos program’s objectives. November 26, 2016 • Redesigning the technical audit to avoid further delays. The audit was split into a financial audit conducted by the government and a technical audit of the unit costing methodology of the performance-based disbursement mechanism, conducted by an independent consultant. January 11, 2016 • Extending the closing date by six months (to September 30, 2016) to allow for the completion of project activities, given the extensive delays in project effectiveness, and the inability to conduct the survey of health establishments (Encuesta de Productos Priorizados), which was required for the final disbursement. September 17, 2016 • Extending the closing date by nine months (to June 30, 2017) from the original closing date. This allowed expansion of the social monitoring strategy and an assessment of its contribution to increasing access to nutrition services, an update of the Indigenous People’s Policy Framework and a new social assessment, and reallocation of $1.7 million to the priorities of the new government, including Juntos, but also to a broader assessment of the determinants of anemia and stunting in the Amazonian regions. Source: World Bank 2017. 47 Table D.2. Implementing Agency by Component Component Implementing Agency Component 1. Strengthen and consolidate the • Ministry of Finance, Sectoral Projects design and operational capacity of the Juntos Coordinating Unit conditional cash transfer program • Ministry for Development and Social Inclusion (Juntos program) • Ministry of Finance, National Directorate for Public Budget Component 2. Improve the coverage and quality of • Ministry of Finance, National Directorate for the provision of basic preventive health and nutrition Public Budget services in Juntos areas • Regional Health Directorates • Health facilities Component 3: Strengthen the government’s capacity • Ministry of Finance, Sectoral Projects to influence nutritional outcomes by improving Coordinating Unit budgetary planning and monitoring of results for • Ministry of Finance, National Directorate for selected activities of the National Nutrition Program Public Budget • Integral Health Insurance • Regional governments Source: World Bank 2011, 2017. Note: At the time of appraisal, the Ministry of Health was not an implementing agency. Once implementation started, the ministry was responsible for implementation of activities related to the supply side of the basic health and nutrition activities. 48 Appendix E. Key Performance Indicators Table E.1. Juntos Results for Nutrition Outcome Indicators (percent) Indicator Name or Target Baseline Original Revised Actual PPAR Juntos children younger than 63.9 80.0 80.0 86.4 89.0 12 months who received the complete CRED scheme according to their agea Date 31/12/2011 31/12/2011 30/12/2016 30/06/2017 31/12/2018 Children younger than 36 Amazonas: 59.8; 73.0 73.0 70.0 Amazonas: 73.5; months who received the Cajamarca: 68.3; Cajamarca: 65.3; complete CRED scheme Huánuco: 69.3 Huánuco: 67.4 according to their age in the areas of intervention of the Juntos program targeted by this operationb Date 12/31/2013 31/12/2011 30/09/2016 30/06/2017 30/12/2018 Source: Instituto Nacional de Estadística (National Institute for Statistics)—Encuesta Demográfica y de Salud Familiar (2012–17), https://proyectos.inei.gob.pe/endes/resultados.asp; Ministry for Development and Social Inclusion 2018 data. Note: a. The areas of intervention of the Juntos program targeted by this operation are Amazonas, Cajamarca, and Huánuco. The indicator focuses on the first period of a child’s life, when 11 of the 21 CRED visits take place. The intent of the indicator was to signal changes in the proportion of young children affiliated with, or enrolled in, the Juntos program. The program was having difficulties in getting families to enroll their youngest children. When the project was restructured in July 2013, the indicator was revised to include Amazonas, Cajamarca, and Huánuco, and all other Juntos areas of intervention in similar regions of the country. However, in the February 2014 restructuring the original definition of the indicator was reinstated. The value at the time of the PPAR for the three regions, including both Juntos and non-Juntos children was: Amazonas (66.4 percent); Cajamarca (72.0 percent); and Huánuco (60.0 percent). b. The indicator was designed to capture improvement in the supply of nutrition services in the project’s three regions, Amazonas, Cajamarca, and Huánuco. It covers maintenance visits under the CRED program. At the time of the Project Performance Assessment Report, maintenance visits consisted of six visits for children aged 12 to 23 months and four visits for children aged 24 to 35 months. These data differ from those reported in the ICR, which increased from a baseline of 67.2 percent (as of March 30, 2012) to 70 percent, falling short of the targeted 73 percent. 49 Table E.2. Juntos Results for Nutrition Intermediate Outcome Indicators Indicator Name or Target Baseline Original Revised Actual PPAR Component 1: Support the strengthening and consolidation of the design and operational capacity of the Juntos conditional cash transfer program for families with children younger than 36 months. Juntos beneficiary children younger 18.0 — 67.0 74.1 National: 95.0; than 24 months registered in the Amazonas: Juntos information system with a 84.0; birth certificate, identity document, Cajamarca: or SIS affiliation (percent) 98.0; Huánuco: 98.0 Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Children born each year in Juntos 9.6 — 40.0 39.9 National: 46.0 households and registered in the Amazonas: Juntos information system before 30 28.0 days (percent) Cajamarca: 41.0 Huánuco: 64.0 Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Component 2: Improve coverage and quality of the provision of basic preventive health and nutrition services in the Juntos areas. Children younger than 36 months 43.0 — 65.0 88.5 — affiliated to SIS and with a complete and timely scheme in the areas of intervention of Juntos in Amazonas, Cajamarca, and Huánuco (percent) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Mothers in Juntos families, with 0.0 — 50.0 28.8 — children younger than 24 months, who have attended at least one demonstration session in Amazonas, Cajamarca, and Huánucoa (percent) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Health facilities with basic equipment 0.0 — 40.0 91.0 91.0 to give CRED services and vaccines in Juntos districts (percent) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Component 3: Strengthen the government of Peru’s capacity to influence nutritional outcomes by improving budgetary planning and monitoring of results. Health facilities with updated 98.4 — 100.0 100.0 100.0 information in the SIGA (percent) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Districts in the first quintile where 0 0 75 156 156 there has been discussion about CRED coverage and vaccines (social monitoring) and are registered on the web page of the Ministry of Economy and Financeb (no.) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 50 Indicator Name or Target Baseline Original Revised Actual PPAR Health micronetworks annually 0.0 0.0 60.0 44.1 Amazonas: verified that they meet the 471 health certification criteria in Amazonas, facilities (of Cajamarca, and Huánuco (percent)c 505) Cajamarca: 841 (of 938) Huánuco: (of 343) Date 31/12/2011 — 30/06/2017 30/06/2017 30/12/2018 Newborns registered in SIS before 30 11,759 0 15,000 19,878 Amazonas: days in Juntos districts in the areas of 3,887; intervention of the operationd (no.) Cajamarca: 13,937; Huánuco: 8,483 Date 31/12/2011 — 30/06/2017 30/06/2017 30/01/2018 Sources. Integral Health Insurance Program data; Juntos Administrative data; Ministry of Economy and Finance data; Instituto Nacional de Estadística e Informática (National Institute for Statistics)—Encuesta Demográfica y de Salud Familiar data (https://proyectos.inei.gob.pe/endes/resultados.asp). Note: CRED = Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo); PPAR = Project Performance Assessment Report; SIGA = Integrated Administrative Management System (Sistema Integrado de Gestión Administrativa); SIS = Integral Health Insurance (Program Seguro Integral de Salud). a. This indicator does not reflect the actual number of attendees because there is not a protocol issued by the Ministry of Health or Ministry for Development and Social Inclusion to establish a registration process. Demonstration sessions are part of the counseling process for families. During the PPAR mission, all interviewees said they participated in demonstration sessions carried out by Juntos field staff on food preparation and other content. b. The social monitoring mechanism as supported by the project was not formally continued by the Ministry of Economy and Finance after completion. There was not an increase in the number of districts in the first quintile using the social monitoring mechanism as described in the project. Nevertheless, during the mission, it was observed that there was continuity in using the arrangements supported by the social monitoring mechanism after project completion. c. This indicator is not monitored by the National Institute for Statistics Encuesta Demográfica y de Salud Familiar since the last survey in 2016. The Ministry of Health monitors this information. The information provided at the Project Performance Assessment Report stage are the number of health facilities (hospitals, health centers, and health posts) in each of the targeted departments that are well-equipped to provide CRED services, immunizations, nutrition supplements, and prenatal controls. d. The figure covers Amazonas, Cajamarca, and Huánuco as a whole. Juntos works in 1,324 districts nationwide out of a total 1,860 districts. The figures are for early registration of children with identification (afiliación temprana). Table E.3. Challenges in Malnutrition in Amazonas, Cajamarca, and Huánuco (percent) Amazonas Cajamarca Huánuco Indicator 2013 2018 2013 2018 2013 2018 Children younger than five 47 51 51 38 51 45 years with anemia Children younger than six 87 76 86 68 97 75 months exclusively breastfed Children with low weight at 10 8 10 9 10 9 birth Source: Instituto Nacional de Estadística e Informática. Indicadores de Resultados de los Programas presupuestarios del Primer Semestre de 2018. Resultados Preliminares. 51 Appendix F. Main Characteristics of Juntos Table F.1 summarizes the main features of the Juntos conditional cash transfer program at the time of appraisal and changes to the program reflected at project completion. Table F.1. Characteristics of the Juntos Conditional Cash Transfer Program, 2005–18 Juntos Characteristics 2005 2009 2018 Juntos established Juntos first launched Target population Poor rural households with Poor rural households children younger than 14 with children up to 19 years years old Targeting method Geographic, proxy means, and community validation; being considered poor under the Sistema de Focalización de Hogares (SISFOH) Coverage 450,110 households in 638 730,206 households, districts. with 41,674 children younger than three years Benefit structure S/. 100 disbursed once S/. 100 disbursed once per S/. 100 ($33) disbursed per month month bimonthly Payee Mother Payment method Through bank accounts at the Banco de la Nación and associated debit cards; the most remote communities have access to mobile banks drive to receive their benefits Health coresponsibilities • Regular health visits for pregnant women and children younger than five years • One prenatal visit for pregnant mothers • One postnatal visit for new mothers • A full inoculation schedule, vitamin A, iron and folic acid supplements, and attendance at nutritional, reproductive health, and food preparation discussions for pregnant women and mothers • A full inoculation schedule, iron supplement, growth and development monitoring, and 52 deworming for children up to five years old • Chlorine tablets for beneficiary households Education coresponsibilities • School attendance of at least 85 percent for children aged between 6 and 14 years who have not completed elementary education Other coresponsibilities Identification access for women beneficiaries at the start and for all beneficiaries (children) older than one year Compliance verification Every three months S/. 120 million S/. 607 million S/. 937 million Source: World Bank 2011c, 43; https://www.gob.pe/588-programa-juntos. Note: Juntos is a conditional cash transfer program that provides monetary incentives to poor and very poor households under the Ministry for Development and Social Inclusion. Households present a 40 percent poverty index under the SISFOH and have children younger than 19 years. Criteria to be eligible include living for at least six months in the district of intervention, having at least one beneficiary among its members (pregnant woman, child, or youth younger than 19 years), and considered poor under the targeting system SISFOH. S/. = nuevo sol; SISFOH = Sistema de Focalización de Hogares (Household Targeting System). 53 Appendix G. Performance-Based Budgeting and the Articulated Nutrition Program The National Nutrition Budget Program The performance-based budget, which started in 2007/2008, aims to improve the budget elaboration process and execution. It aims at breaking inertia in the budget preparation process by setting goals to allow evaluating the efficiency of the expenditure through an integrated vision of planning and budgeting. Sectors must first set their goals, then build a logical model and define the key actions to be taken that will move their targeted indicators toward the set goals. A set of products is prioritized, and the cost of the inputs needed to produce them is calculated to inform budget allocations. The Integrated Administrative Management System planning module is used for this exercise. Each strategic program in the budget involves several institutions and may involve more than one sector. Hence, the responsibility to achieve the targets is shared by every entity involved. The outcomes (resultados) are then linked to the principal products that government programs provide, which in turn are assigned specific funds allocated by agency. The individual budget units track the outputs in regular reports. The Integrated Administrative Management System execution module is used to monitor progress and provide monthly data on physical and financial indicators for all the outputs in the strategic programs. The main objective of the National Nutrition Program (Programa Articulado de Nutrición; PAN) is to reduce infant chronic malnutrition. The program is expected to lead to a reduction in chronic malnutrition through intermediate outcomes of lower incidence of low birth weight, better feeding and nutrition of children younger than 36 months, and reduced morbidity due to acute respiratory infection and acute diarrheal disease for children younger than 25 months (World Bank 2011c, 42–43). The following institutions receive funds under PAN: • The Ministry of Health to purchase and cover the provision of vaccines and to carry out activities of monitoring, evaluation and control, surveillance, research of technologies for nutrition, qualification of healthy municipalities and families, accreditation of healthy schools, and development of norms and technical guides for nutrition. • The Ministry of Social Inclusion and Development for the Juntos program. At the time of appraisal and before the creation of the Ministry for Development and Social Inclusion, the Presidency of Ministers Council received PAN 54 resources for Juntos. The resources covered assistance to beneficiary households and the implementation of the National Strategy (Crecer) that established an articulated intervention of the national, regional, and local government against malnutrition. • The National Institute of Health to carry out surveillance, research, and technology in nutrition, as well as implementing the monitoring, evaluation, and quality control of the food. The Centro Nacional de Alimentación y Nutrición and the Centro Nacional de Salud Pública, part of the National Institute of Health, participate in executing these activities. • The Integral Health Insurance to cover the provision of complete Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo; CRED) services and supplements of vitamin A and iron, as well as prevention and treatment of acute respiratory infection, acute diarrheal disease, and intestinal parasites. • Regional governments to complement the intervention of the Ministry of Health through the programs of prevention (immunization and training among others) and to finance the management of the new performance-based budgeting to be carried out by the regions. Table G.1. Guaranteed Health and Nutrition Package Output Activities CREDs for Juntos children CREDs for children younger than 12 months not attended yet in the Juntos younger than 12 months areas not attended CREDs for Juntos children CREDs for children aged 13–24 months not attended yet in the Juntos areas aged 13–24 months not attended CREDs for Juntos children CREDs for children aged 15–36 months not attended yet in the Juntos areas aged 25–36 months not attended Training on CRED Recurrent training of personnel on CRED Community-based Community-based behavioral change methodologies and tools on feeding behavioral change practices, hand washing, and breastfeeding activities Children younger than Children younger than one year who have a complete immunization scheme: one year with complete including vaccines for tuberculosis (BCG), rotavirus, pneumococcus, influenza, immunization scheme hepatitis B, and polio Source: World Bank 2001b, 59–60. Note: CRED = Control of Growth and Development for Infants and Children in Peru (Control de Crecimiento y Desarrollo). 55 Appendix H. Summary of Field Visits The PPAR mission visited three districts within 1.5 hours from Cajamarca, the location of the Juntos technical provincial unit. In the district of Namora, the mission visited the health center and interviewed the director, a doctor, several nurses, and health staff. They met with mothers and family members who were at the health center for the growth control of their children. In the district, the mission visited the home of a mother leader for Juntos in her community. The mission visited three households and had conversations with mothers and their families. In Namora, the mission met with a team in the municipality, consisting of the specialist in the Padron Nominal, the director, and the deputy director for the municipal department for social development. In the district of La Encañada, the mission met with a group of 25 mothers, all beneficiaries of Juntos, in the offices of the Local Registration Office. The mission visited a family at their house and interviewed the mother. The mission visited the health center in La Encañada; training of health staff on guidelines and norms was ongoing during the visit. The mission interviewed health staff on the content of the training and visited the facilities. In the district of Baños del Inca, the mission visited the community of Shaulo Chico, meeting with a group of eight mothers, all beneficiaries of Juntos, including the mother leader for Juntos in that community. The mission visited the health center in Baños del Inca and held a meeting with the director. In the district of Jesús, the mission visited a health center and met with health staff, toured the facilities, and spoke with mothers (and a father). The mission visited the home of a mother leader to discuss her role and experience in her community and another two Juntos households. In Cajamarca, the mission held a meeting with a group of approximately 35 field staff (gestores) in the Juntos technical unit (out of 166). It met with three experts of the social monitoring mechanism who had worked under the project and knew the project in depth. A summary of main findings by stakeholder is in box H.1. 56 Box H.1. Stakeholders’ Views from the Field Summary of mission meetings with Juntos field staff: • Continuity in the job. They had been in Juntos at least for three years. Some had been in Juntos for 12 years. They rotate every 1.5 years within the department, but within different districts. • Challenges for Juntos staff. Field staff reported significant workloads. A typical staff member might have a caseload of 600–700 households. Staff rely on mother leaders to pass along information and organize activities and monitor changes in their communities. • Access to health services has improved. The interviewees confirmed that changes have taken place in recent years: health professionals reach out more and visit communities. • Commitment to improving nutrition. Respondents reported they had observed changes in their districts related to nutrition. For example, new strategic alliances have been established at the local level. • Juntos coresponsibilities more likely to be met. Staff have observed improvements in the number of families fulfilling Juntos coresponsibilities in education (decreased absenteeism and dropouts in youth aged 15–18 years) and health. Mothers fulfilled Juntos health coresponsibilities, reporting that these were not considered an obligation despite the effort required to get to health centers, with distance and the cost of transportation as potential challenges. Mothers felt supported by Juntos field staff and staff in the health centers. • Behavior is changing. Field staff report that behavior change takes time, but changes are happening. For example, they observe healthy practices in the households during unannounced visits, husbands now attend demonstration sessions, mothers are actively engaged in the demonstration sessions—asking questions and speaking up in the sessions, women understand and appreciate the importance of having identification cards for their children, and young first-time pregnant women are already attending health centers for pregnancy controls. However, more time is needed for counseling support to mothers and families to promote these changes in behavior. • Juntos has empowered women. Women’s dignity has increased. Empowerment has increased through financial literacy, their appreciation for education (reading and writing), and control over the management of their household economy. Maternal mortality has decreased with longer spacing between pregnancies. Respondents agreed that Juntos contributed to greater gender equity in the rural and poorer areas of intervention. 57 Box H.1, continued Summary of mission meetings with Juntos beneficiary mothers: • Comply with Juntos obligations. Mothers affirm that they regularly comply with the conditions of Juntos. They were all aware of the nutritional status of their children and of their anemia classification after their last CRED visit. • Juntos cash transfers are allocated to food or education. The mothers reported using their Juntos monetary incentive on food or school items. • Health practices are followed. In general, mothers claimed to follow health practices when preparing food or feeding their children. They discussed in detail what they prepare to feed their children and the frequency of the feedings. There are times when practices are not followed, but these barriers could be addressed. For example, in Shaulo Chico, mothers acknowledged they are not boiling water because they no longer have traditional containers to keep water in, and the new containers (made from old tires) give the water a bad taste. • Mother leaders are effective. Women appreciate having control of the Juntos cash transfer and understand the importance of keeping their identification cards secure to obtain the transfer. They also appreciate their role in managing the Juntos transfer. Mother leaders support all mothers in their communities in these processes. • Mothers are engaged and active in their communities. Mothers regularly attend demonstration and information sessions and check on their neighbors. The women consider the sessions informative and helpful. If mothers needed help with rebooking appointments for their growth control visits, or if they missed them, both Juntos field staff and mother leaders assist and support them. • Mothers face challenges in complying with all health practices. Women face hard conditions in their communities. They understand the need for healthy practices but complying with all of them is difficult. For example, boiling water, planting vegetables or having a garden due to lack of water, and feeding their children certain foods that are too expensive (due to lack of jobs and low incomes). • Cultural traditions reinforce some health practices. It is traditional to stay home the first two weeks after a baby is born, which reinforces practices for caring for newborn babies. • There were some reports that not all health staff were considerate or eager to help mothers. In La Encañada, women complained that health personnel were inflexible with the appointments, health centers had long lines, and nurses’ attitudes (not doctors) toward the mothers were disrespectful. • Juntos helps improve health, nutrition, and education status in these areas. Juntos brings government services to the local level, whereas beneficiaries perceive the ministries of health and education as remote institutions. 58 Box H.1, continued Summary of mission meetings with health professionals: • Health professionals’ attitudes toward beneficiaries have improved. The mission team witnessed the visible trust building in the relationship between health center staff and mothers. Beneficiaries are looked at in a holistic manner, as part of their communities. • Health professionals make house calls. Health professionals now visit beneficiary communities, a change in practice from the past. • Crowded health centers. On market days, health posts in remote areas are overwhelmed by beneficiaries. Respondents reported prioritizing their time with those who live far away. • Health staff have received training. Staff report receiving substantial training on ways to provide health and nutrition services to mothers and children and how to improve service provision. However, continued training of health staff is required to change their attitude toward patients and to be more aware of their circumstances and more responsive to their needs. • Administrative tasks have increased. Interviewees reported more administrative work at the health centers (with an electronic health history). • Social programs, including Juntos, coordinate with health programs. Health workers report close coordination with social protection programs, which has improved overall support to beneficiaries through information sharing, bridging gaps (for example, arranging appointments, and getting mothers to deliver babies in health centers), and playing a role in the overall fulfillment of health commitments in their districts. Juntos is seen as key in identifying pregnant women and supporting the health centers in reaching out to them. Summary of mission meeting with the municipal team leading social development in Namora: • Social monitoring ended with the project. The social monitoring activity associated with the project started in 2014 and concluded when the project closed in 2017. • Social monitoring and other project activities have been institutionalized and sustained. Namora has continued holding monthly meetings around the Mesa de Concertación. All actors who may influence nutrition join the monthly meetings (Dirección Regional de Salud (Regional Health Directorate), Juntos, Cunamas, País, subprefectura, nongovernmental organizations, and others). Namora has also institutionalized social monitoring. It has continued updating the Padron Nominal in the last year, which suggests the sustainability of the project’s activity. • Institutional knowledge. The outgoing municipal team prepared a dossier for the new municipal team, who are due to take office in January 2019 (after the 2018 municipal elections). The dossier included information on ongoing efforts to reduce malnutrition in Namora, achievements, challenges, and proposed actions for the future. 59 Box H.1, continued • Communication clarity. The use of the PAN indicators with red, green, and yellow color coding to guide goals and outcomes and the incentives for improving management and performance in municipalities (the Plan de Incentivos a la Mejora de la Gestión Municipal) helped motivate and articulate actors at the local level. Source: Independent Evaluation Group. 60 Appendix I. Relevant Nutrition and Health Support to Peru, 2009–17 World Bank Projects and Advisory Services and Analytics, 2009–17 Juntos for Nutrition Non-Lending Technical Assistance (P111717). Provided an analysis of the government’s Juntos conditional cash transfer (CCT) program with guidance on how to improve the program to strengthen its nutritional impact. The team provided technical assistance to improve its Juntos’ design and implementation and to help identify relevant lessons from CCT programs and health nutrition interventions in other countries to ensure that Juntos was designed according to international best practice (at that time), to be as effective and efficient as possible, particularly with respect to improving chronic malnutrition as part of the national nutrition strategy Crecer (To Grow). Result and Accountability Development Policy Loans I–III (P101086, P101177, P116264). This series supported strengthening the results frameworks and accountability relationships in education, health, and nutrition, by supporting policies expected to (i) improve parental understanding of expected outcomes in education, health, and nutrition; (ii) improve outcomes in second-grade literacy (especially in rural schools); (iii) reduce maternal and neonatal mortality; and (iv) improve nutrition outcomes. Health Reform, Second Adaptable Program Loan (P095563). Support under the project complemented the support provided under the Juntos Results for Nutrition Project. It supported the government of Peru’s efforts to increase the quality of health services provided by improving family care practices for women (during pregnancy, delivery, and breastfeeding), and children younger than three years; strengthening health service networks by capacity building to solve obstetric, neonatal, and infant emergencies, and providing comprehensive health services; and supporting the Ministry of Health’s governance of the regulation, quality, efficiency, and equity of service delivery under the new decentralized health delivery model for maternal and child health care. The project’s goals were to reduce maternal and infant mortality rates in the nine poorest regions of the country (Amazonas, Huánuco, Huancavelica, Ayacucho, Apurímac, Cusco, Cajamarca, Ucayali, and Puno). Programmatic Fiscal Management and Competitiveness Development Policy Loans II- IV (P10510, P115120, P106724, P116214). The World Bank began supporting the government’s program in 2007, overlapping with the Juntos Results for Nutrition Project for the period of 2009–11. The program’s goals were to increase the rate of economic growth and do so in a sustainable, inclusive manner. The government’s development 61 addressed issues of inefficient public institutions and high poverty rates, to ensure the benefits of its plan reached the poor. To improve the quality and impact of public spending, the government aimed to build on the prior introduction of a regulatory framework for performance-based budgeting to ensure that an increasing percentage of the government budget was linked to performance indicators, World Bank–supported obstacles to private sector activities, and high poverty rates. These projects built on the existing regulatory framework for performance-based budgeting and aimed to increase the share of performance-based budgeting in the government’s total budget. The National Nutrition Program (Programa Articulado de Nutrición; PAN) was one of the first performance-based budgeted programs with corresponding goals to improve the quality and impact of public spending on nutrition services. Strengthening the capacity of the Ministry for Development and Social Inclusion (Ministerio de Desarrollo e Inclusión Social; MIDIS) to provide integrated services at the subnational-level Non-Lending Technical Assistance (P152321). The project was part of a multisectoral package of support to MIDIS and was implemented in close coordination with the other projects in the package and existing programs such as the Juntos Results for Nutrition Project and the Social Inclusion Technical Assistance Lending Project (P131029), Poverty Non-Lending Technical Assistance (P151096), and Second Social Inclusion Development Policy Loan (P147216). The project’s objective was to improve the quality of MIDIS’s program delivery to achieve synergies in coordinating its social inclusion policies and programs and implementation of Incluir para Crecer (Include for Growth) through child nutrition, early childhood development, childhood and adolescence protection, economic inclusion, and elderly protection. The project supported the design and organizational model for integrated service provision at the subnational level, development of guidelines for the subnational implementation of the model, and design of a communication strategy to support rollout of the model. Peru Nutrition Engagement Notes (P156602). The objective of these notes is to support sustained dialogue on nutrition with a new government thorough examination of the evolution of the fight against malnutrition (from conceptualizing malnutrition to tackling it), which led to a significant reduction in malnutrition rates, identified remaining challenges, and proposed possible ways to address the remaining and new challenges. It will also investigate the impact intermediate cities might have on promoting healthy diets and long-term improvements in nutrition, such as anemia, and preventing the emergence of noncommunicable diseases related to poor diets and high chronic malnutrition rates in childhood. Peru Nutrition (P162483). Analytical work designed to describe Peru’s success in reducing chronic malnutrition, determine which policies and interventions helped reduce stunting, investigate whether the policies and interventions worked, and 62 produce a set of lessons for other countries to learn from (see Standing Tall: Peru’s Success in Overcoming its Stunting Crisis). More effective and efficient social inclusion services (P167417). One of the challenges for implementation of the national development and social inclusion policy in Peru is the alignment of different social interventions across sectors (intersectoral) and between government units and territorial levels (intergovernmental) to provide services to the same target population. MIDIS is the governing body of the Social Inclusion policy, playing a steering role in coordinating actions and developing policy instruments for other sectors (mainly health, education, work, housing, women, and vulnerable populations). In addition, MIDIS implements the five social programs with the greatest national coverage for different target groups. These interventions are the early childhood development program, Cuna Más; Juntos CCT; the school feeding program, Qali Warma; the social pension, Pension 65; and the Social Development and Compensation Fund, Fondo De Cooperation Para Del Desarrollo Social—all of which are rolled out in large portions of the national territory. Donor Support, 2009–17 Europan Project. The European Commission’s Europan project supported the development of the Nutritional Articulated Program and performance-based budgeting. The project sought to improve regional-level coordination of the Ministry of Health and Juntos, Integral Health Insurance Program (Seguro Integral de Salud; SIS), and the MIDIS programs (existing at the time Europan started). It covered 54 districts in the Regions of Ayacucho, Apurímac, and Huancavelica, and its main objectives were to reduce chronic malnutrition and mother and child mortality. The project introduced and piloted budget support agreements (convenio de apoyo presupuestal) between local governments and the Ministry of Economy and Finance. That project supported PAN and tracked progress on selected indicators, such as increased coverage of health checkups (Control de Crecimiento y Desarrollo; CRED) and immunizations in Peru’s three poorest regions Supporting the Strengthening of the Crecer Strategy and the Juntos Program. The Inter-American Development Bank’s project focused on reforming Juntos regulations and operating processes, increasing the impact of its transfers, and improving the verification of its coresponsibilities. It supported improvements in access to basic health services among the poor through reforms to SIS that would standardize processes, improve service procurement, increase levels of enrollment, and facilitate payment of claims to providers. The Inter-American Development Bank and World Bank coordinated their support to Juntos and the government’s strategy to reduce malnutrition. 63 Drinking Water and Sanitary Programme. A project supported by KfW (formerly Kreditanstalt für Wiederaufbau Bankengruppe) with the objective to guarantee improved quality in water and sanitation services by working with the water supply and sanitation companies, and the improvement of the institutional framework for achieving sustainable drinking water supply and high-quality wastewater disposal. Human Safety for the Development of Peruvian Women and Children. The United Nations Children’s Fund Pan American Health Organization program focused on providing quality live conditions for women and children, including improved nutrition. Cooperative for Assistance and Relief Everywhere. The Cooperative for Assistance and Relief Everywhere has worked in the reduction of chronic malnutrition at the national level with participation in the biannual policy exchange on nutrition, food, and economic and social development between Peru and Bolivia; supported the development of a local advisory council on food security; and has supported the regional health departments in Puno by training staff to improve counseling services. 64 Appendix J. List of Persons Met Name Title World Bank Karina Olivas Senior Country Officer Livia Benavides Program Leader Hugo Brousset Social Protection Specialist Nelly Ikeda Financial Management Specialist Rocio La Vera Procurement Specialist Banco Mundial Alessandra Marini Senior Economist and Task Team Leader Ministry of Economy and Finance (MEF) Jessica Nino de Guzman Health Specialist, Department of Public Expenditure Quality Felipe Sarmiento Caldas Evaluation Coordinator, Department of Public Expenditure Quality Zoila Vigo Consultant, Specialist in Social Monitoring Mechanism Diana Flores Former Adviser, Department of Public Budget Carmen Osorio Consultant Ministry of Development and Social Inclusion (MIDIS) Ariela Luna Vice-Minister, Social Polices and Evaluation Nelly Huamaní Adviser to the Minister Yolanda Zuniga Adviser to the Minister Angel Gallegos Pacheco Juntos program Patricia Jave Arias Coordination of Modernization, Juntos program Davis Zavala Velazco Planning and Budget, Juntos program Ada Palpan Guerra Communications Specialist, Juntos program Maria Elena Arburua Family Counseling Specialist, Juntos program Mona Sandy Gongora Gozales Communications Specialist, Juntos program Ministry of Health (MINSA) Maria Ines Sanchez Grinan Adviser to the Minister of Health United Nations Children’s Fund (UNICEF) Hugo Razuri Health Specialist Maria Elena Ugaz Nutrition and Early Childhood Development Specialist Inter-American Development Bank (IADB) Federico Guanis Senior Health Specialist Rosa Asca CARE National Director Delia Haustein PRISMA Executive Director National Institute of Statistics (INEI) Nancy Hidalgo Technical Director Karen Romero Head of Project 65 National Identification and Civil Status Registry (RENIEC) Danilo Chavez Director Information Technology Gala Briceno Deputy Director Software Engineering Department of Cajamarca Matilde Rodriguez Director, Technical Unit Juntos program in Cajamarca Christopher Navarro Former Communications Specialist, Juntos program Jose Luis XX Area Technical Coordinator, Juntos program Field Staff (Gestor) District of Namora Field Staff (Gestor) District of La Encaniada Field Staff (Gestor) District of Baños del Inca Gustavo Quispe Salazar Field Staff, District of Chetilla Edgar Vargas Director of the Health Center Namora Dr. Ines Sanchez Medical Doctor at the Health Center in Namora Dr. Wilson Leon Hospital Director, District Baños del Inca Dr. Cesar Aliaga Mayor of the City of Cajamarca Zoila Vigo Consultant for the MEF, Social Monitoring, Padron Nominal Martha Segarra DIRESA Cajamarca María Nacarino Díaz DIRESA Cajamarca Note: CARE = Cooperative for Assistance and Relief Everywhere; DIRESA = Dirección Regional de Salud (Regional Health Directorate). 66