EL SALVADOR Income Support and Employability Project Report No. 135490 APRIL 23, 2019 © 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2019. El Salvador—Income accuracy of the data included in this work. The Any queries on rights and licenses, including Support and Employability Project. boundaries, colors, denominations, and other subsidiary rights, should be addressed to Independent Evaluation Group, Project information shown on any map in this work do World Bank Publications, The World Bank Performance Assessment Report 135490 not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC Washington, DC: World Bank. World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 135490 PROJECT PERFORMANCE ASSESSMENT REPORT EL SALVADOR INCOME SUPPORT AND EMPLOYABILITY PROJECT (LOAN NO. 7811-SV) April 23, 2019 Human Development and Economic Management Independent Evaluation Group Currency Equivalents (annual averages) Currency Unit = U.S. dollar (El Salvador adopted the U.S. dollar in 2001.) 2009 $1.00 2013 $1.00 2010 $1.00 2014 $1.00 2011 $1.00 2015 $1.00 2012 $1.00 2016 $1.00 2009 $1.00 2017 $1.00 2010 $1.00 2018 $1.00 2011 $1.00 2019 $1.00 2012 $1.00 Abbreviations CPF Country Partnership Framework FISDL Fondo de Inversión Social para el Desarrollo Local (Social Investment Fund for Local Development) GDP gross domestic product IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IEG Independent Evaluation Group INSAFORP Salvadoran Institute of Professional Training LMO labor market observatory M&E monitoring and evaluation MTPS Ministerio de Trabajo y Previsión Social (Ministry of Labor) PAC anticrisis plan PATI Programa de Apoyo Temporal al Ingreso (Temporary Income Support Program) PJCT Youth with Everything Program PPAR Project Performance Assessment Report RUP Registro Único de Participantes (Unique Registry of Beneficiaries) SIPATI Information System of Programa de Apoyo Temporal al Ingreso SPSU Sistema de Protección Social Universal (Universal Social Protection System) STP Secretaría Técnica y de Planificación de la Presidencia (Technical and Planning Secretariat of the Presidency) All dollar amounts are U.S. dollars unless otherwise indicated. Fiscal Year Government: January 1–December 31 Director-General, Independent Evaluation Ms. Alison Evans Director, Human Development and Economic Management Mr. Auguste Tano Kouame Manager, Corporate and Human Development Ms. Emanuela Di Gropello Task Manager Mr. Eduardo Fernandez Maldonado PPAR Primary Author Ms. Carla Pazce ii Contents Preface ................................................................................................................................................................vii Summary ............................................................................................................................................................. ix Lessons .......................................................................................................................................................... xii 1. Background and Context ..........................................................................................................................1 Project Context .............................................................................................................................................2 Relevance of the Objectives and Design..............................................................................................3 Objectives ..................................................................................................................................................................... 3 Design ............................................................................................................................................................................ 5 2. Implementation ............................................................................................................................................7 Implementation Experience ......................................................................................................................8 Safeguards Compliance ........................................................................................................................................... 9 Financial Management and Procurement ....................................................................................................... 10 3. Achievement of the Objectives ............................................................................................................ 11 Objective 1: Provide Temporary Income Support to the Urban Vulnerable Poor .................11 Outputs ......................................................................................................................................................................... 11 Outcomes ................................................................................................................................................................... 12 Objective 2: Improve the Coverage of Labor Intermediation and Training Services to the Urban Vulnerable Poor ............................................................................................................................ 16 Outputs ........................................................................................................................................................................ 16 Outcomes ................................................................................................................................................................... 16 Objective 3: Improve the Institutional Capacity of the Borrower to Develop an Integrated Social Protection System ......................................................................................................................... 18 Outputs ........................................................................................................................................................................ 18 Outcomes ................................................................................................................................................................... 19 4. Efficiency ...................................................................................................................................................... 22 5. Ratings .......................................................................................................................................................... 23 Outcome ...................................................................................................................................................... 23 Risk to Development Outcome ............................................................................................................ 23 Objective 1 ................................................................................................................................................................. 23 Objective 2 ................................................................................................................................................................ 24 Objective 3 ................................................................................................................................................................ 24 iii Bank Performance .................................................................................................................................... 25 Quality at Entry ........................................................................................................................................................ 25 Quality of Supervision ........................................................................................................................................... 26 Borrower Performance ............................................................................................................................ 27 Government Performance.................................................................................................................................... 27 Implementing Agency Performance................................................................................................................. 27 Monitoring and Evaluation .................................................................................................................... 28 Design ......................................................................................................................................................................... 28 Implementation ....................................................................................................................................................... 29 Use................................................................................................................................................................................ 30 6. Lessons.......................................................................................................................................................... 30 Bibliography..................................................................................................................................................... 34 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 37 Appendix B. Theory of Change................................................................................................................. 39 Appendix C. The Temporary Income Support Program.................................................................. 41 Appendix D. Impact Evaluation: Temporary Income Support Program.................................... 44 Appendix E. Qualitative Evaluation Temporary Income Support Program.............................. 51 Appendix F. List of Persons Met............................................................................................................... 56 This report was prepared by Eduardo Fernandez Maldonado and Carla Pazce, who assessed the project in November 2018. The report was peer reviewed by Hjalte S. A. Sederlof and panel reviewed by Judyth L. Twigg. Gaby Loibl provided administrative support. iv Principal Ratings Indicator ICR ICR Review PPAR Outcome Satisfactory Satisfactory Satisfactory Risk to development Substantial Substantial Substantial outcome Bank performance Satisfactory Satisfactory Satisfactory Borrower performance Satisfactory Moderately satisfactory Moderately satisfactory Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. PPAR = Project Performance Assessment Report. Key Staff Responsible Management Appraisal Completion Project Team Leader Edmundo Murrugarra Nancy Rocio Banegas Raudales and Edmundo Murrugarra Sector Manager or Practice Manager Helene Ribe Margaret Grosh Sector Director or Senior Global Evangeline Javier Michal Rutkowski Practice Director Country Director Laura Frigenti J. Humberto Lopez v IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vi Preface This is a Project Performance Assessment Report of the Income Support and Employability Project (P117440) in El Salvador. The project development objective was to (i) provide temporary income support to the urban vulnerable poor, (ii) improve the coverage of labor intermediation and training services to the urban vulnerable poor, and (iii) improve the institutional capacity of the borrower to develop an integrated social protection system. The project was approved in November 2009 (fiscal year [FY]10) with an original closing date envisioned for December 2014 (FY15). The project underwent five restructurings throughout implementation. The final closing date was August 2016 (FY17). At appraisal, project cost was estimated at $50 million and government counterpart funding was $4 million. By project closing, $49.3 million of the loan proceeds were disbursed. The Independent Evaluation Group (IEG) undertook a mission to El Salvador in November 2018, during which it interviewed World Bank staff and government officials involved in the project design, implementation, and evaluation. In addition, IEG interviewed relevant development partners and conducted site visits to labor intermediation offices. IEG also interviewed additional World Bank staff in Washington, DC. The assessment used a mixed methods approach that included a desk review of documentation (midterm review, evaluation studies, and appraisal, supervision, and completion reports), site visits, and interviews with key project stakeholders. IEG thanks the El Salvador government officials and World Bank staff involved in the project for sharing their experiences in designing, implementing, and self-evaluating the project. IEG also thanks other development partners who shared their views on World Bank support, and the World Bank Country Office in San Salvador for the outstanding administrative and coordination support provided for conducting this assessment. Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback. The borrower did not have any comments on the report. vii Summary The government of El Salvador lacked adequate responses to help the urban poor mitigate the impact of the 2009 global economic crisis or cope with its effects. The crisis quickly destabilized the economy and disproportionally affected urban areas because of the location of maquilas, the incidence of remittances, and the fact that urban areas have a higher concentration of El Salvador’s poor population (58 percent) than other areas of the country. When the 2009 crisis affected El Salvador, the country lacked a comprehensive and universal social protection system, and its antipoverty programs centered on rural areas; consequently, the urban poor were not covered by any intervention that would help them mitigate the impact of the crisis or cope with its effects. Additionally, interventions to increase employability had limited geographical coverage and primarily served a population with a secondary or university education. The government launched an anticrisis plan (PAC) to address the impact of the crisis on the urban poor. It asked for World Bank support to develop short-term interventions for the urban poor and to design a medium-term, overarching social protection system. Immediately after taking office in June 2009, the Mauricio Funes administration launched a PAC comprising short- and medium-term interventions to protect the most vulnerable. As part of the short-term interventions for the urban poor, the government identified the need to design and implement the Temporary Income Support Program (Programa de Apoyo Temporal al Ingreso; PATI) and to strengthen labor market intermediation to improve economic opportunities. Regarding medium-term interventions, the government identified the need to provide a minimum standard of services and economic opportunities for all Salvadorans through the establishment of the Universal Social Protection System (Sistema de Protección Social Universal; SPSU) with a rights-based approach. The Funes administration asked for support for the PAC in areas in which the World Bank has a comparative advantage: income support and temporary employment programs, labor intermediation, conditional cash transfer programs, and capacity building for designing and implementing social policies. The World Bank provided support through the Income Support and Employability Project to strengthen the government’s PAC. The project development objectives were (i) to provide temporary income support to the urban vulnerable poor (objective 1), (ii) improve the coverage of labor intermediation and training services to the urban vulnerable poor (objective 2), and (iii) improve the institutional capacity of the borrower to develop an integrated social protection system (objective 3). These objectives were not modified during implementation. To achieve its objectives, the project financed inputs and activities to (i) support the design and implementation of PATI, (ii) strengthen labor intermediation, information and occupational training systems, and (iii) support the design and development of the SPSU. ix The relevance of objectives is rated substantial. The project development objectives were aligned with the development level of the social protection system in the country. Until the crisis, the country lacked interventions to address poverty and employability challenges among the urban vulnerable. Additionally, the country lacked a universal social protection system. Project objectives were fully aligned with country priorities and World Bank strategies at appraisal. At closing, these objectives remained aligned, though more so objectives 1 and 2 than objective 3 since the government’s Five-Year National Development Plan (2014–19) and the World Bank Country Partnership Framework (2016–19) deemphasized the focus on the SPSU (objective 3) that was a characteristic of the 2009 PAC and the World Bank Country Partnership Strategy (2010– 12). The relevance of project design is rated substantial. The project design articulated a convincing results chain and had activities consistent with the stated objectives. The results chain had a minor shortcoming, namely it lacked a clear, up-front articulation of some program assumptions. For instance, the results chain did not identify a critical assumption at appraisal, which is that PATI implementation required free mobility of both program personnel and participants. Although this issue generated challenges throughout implementation, implementing agencies managed to address it succesfuly. In retrospect, the identification of these assumptions could have helped the team to design project activities better. The achievement of objective 1 (provide temporary income support to the urban vulnerable poor) is rated substantial. The program met or exceeded most of its targets regarding delivering income support to the urban vulnerable poor (with some minor shortcomings in the program uptake by some key groups). By supporting eight rounds of PATI that provided temporary monetary transfers to 44,239 participants (against a target of 43,000), the project benefited 22.4 percent of the population 16 years or older living in the targeted, most-vulnerable and precarious urban settlements (against a target of 20 percent). The program exceeded its target regarding female participation but fell slightly short of its target for participation of youth and individuals completing community and training activities. The program also achieved overall positive related outcomes beyond the scope of the project’s objectives on labor force participation, employment, income, and institutional strengthening. The achievement of objective 2 (improve the coverage of labor intermediation and training services to the urban vulnerable poor) is rated substantial. Even though progress was limited in establishing a fully functional labor market observatory, the project contributed to improving the coverage and quality of labor intermediation services while improving access to these services by vulnerable populations such as women and youth. As of 2016, the coverage of labor intermediation services had x increased and exceeded project targets, notably in the number of employment offices, job seekers, and employers enrolled in the National Employment Network; employment fairs organized; and job seekers participating in those fairs. Additionally, as a possible encouraging sign of the effectiveness of the services, the project also improved the quality of the staff from the Ministry of Labor conducting labor intermediation, and interviews revealed that private sector actors were generally satisfied with the ministry’s intermediation services. The achievement of objective 3 (improve the institutional capacity of the borrower to develop an integrated social protection system) is rated modest. Although the project achieved the enactment of a regulatory framework for the Universal Social Protection System (a notable result), the design and implementation of key policy tools for social protection policy had several shortcomings. For example, although the Unique Registry of Beneficiaries is in place, its status as a unique registry remains unclear, and there are concerns about its coverage, quality of the data, and sustainability of the Unique Registry of Beneficiaries team. Additionally, the capacity of the monitoring and evaluation (M&E) system has deteriorated, and the system’s sustainability is uncertain. Furthermore, the project made no progress on implementing a social information system. Efficiency is rated substantial. The cost-benefit analysis conducted at appraisal showed that the project was worth the investment, with a net present value of $41.5 million and an internal rate of return of 10 percent. At closing, the project updated the cost-benefit analysis using data from a quantitative impact evaluation of the PATI program. The analysis estimated the internal rate of return at 11 percent, and the estimation of the net present value of benefits (using three discount rates) showed that for each dollar invested per beneficiary, the program obtained a benefit in wage gains equal to $3.70 (3 percent discount rate), $2.60 (6 percent discount rate), and $1.50 (12 percent discount rate). Implementation had some minor shortcomings that diminished efficiency, but the shortcomings were adequately addressed and efficiency bottlenecks removed. Considering the substantial rating for relevance of objectives and design, the ratings for the three objectives (substantial for objectives 1 and 2 and modest for objective 3), and the substantial rating for efficiency, the overall outcome is rated satisfactory. Risk to development outcome is rated substantial, driven by the high likelihood that the modest progress made under objective 3 will not be maintained. Overall Bank performance is rated satisfactory. Quality at entry is rated satisfactory because the World Bank developed an adequate technical design for the project that aligned with the country context and priorities and was based on lessons learned from similar projects in Latin America and on existing analytical work. Additionally, the World Bank identified most project risks and elaborated adequate mitigation measures. xi The World Bank also adequately assessed the implementing agencies’ institutional capacity and designed implementation arrangements that were in line with this capacity, though with minor shortcomings. Quality of supervision is rated satisfactory because World Bank supervision efforts were adequate and proactive. Furthermore, the World Bank made several course corrections during implementation that considered the government’s changing priorities and incorporated lessons learned from evaluations and ongoing implementation feedback to maximize development impact. Overall borrower performance is rated moderately satisfactory. Government performance is rated moderately satisfactory. Generally, the government showed ownership and commitment to achieving development objectives. However, commitment to achieving objective 3 (improve the institutional capacity of the borrower to develop an integrated social protection system) wavered throughout implementation. Implementing agencies performance is rated satisfactory. The project had complex implementation arrangements involving several agencies that generally performed adequately. However, there were shortcomings regarding implementation readiness for one agency and insufficient coordination between two of the implementing agencies for the interoperability of their M&E systems. M&E is rated substantial. Although the results framework had an output orientation consistent with the project objectives, the overall M&E arrangements included several management and information systems and evaluation studies that together generated additional information on critical longer-term outcomes implicit in the project’s theory of change. Implementation of the M&E arrangements was generally adequate, though with minor shortcomings. Regarding use, overall evidence shows that the project team and the implementing agencies used M&E findings to maximize development impact through program adjustments and to inform the design of subsequent interventions. Lessons A crisis period can create opportunities for broader reform of an existing social protection system through projects that adequately balance short and long-term objectives. The onset of the global economic crisis in El Salvador revealed the social protection system’s limitations. The government’s desire to protect affected groups quickly, coupled with its limited fiscal space, showed the need for coherent and sustainable social programs that are cost-effective and reach the neediest. Project design supported the PATI while incorporating longer-term support to build a universal social protection system. Additionally, the reforms of the social protection system initiated with the project provided the government with the institutional framework and tools that, if sustained, would enable a more rapid response in the face of future shocks. xii Ensuring high participation and higher incomes for vulnerable groups requires interventions that are specifically tailored to address the needs of these groups. The project included women and youth as part of its categorical targeting, but both faced barriers to participation in the program. For example, youth faced barriers to participation and completion, specifically the risk of violence and coercion because they had to move across different gang territories to attend training sessions and community activities. Women’s participation could have been higher if the program had arranged for childcare. Additionally, evidence indicates that women and youth also faced several barriers to paid employment such as traditional gender roles and employers refusing to hire youth from high-crime areas. These findings call for rejecting one-size-fits-all approaches in favor of interventions specifically tailored to the needs of each vulnerable group. In an emergency context, a project can be implemented rapidly and effectively by using the country’s existing capacity when the project relies on an implementing agency with a proven track record and is accompanied by close World Bank supervision. The project relied on the Social Investment Fund for Local Development for implementing the PATI program because the agency had experience working in similar settings, including engaging with municipalities and providing monetary transfers based on targeting mechanisms. Proactive and flexible World Bank supervision complemented this preexisting capacity. Consequently, the government started piloting PATI before effectiveness, and once the project started disbursing, it reached 56 percent of beneficiaries within the first two years, despite being a new and complex program targeting urban high-crime areas. The support of knowledgeable local actors is critical for the successful implementation of an intervention in areas affected by high levels of crime and violence. The territorial division of gangs in El Salvador limited the mobility of PATI beneficiaries and program staff by creating borders that, though invisible, could not be crossed without facing coercion and violence. Most locals know these invisible borders, but they are unknown to outsiders. Municipalities brought deep contextual knowledge of crime and violence dynamics that were highly instrumental to maneuvering in a dangerous environment. Auguste Tano Kouame Director Human Development and Economic Management Independent Evaluation Group xiii 1. Background and Context 1.1 El Salvador is a small and densely populated lower-middle-income country. It is highly exposed to natural hazards and characterized by a dollarized economy with low levels of savings and investment. With a surface area of 8,124 squares miles and a population of 6.3 million, El Salvador is the smallest country in Central America and one of the most densely populated in the western hemisphere. El Salvador is among the countries most affected by weather-related events and other hazards, incurring annual losses of about 2.5 percent of gross domestic product (GDP; World Bank 2015b). El Salvador has a GDP purchasing power parity per capita of $7,760. During 2000–13, gross national savings stood at an average of 11 percent of GDP, lower than the average for Latin American countries (16 percent) and lower-middle-income countries (20 percent). Similarly, investment rates are low and place El Salvador in the bottom 7th percentile worldwide with 14 percent of GDP. 1.2 Elevated levels of crime and violence and high rates of migration and remittances also characterize El Salvador. As of 2013, El Salvador ranked in the top five countries worldwide for homicide rates (UNODC 2013). Since 2000, the number of homicides has consistently been more than five times the world average, peaking in 2015 to a high of 105 homicides per 100,000.1 In addition, the country also suffers from high levels of crime related to street gangs (maras). The economic costs of crime and violence are a huge drag on the economy, and some studies estimate their economic costs at about 11 percent of GDP (World Bank 2011a). The country is also characterized by high rates of migration with one in three Salvadorans living abroad, especially in the United States as of 2011 (World Bank 2015a). Migration has been accompanied by large remittance flows (amounting to 16 percent of GDP in 2012) that constitute an important source of income for many households, including the bottom 40 percent of the income distribution. 1.3 The country’s development performance has been characterized by perpetuating vicious circles of low economic growth since the end of the civil war in 1992. El Salvador stands outs as one of the slowest-growing economies in the Latin America and the Caribbean Region with an average growth rate of real GDP per capita of less than 1 percent from 1996 to 2001, and an average 1.5 percent after 2001. According to the recent World Bank Systematic Country Diagnostic, three vicious circles reinforce this low growth (World Bank 2015a). The first is created by the combination of low growth and violence. Crime and violence negatively affect overall economic growth, which leads to fewer opportunities for the population, thus creating incentives for at-risk groups to engage in criminal activities. The second vicious circle is created by the combination of low growth and high migration and remittance flows. Salvadorans 1 emigrate to the United States because of limited economic opportunities and high crime rates. Remittances associated with these migration flows negatively affect labor force participation by increasing reservation wages. Moreover, these remittance flows lead to an appreciation of the real exchange rate. These factors combine to undermine the country’s competitiveness and, ultimately, economic growth. A third vicious circle is created by the interaction between low growth, savings, and investment. According to empirical evidence, the savings rate associated with remittance flows is lower than the rate for nonremittances income, resulting in a lower aggregate savings rate that constrains investments and leads to low growth. 1.4 Although income equality and access to basic social services have improved, the persistent low growth has led to slow poverty reduction. Access to basic social services improved since the end of the Civil War and the Gini coefficient declined by 5 points, from 0.47 in the early 2000s to 0.42 in 2012, reflecting an improvement in income distribution and putting the country’s level of inequality below the Latin America and the Caribbean average (0.52 in 2012). However, poverty remains high and its reduction has been sluggish compared with the Latin America and the Caribbean average. As of 2012, 41 percent and 11 percent of the population were poor and extreme poor based on national definitions. Regarding poverty reduction, poverty declined in the Latin America and the Caribbean Region three times faster than in El Salvador between 2003 and 2012. Project Context 1.5 The project was prepared against the backdrop of the food, fuel, and financial crisis of 2009 that destabilized the economy, reversed poverty reduction gains, and hit the urban poor particularly hard. GDP growth decelerated from 4.7 percent in 2007 to 1.3 percent in 2008, exports shrank by 16 percent in the first five months of 2009, and more than 30,000 formal jobs were lost between September 2009 and April 2009 (83 percent of them in the maquila sector).2 Moreover, remittances dropped from 18 percent of GDP in 2008 to 11.7 percent in 2009, and fiscal revenues declined on a net basis by 10.9 percent by August 2009. The crisis disproportionally affected the urban vulnerable because of the location of maquilas, the incidence of remittances, and the fact that urban areas have a higher concentration of El Salvador’s poor population (58 percent). At the time of the crisis, El Salvador had only one main antipoverty program but lacked a systematic approach to social protection. Additionally, the urban poor were not covered by any intervention that would help them mitigate the impact of the crisis or cope with its effects. The situation of the urban poor contrasted with the situation of rural populations who were covered by the Red Solidaria program (later Comunidades Solidarias Rurales), which provided conditional cash transfers to the rural poor. 2 1.6 The government launched an anticrisis plan (PAC) to address the impact of the crisis on the urban poor. It asked for World Bank support to develop short-term interventions for the urban poor and to design a medium-term, overarching social protection system. Immediately after taking office in June 2009, the Funes administration launched a PAC comprising short- and medium-term interventions to protect the most vulnerable. As part of the short-term interventions for the urban poor, the government identified the need to design and implement the Temporary Income Support Program (Programa de Atención Temporal al Ingreso; PATI) and to strengthen labor market intermediation to improve economic opportunities. Regarding medium-term interventions, the government identified the need to establish the Universal Social Protection System (Sistema de Protección Social Universal; SPSU) with a rights-based approach to provide a minimum standard of services and economic opportunities. The Funes administration asked for support for the PAC in areas in which the World Bank has a comparative advantage: income support and temporary employment programs, labor intermediation, conditional cash transfer programs, and capacity building for designing and implementing social policies. Relevance of the Objectives and Design Objectives 1.7 The project development objectives were to (i) provide temporary income support to the urban vulnerable poor, (ii) improve the coverage of labor intermediation and training services to the urban vulnerable poor, and (iii) improve the institutional capacity of the borrower to develop an integrated social protection system (World Bank 2010, 6). The objectives were not modified throughout implementation, but the third restructuring (July 2013) increased the target for one of the outcome indicators (number of individuals who receive PATI support), reflecting expansion of the PATI program. This Project Performance Assessment Report (PPAR) will not do a split evaluation because the indicator was changed only to increase the target, and that target was exceeded by project closing. Relevance of the Objectives 1.8 The project development objectives were aligned with the level of development of El Salvador’s social protection system. The project was prepared against the backdrop of the 2009 crisis that negatively affected the urban vulnerable. Until the crisis, El Salvador lacked a comprehensive and universal social protection system. Moreover, its antipoverty interventions centered on rural areas, and interventions to increase employability had limited geographical coverage, serving primarily a population with a secondary or university education. Although objectives 1 and 2 had output-oriented 3 formulations, these two objectives had an adequate level of ambition given the crisis context in which the project was prepared and the development trajectory of El Salvador’s social protection system. In 2016, the International Labour Organization constructed a social protection index that assessed social protection systems on three dimensions (universality, solidarity, and social spending) and ranked social protection systems in three categories (limited, intermediate, and comprehensive). The study found that El Salvador can be characterized as having a limited system, together with Guatemala, Honduras, and Nicaragua (Ocampo and Gómez-Arteaga 2016). 1.9 The project development objectives were broadly aligned with country priorities at appraisal and closing. At appraisal, the Funes administration launched a PAC to address the 2009 crisis that increased poverty and unemployment among the urban vulnerable. The first project objective supported the PAC in its objective of creating interventions to address urban poverty through the provision of income support. The second project objective supported the PAC in its objective of expanding the coverage of labor intermediation services with the aim of improving the employability of the urban vulnerable in the medium term. The third project objective supported the PAC in its objective of creating a comprehensive SPSU with a rights-based approach. At closing, the objectives remained broadly aligned with the Sánchez Cerén administration’s Five- Year National Development Plan (2014–19) that included actions to strengthen labor intermediation (under objective 1) and actions to reduce poverty and strengthen the social protection system (under objective 5). However, as shown by the lack of concrete actions to continue strengthening the SPSU, the plan deemphasized the focus on the SPSU that was a characteristic of the PAC. 1.10 Similarly, the project development objectives were broadly aligned with the World Bank country strategies at appraisal and closing. The objectives were fully aligned with strategic objectives 2 (strengthen delivery of social services) and 3 (increase economic opportunities, particularly for the poor) of the World Bank’s Country Partnership Strategy 2010–12, which was valid at appraisal. At closing, the objectives remained broadly aligned with the World Bank’s 2016–19 Country Partnership Framework (CPF), valid at closing. However, although the first and second objectives were fully aligned with the CPF 2016– 19 objective 3 (enhance youth employability and skills), the third objective (improve the institutional capacity of the borrower to develop an integrated social protection system) was not reflected as a priority in the CPF 2016–19. 1.11 The overall relevance of the project development objectives is rated substantial. 4 Design 1.12 Components. The project design had three components: (i) support the design and implementation of the PATI, (ii) strengthen the labor intermediation, information, and occupational training systems, and (iii) support the design and development of the SPSU. The Technical and Planning Secretariat of the Presidency (Secretaría Técnica y de Planificación de la Presidencia; STP) had overall coordination responsibility for the project given that it was the government agency accountable for the strategic direction and coordination of El Salvador’s SPSU. In addition, the project designated the STP as the main World Bank interlocutor on implementation progress and assigned it the responsibility for ensuring that course corrections to address shortfalls in targets would take place if needed. 1.13 Component 1: Support the design and implementation of the PATI (appraisal: $37.3 million; actual: $40.7 million).3 The Social Investment Fund for Local Development (Fondo de Inversión Social para el Desarrollo Local; FISDL) was the implementing agency for this component, whose aim was to support the implementation of a six- month income support program (PATI) for vulnerable individuals in urban areas. As a condition for receiving the benefits, individuals would need to take part in approved PATI subprojects that included community and training activities. Specifically, the component financed (i) temporary income support for participants, (ii) training for participants, (iii) institutional strengthening of the FISDL and municipalities to manage and implement PATI, and (iv) municipalities’ administrative expenses for managing PATI subprojects. Appendix C provides a description of the PATI program, including details on cash transfers and conditionalities, targeting, community subprojects, and training activities. Although the FISDL was responsible for implementing this component, municipalities and the Salvadoran Institute of Professional Training (Instituto Salvadoreño de Formación Profesional; INSAFORP) played important roles. Specifically, municipalities were responsible for the identification and design of community subproject proposals, and FISDL was responsible for approving subproject proposals submitted by municipalities. When approved, municipalities were responsible for their implementation and supervision, and for reporting beneficiaries’ attendance to the FISDL monthly. The INSAFORP supported municipalities by arranging training for PATI subprojects through the identification of local trainers and suitable training modules. Because attendance to training was a prerequisite for receiving income support, the INSAFORP was responsible for monitoring and reporting attendance to FISDL. The FISDL was also responsible for contracting a financial agent to make monthly payments to PATI participants. 5 1.14 Component 2: Strengthening of labor intermediation, information, and occupational training systems (appraisal: $10.3 million; actual: $6.7 million). This component had four activities: • Strengthen the National Employment Network by expanding the coverage of local employment offices (bolsas de empleo) to reach the urban vulnerable • Strengthen the National Employment Network by improving the quality of the services provided by local employment offices through (i) technical assistance for assessing the role and services provided by the local employment offices, (ii) training for local employment offices’ personnel for improving the adequacy and efficacy of the services provided to the urban vulnerable, and (iii) the establishment of coordination mechanisms with municipalities and the private sector for better identifying employment opportunities • Improve the quality of labor market information that guides the policy formulation process through the creation of a labor market observatory (observatorio del mercado laboral; LMO) • Improve the coordination of the current occupational training system to address labor needs through the development of a national strategy for occupational training. The project dropped this last activity at the third restructuring (July 2013) because it was no longer a government priority. The original design assigned the Ministry of Labor (Ministerio de Trabajo y Previsión Social; MTPS) overall responsibility for implementing this component, though financial management and procurement responsibilities were placed under the responsibility of the FISDL. The third restructuring changed this division of labor and brought these responsibilities under the MTPS. 1.15 Component 3: Support the design and development of the SPSU (appraisal: $2.4 million; actual: $2.6 million). This component had two activities. The first activity was to strengthen the STP institutional capacity through the following activities: (i) preparation of an assessment of STP’s capacity to implement the SPSU, (ii) development of a regulatory framework for the institutional coordination of the SPSU, (iii) training for STP personnel, and (iv) development of a communication strategy for the SPSU. The second activity was to support the development of instruments for the design, evaluation, and dissemination of social protection policies. Specifically, the component supported the following instruments: (i) a Unique Registry of Beneficiaries (Registro Único de Participantes; RUP) of all social programs, (ii) a monitoring and evaluation (M&E) system, and (iii) a social information system for citizens. The STP was 6 responsible for the administration, implementation, financial management, and procurement of all activities under this component. Relevance of the Design 1.16 The project design articulated a convincing results chain and had activities consistent with the stated objectives. The articulated results chain clearly linked funding and planned activities to outputs, intermediate outcomes, and outcomes, as follows: (i) The provision of monetary transfers to the urban poor would allow them to smooth their consumption, preventing them from falling deeper into poverty; (ii) by conditioning the transfers to the participation in training programs and community activities, the beneficiaries would experience sustainable improvements in skills and behaviors that would enhance their employability and ultimately allow them to take part in the labor market and improve their income prospects in the medium term; (iii) expanding and improving the quality of labor intermediation and training services would likely improve the access of the urban vulnerable to these services, and improve overall coverage and employability; and (iv) the enactment of a regulatory framework and the design and implementation of key policy tools for social protection policy would likely improve the borrower’s institutional capacity to develop an integrated social protection system. Overall, the project activities could plausibly achieve the stated objectives. See appendix B for an Independent Evaluation Group (IEG) reconstructed theory of change. 1.17 However, the results chain had some minor shortcomings, specifically the lack of a clear up-front articulation of program assumptions. Two examples illustrate this point. First, the results chain did not identify a critical assumption at appraisal, which is that PATI implementation required free mobility of both program personnel and participants. During implementation, this assumption proved untenable given the high level of gang violence in PATI areas. To address this issue, the project team took proactive measures that were successful in addressing this challenge. Second, the PATI program prioritized participation of women head of households, but their participation could have been higher if the program had arranged for childcare. In retrospect, the identification of these assumptions could have helped the team to design project activities better. 1.18 The overall relevance of design is rated substantial. 2. Implementation 2.1 Project cost and financing. The total project cost estimated at appraisal was $54 million, of which $50 million was expected to be financed by an International Bank for Reconstruction and Development (IBRD) loan, and $4 million by the borrower as 7 counterpart financing. By project closing, $49.4 million of the IBRD loan proceeds had been disbursed. The Implementation Completion and Results Report (ICR) reported that the $4 million borrower contribution did not materialize (World Bank 2017a, 23). 2.2 Dates. The project was approved on November 24, 2009 and closed on August 31, 2016, lasting almost seven years. Project effectiveness was reached on January 18, 2011, about a year later than envisioned because of delays in project approval by congress. However, even before World Bank Group Board of Executive Directors approval, the government started piloting the PATI program using their own funds and following the project operational manual they had prepared for the project. Appendix A includes details on key project data, actual disbursements, and staff time and costs. 2.3 Restructurings. The project underwent five restructurings. The first restructuring (December 2010) redefined the relationship between FISDL and INSAFORP; it also adopted a new output-based disbursement modality to transfer funds from the FISDL to the INSAFORP based on the number of individuals who completed training rather than assuming a traditional disbursement mechanism would pay for training inputs (World Bank 2017a, 5). The second restructuring (March 2012) changed the eligibility of costs to be financed under the project to include incremental salary costs of locally contracted employees of FISDL, and it reallocated funds between components to allow FISDL to gather the baseline data for the impact evaluation. The third restructuring (August 2013) changed the implementation arrangements by giving the MTPS fiduciary responsibility for component 2. In addition, the restructuring introduced changes to seven indicators in the results frameworks that clarified the formulation of the indicators but did not involve changes in their substance. This restructuring also reallocated funds from component 2 to others, accounting for the expansion of the PATI (from 40 to 43 thousand beneficiaries) and the elimination of activities related to the strengthening of the occupational training system (component 2), which were no longer a government priority. The fourth restructuring (July 2014) reallocated funds and extended the project duration for one year to complete planned activities because delays during the first two years after approval affected the project implementation schedule. The fifth restructuring (September 2015) reallocated funds and extended the project duration for another eight months to allow the World Bank to support the development of a new government youth employment strategy, another round of PATI incorporating elements of the new strategy, and other key activities related to component 2. Implementation Experience 2.4 Factors outside of government control. The activities and territorial division of gangs in El Salvador affected the project implementation at the community level. The two main gangs—Mara Salvatrucha (MS-13) and Barrio 18—divided communities with 8 invisible lines, and people living in those areas (especially youth) had to respect those limits or face threats of violence or coercion. This discouraged eligible people from registering in the program and limited the mobility of PATI participants as they attended training and community activities and collected their payments. The disruptive effect of gangs was present throughout project implementation, but it became more acute in about 2014, when a gang truce reached in 2012 fell apart. 2.5 Factors within government control. Project effectiveness was delayed for a year because the project required congressional approval, and the governing party lacked a majority in congress. Overall, political polarization in the country complicated the approval process, but the World Bank engaged in intensified policy dialogue and stakeholder consultations to build consensus for the program. Moreover, the World Bank intended to mitigate the risk of delayed approval by presenting a package of loans that focused on priority areas where there was broad support (World Bank 2009b, 2017). 2.6 The change in government in 2014 affected project implementation in several ways. Responsibilities for the STP shifted from coordinating the SPSU to preparing and monitoring the national development plan, and additional funding needed for continuing RUP implementation was not provided. Similarly, the approval of the Regulation of Social Protection and Development Law was delayed and approved two years after the law was enacted. 2.7 In the context of campaigning for municipal elections in late 2011 and political turnover in early 2012, some authorities sought to extract political benefit from community subprojects by redirecting activities toward campaigning or stopping activities to distance themselves from the previous administration. After this experience and in preparation for the next election cycle, FISDL signed an agreement with local authorities to protect PATI from political interference. FISDL implemented information campaigns among participants and the community and promoted accountability efforts, such as the preparation of completion reports (El Salvador 2014b). Safeguards Compliance 2.8 Project design adequately incorporated safeguard measures to manage potential environmental impacts. During project preparation, it was determined that project activities—community subprojects that would be part of PATI—would have limited immediate impact on the environment. The project was then classified as category B, and the safeguard policy on Environmental Assessment (OP/BP 4.01) was triggered. However, considering that potential subprojects were small and varied (thus limiting environmental risks) and that PATI activities were to comply with FISDL environmental guidelines as stipulated in the operations manual, no environmental assessment or 9 consultations with stakeholders were conducted during project preparation. Environmental mitigating measures and supervision activities were part of the project design as stipulated in the environmental guidelines. The guiding principle for the community subproject process was to incorporate environmental criteria into the selection and decision process. Key tools to reduce environmental risks were a positive list of eligible subproject types and the required screening of each subproject proposal against a simple checklist to ensure no adverse impacts. Shortcomings in the implementation of these provisions were found during the World Bank’s midterm review. In response to the findings, PATI guidelines and procedures were adjusted, FISDL appointed more staff dedicated to monitoring compliance, and training was provided to municipalities. The shortcomings in the implementation of the environmental guidelines were resolved as later World Bank supervision missions verified the improvements, which led to highly satisfactory ratings for the rest of the project in this area. Financial Management and Procurement 2.9 Financial management. Project financial management was generally adequate and produced reliable and timely information to properly manage and implement project activities. However, midway through implementation, the operational audit for 2013 revealed moderate shortcomings in the project internal controls. Specifically, it found issues related to eligibility verification of several PATI participants who should not have received the transfers. After conducting a special review of these questionable expenditures, the auditors concluded that 54 beneficiaries (of 44,239) did not qualify for the transfers, amounting to $5,600. FISDL acknowledged the issue. Considering that the issue constitutes a small amount of loan proceeds (0.014 percent), the World Bank declared the expenses ineligible but provided a waiver and did not seek recovery of funds. 2.10 Procurement. The project design foresaw that each executing agency had its own procurement plan, which would allow implementation under each component to proceed independently and efficiently. FISDL received training related to procurement and financial processes because it did not have recent experience with World Bank systems. In August 2013, after delays in implementation in component 2, the project was restructured to assign fiduciary responsibilities to MTPS, which was accompanied by hiring additional staff, training, and support from the World Bank procurement team. Procurement processes were rated satisfactory for most of the project period but were downgraded to moderately satisfactory about a year before project closing because of moderate shortcomings in the contract administration of implementing agencies. 10 3. Achievement of the Objectives 3.1 The efficacy discussion is organized by three objectives: (i) provide temporary income support to the urban vulnerable poor, (ii) improve the coverage of labor intermediation and training services to the urban vulnerable poor, and (iii) improve the institutional capacity of the borrower to develop an integrated social protection system. This PPAR uses the following sources of information to assess efficacy: (i) the ICR (World Bank 2017a), (ii) IEG interviews and site visits conducted in November 2018, (iii) a quantitative impact evaluation (FUSADES 2014), and (iv) a qualitative evaluation (FUSADES 2015). The quantitative impact evaluation assessed the short- and medium- term impact of PATI on beneficiaries’ income and likelihood of being employed. In addition, the evaluation measured PATI’s impact on other variables, such as empowerment, social cohesion, and levels of victimization. The methodological strategy relied on propensity score matching to create a comparison group and on a difference- in-differences approach to estimate the program’s impact. The qualitative evaluation sought to complement the findings of the quantitative evaluation by asking about participants’ perceptions regarding PATI’s impact on their agency and employability. Appendixes D and E provide detailed descriptions of the quantitative and qualitative evaluations. Objective 1: Provide Temporary Income Support to the Urban Vulnerable Poor Outputs 3.2 The project supported the implementation of a six-month income support program (PATI) to achieve this objective. The program targeted vulnerable individuals living in precarious urban settlements conditioned to their participation in community subprojects and training. Specifically, the project financed income transfers for 44,239 unique PATI participants, amounting to $26.49 million. In addition, the project provided capacity building support to the FISDL for implementing the PATI program ($3.38 million) and strengthened municipalities’ capacity to implement PATI subprojects ($0.767 million) while covering their administrative expenses for managing PATI subprojects ($3 million). The project also financed the provision of training for PATI participants by the INSAFORP ($6.22 million). Appendix C provides a description of the PATI program, including details on cash transfers and conditionalities, targeting, community subprojects, and training activities. 11 Outcomes Coverage and Targeting Outcomes 3.3 The project exceeded its coverage targets regarding providing temporary income support to the urban vulnerable poor. By supporting eight rounds of PATI providing temporary monetary transfers to 44,239 participants (against a target of 43 thousand), the project benefited 22.4 percent of the population 16 years or older living in the targeted, most vulnerable precarious urban settlements (against a target of 20 percent). 3.4 The quantitative impact evaluation confirmed that overall, the targeting worked as expected, that is, it attracted those from the poorest households who had less human capital and low-paying jobs (FUSADES 2014). In interviews with IEG, key project staff identified two main factors contributing to the successful targeting. First was that the PATI transfer was about half the minimum wage salary in El Salvador ($100), an amount that attracted the unemployed or subemployed without dissuading potential participants from looking for a job. Second, key project staff noted that throughout implementation, the project placed special emphasis on outreach and monitoring efforts to make sure that individuals living in precarious urban settlements signed up for the program. 3.5 However, program uptake and completion had some shortcomings. Regarding uptake by key groups, the program exceeded its target for female participation (70.8 percent against a 60 percent target). However, participation for youth ages 16–24 fell slightly short of its target (37 percent against a 40 percent target). Similarly, the share of individuals completing community and training activities fell slightly short of its target (86.5 percent against a target of 90 percent). IEG interviews with key program stakeholders and the FUSADES (2015) qualitative evaluation identified several barriers to uptake and completion for women and youth. Regarding women, key program staff noted that women’s participation could have been even higher if the program had provided participants with childcare services. Youth also faced barriers to participation and completion, specifically the risk of violence and coercion because they had to move across different gang territories to attend training sessions and community activities (El Salvador 2014b; FUSADES 2015). Key program stakeholders noted that youth participation would have been lower if the program had not taken specific measures to mitigate these risks. For example, they noted that municipalities implemented several measures (with varying degrees of success) to protect participants from gang violence during PATI participation (El Salvador 2014b; World Bank 2017). 12 Labor Market and Poverty Outcomes 3.6 Looking beyond the narrowly defined objectives, the quantitative impact evaluation shows that the program also had an impact on several labor market–related outcomes. PATI participation was associated with a 4.6 percentage point increase in labor market participation (defined as paid and unpaid employment, self-employment, or searching for a job) in the short and medium terms (5.5 percentage points). This increase in labor force participation was observed in women (5.9 percentage points short term and 6.6 medium term), youth ages 16–24 (9 percentage points short term and 8.4 percentage points medium term), and in individuals with ninth-grade education or more (9.0 percentage points short term and 10.9 medium term). Additionally, PATI participation was also associated with increased employment (defined as paid or unpaid employment and self-employment) in the medium term (5.5 percentage points). The study did not find short-term impact. The positive impact on employment in the medium term was observed for women (6 percentage points), youth ages 16–24 (6.2 percentage points), and individuals with ninth-grade education or more (8.5 percentage points). According to the quantitative evaluation, these positive results can be attributed to the fact that PATI participation is associated with an increased share of individuals who feel prepared for conducting a job search or starting self- employment. 3.7 Overall, PATI met its objective of increasing the income of the urban poor, but it did not affect the groups (within the urban poor) that were a specific PATI priority: youth and women. The study found an average $17 increase in individuals’ monthly employment income one year after program completion. This positive impact was observed for men ($56.70), adults ages 25–64 ($27.40), and individuals with less than a ninth-grade education ($24.60). The study did not find impact on average monthly employment income for women and youth, and individuals with ninth-grade education or more. To understand this result, IEG interviewed key program implementers and reviewed the qualitative evaluation (FUSADES 2015). IEG found that women and youth faced several barriers to the type of labor force participation that is likely to have an impact on income, specifically paid employment. According to the qualitative evaluation, beneficiaries perceived very few opportunities for paid employment in the labor market, thus making them less likely to invest effort in searching for this type of employment. Key program staff noted that the lack of beneficiaries’ formal education was an obstacle to paid employment. They also noted that insecurity was another obstacle because paid employment opportunities are usually outside of the neighborhoods where vulnerable youth and women live. Additionally, key program staff noted that many employers would refuse to hire individuals from high-crime areas. Furthermore, the qualitative evaluation found that the presence of traditional gender roles was another deterrent to paid employment for women. Under these traditional 13 roles, caring for the household and children are women’s responsibility, thus making their household commitments sometimes incompatible with those of paid employment (FUSADES 2015). 3.8 PATI’s impact on poverty outcomes was mixed. The quantitative impact evaluation found that PATI participation was not associated with a decrease in poverty. Specifically, after a year of program completion, PATI participation did not have an impact on the share of participants who lived in households with poverty. However, the study found that PATI participation was associated with a decrease in extreme poverty. A year after program completion, the share of PATI participants who lived in households with extreme poverty decreased by 8 percentage points. This decrease was observed for men (11.2 percentage points), youth ages 16–24 (9.8 percentage points), adults ages 25–64 (7.8 percentage points), and individuals with less than a ninth-grade education (9.4 percentage points). By contrast, the study did not find any impact for women and individuals with a ninth-grade education or more. Empowerment, Social Cohesion, and Victimization Outcomes 3.9 The program had a mixed impact on empowerment, social cohesion, and violence measures. Regarding empowerment, the quantitative impact evaluation found that program participation was associated with a positive short-term impact in mood measures, but participation did not increase self-esteem. Furthermore, participation did not increase household decision-making measures for women. Regarding social cohesion, the quantitative impact evaluation hypothesized that participation in community activities would increase social interactions and thus social cohesion. The evaluation found that although participation in civil society organizations and satisfaction with community life increased, trust measures diminished. Regarding violence measures, the quantitative impact evaluation found that PATI participation was associated with a medium-term increase in the share of households reporting victimization (4.4 percentage points) with a higher-than-average medium-term increase for men (9 percentage points) and adults ages 25–64 (5.1 percentage points). IEG interviewed key program implementers to understand the reasons behind this increased victimization. These actors explained that these results might be associated with the fact that PATI raised the incomes of men and adults ages 26–64 and increased the visibility of these individuals in their communities, thus making them more likely to suffer victimization acts. Institutional Strengthening Outcomes 3.10 The experience of implementing the PATI program strengthened FISDL’s capacity, and these gains are likely to be maintained. According to implementation supervision reports, the project provided procurement training to the FISDL that 14 improved its overall fiduciary capacity (World Bank 2011b, 2012). Regarding its operational capacity, interviewed FISDL staff noted that the agency gained important experience in implementing a targeted social transfer program in an urban context, which it had not been done before. FISDL staff also reported that the project strengthened the FISDL’s capacity to supervise compliance with environmental safeguards and improved FISDL’s M&E capacity. Overall, FISDL’s enhanced capacity is likely to be maintained because the PATI experience and many of its tools were captured in operational manuals, procedures, guidelines, and evaluation studies. In addition, IEG fieldwork confirmed that although the PATI team was not maintained as such, several key staff members continue working at the FISDL or have been absorbed by other institutions involved in the implementation or oversight of social programs, such as the STP or the National Institute for Youth—the latter is currently implementing the Youth with Everything Program (Programa Jóvenes con Todo; PJCT), that has a similar design as PATI’s. 3.11 The project strengthened the capacity of some municipalities, but these capacity gains are less likely to be maintained. Municipalities hired staff and received training, equipment, and technical assistance to formulate PATI subproject proposals and to coordinate and supervise their implementation. Overall, FISDL staff who worked closely with municipalities reported that the project strengthened the institutional and technical capacity of municipalities. For example, because of the project, staff from municipalities received training on compliance with environmental safeguards, and many municipalities created environmental units. However, FISDL staff noted that the capacity would likely dissipate over time as new local authorities are elected and staff turnover occurs. 3.12 The project improved INSAFORP’s capacity to provide training for urban vulnerable youth. Before the project, the INSAFORP lacked experience in providing training to these populations. Project documents reported that the INSAFORP managed to adapt its training content and delivery modes to reach and respond to the needs of urban vulnerable youth, but it took time at the initial stages of implementation to do so. For example, INSAFORP shortened the training durations to align them with PATI’s six- month timeline and implemented mobile training brigades to work around the effects of gang activity in urban areas. Generally, FISDL and STP interviewees confirmed that the INSAFORP’s capacity to attend urban vulnerable youth had increased because of the project. Overall, this increased capacity is likely to be sustained because the INSAFORP continues to deliver training programs to the urban vulnerable youth, currently in coordination with the National Institute for Youth for the PJCT Program. 3.13 Efficacy for this objective is rated substantial because the program met or exceeded most of its targets for delivering income support to the urban vulnerable poor 15 (with some minor shortcomings in the program uptake by some key groups). Additionally, the program also achieved overall positive related outcomes beyond the scope of the project’s objectives on labor force participation, employment, income, and institutional strengthening. Objective 2: Improve the Coverage of Labor Intermediation and Training Services to the Urban Vulnerable Poor Outputs 3.14 The project financed a geographical expansion of labor intermediation services, capacity building for the MTPS, and the creation of an LMO to achieve this objective. The project financed equipment and furniture for new employment offices, hired the staff needed for their operations (which included labor counselors, business outreach coordinators, and technical staff for entrepreneurship activities), and bought 30 pickup trucks for intermediation and outreach activities. Additionally, the project financed technical assistance to strengthen the quality of intermediation services and for improving the overall institutional capacity of the MTPS. Regarding the LMO, the project hired and trained human resources for the observatory, acquired computer equipment and software licenses, designed a quarterly report of labor market indicators, conducted a national campaign to disseminate LMO production, and piloted the implementation of a new survey of labor statistics. Outcomes Coverage and Labor Intermediation Services 3.15 The project exceeded its coverage targets for labor intermediation for the urban vulnerable poor. Before the project, the MTPS had limited coverage with 28 employment offices serving mostly individuals with relatively high levels of education (World Bank 2017a, 13). As of 2016, the coverage of labor intermediation services had increased and exceeded all its targets as shown by (i) an increase in the number of employment offices (71 against a target of 66), (ii) an increase in the number of job seekers enrolled in the National Employment Network (237,767 against a target of 50,000), (iii) an increase in the number of employment fairs organized (235 against a target of 50), (iv) an increase in the number of job seekers participating in employment fairs (118,294 against a target of 40,000), and (v) an increase in the number of employers enrolling in the National Employment Network (5,401 against a target of 800).4 Additionally, the coverage of intermediation services among the vulnerable also increased as shown by an increased enrollment for youth ages 18–25 (127,293 against a target of 44,000), women (51 percent against at target of 53 percent), and individuals with less than a complete secondary 16 education (57 percent against a target of 60 percent). To date, the MTPS reports that there are 60 employment offices operating in the entire territory, which is less than the 71 offices operating by the end of the project (El Salvador 2018). To understand this decrease, IEG interviewed MTPS staff who reported that in some municipalities, authorities declined to continue operating employment offices, presumably for political reasons. Despite the decrease, the MTPS retains its presence in the country’s 14 departments and continues to forge alliances with municipalities and other government programs to operate the National Employment Network (El Salvador 2018).5 Overall, the increased geographical expansion of labor intermediation services is unlikely to be reversed. 3.16 The project improved the quality of the MTPS staff conducting labor intermediation. With technical assistance provided by the project, the MTPS defined competencies and elaborated job descriptions for the positions of labor counselor and business outreach coordinator. Using these job descriptions, the project supported the hiring of individuals to fill these positions. Interviews with MTPS staff confirmed the importance of these new positions in improving the quality of labor intermediation and contrasted these improvements with the situation before the project, when the MTPS lacked qualified staff for conducting labor counseling and business outreach. The ICR did not report on the number of labor counselors, business outreach coordinators, and other staff hired by the project, and IEG could not find precise data on staff numbers through interviews with MTPS staff. However, staff informed IEG that at project closure, the MTPS allocated budget resources to hire personnel previously financed by the project, though the MTPS absorbed 22 labor counselors, 20 business outreach coordinators, and 5 entrepreneurship technicians—only a fraction of the project- financed staff (El Salvador 2016d). Overall, given that the MTPS assigned budget resources to absorb some of the project staff, these quality improvements are unlikely to be reversed. 3.17 Although quality of intermediation services has improved and job placement rates have increased, the MTPS lacks comprehensive measures to show that improved quality translated into more effective and efficient intermediations services. According to data provided by the MTPS, job placement rates increased from 12 percent in 2009 to 52 percent in 2018.6 However, IEG fieldwork found that the MTPS lacked other measures to demonstrate effectiveness. For example, IEG found that no data are available on job retention rates because the system does not follow individuals beyond confirmation of employment. Similarly, the MTPS lacked measures to track how quickly participants become employed after receiving labor intermediation services or to assess the efficiency of labor intermediation services.7 While interviewing MTPS staff, IEG found that the MTPS lacked a formal feedback mechanism to track private sector satisfaction with 17 intermediation services. To assess the quality of intermediation services from a private sector perspective, IEG contacted the five firms that use the MTPS services the most and the Salvadoran Chamber of Commerce. Interviews revealed that private sector actors were generally satisfied with the MTPS intermediation services and reported adequate responsiveness of business outreach coordinators to support them in filling their jobs needs. However, some actors questioned the extent to which outreach coordinators showed initiative in approaching them to ask about future needs, suggesting that they might react only to companies’ job postings in the system. Labor Market Observatory 3.18 Progress has been limited in establishing a fully functional LMO. Although the LMO has been established and is operating, its four staff members do not have the leadership needed, and the unit has been without a head for more than a year and a half. At project closing, the observatory had produced 39 bulletins and nine labor market studies. Some of these studies were disseminated with project support, but they are currently not available on the MTPS’ website. Some interviewees from the implementing agencies were unaware of the LMO’s reports, and others were critical of their usefulness for policy making. Fieldwork evidence suggests that the LMO lacks technical expertise (key staff left for positions outside the ministry), and the lack of clarity regarding the LMO’s functions affected the reports’ usefulness for making policy decisions within the ministry. Currently, the U.S. Department of Labor is supporting improvements to the LMO, including training its personnel and a describing its vision and objectives better, which allows some sense of continuation of the work started with the project. 3.19 This objective is rated substantial considering that even though progress was limited in establishing a fully functional LMO, the project contributed to improving the quality and coverage of labor intermediation services while improving access to these services by vulnerable populations such as women and youth. Objective 3: Improve the Institutional Capacity of the Borrower to Develop an Integrated Social Protection System Outputs 3.20 The project provided overall capacity support for the STP, technical assistance for the design and implementation of key social protection policy tools, and the development of a regulatory framework for the SPSU to achieve this objective. Regarding overall capacity support and development of a regulatory framework, the project financed (i) an assessment of the STP’s institutional capacity to implement the SPSU, (ii) consultancy services for developing and disseminating a regulatory framework for the SPSU, (iii) knowledge and training activities for STP staff, and (iv) 18 activities to position the SPSU in the national agenda. Regarding key tools for social protection policy, the project financed technical assistance, training for staff, and equipment to support the design of a RUP for all social programs, along with an M&E system and a social information system for citizens. Outcomes Regulatory Framework 3.21 The project strengthened the STP’s institutional capacity and met its target for developing a regulatory framework for the SPSU. The project financed an institutional capacity assessment of the STP and identified capacity gaps. To fill those gaps, the STP staff received technical training and took part in knowledge-generation activities. Regarding the development of a regulatory framework, the Salvadoran Congress approved the Social Protection and Development Law in April 2014 that established a legal framework for the SPSU. Thus, the project met the outcome target for this objective (normative framework for the SPSU approved). The law mandated three key actions: (i) the development of the National Development and Social Protection Plan, which was approved in January 2015, (ii) the enactment of a regulation of the law (Decree Number 43, 2016), which was approved in July 2016, and (iii) the design of a prioritization mechanism, which was developed through the RUP with project support. Interviews with STP staff reveal that the World Bank, together with the United Nations Economic Commission for Latin America and the Caribbean, played a key role in the development of the law. Moreover, STP staff noted that the project supported activities (such as the 2013 Social Protection Forum and the elaboration of conceptual documents) that were key for positioning the STP as the coordinator of social protection policy in the country and for disseminating the law among congressional representatives. 3.22 STP staff acknowledged that the enactment of the regulatory framework was an important milestone, but they also noted that it had shortcomings. Although the law and its regulations specified that a governmental agency would be responsible for coordinating the SPSU, they failed to specify whether this agency was the STP or another governmental body. Fieldwork evidence confirmed that the capacity to coordinate social protection policy remains in the STP because key personnel who have institutional knowledge, were involved in discussions of the law, and are familiar with the overall vision for the SPSU continue to work there. However, it is unclear whether future Salvadoran governments will increase the STP’s profile, giving it a more explicit and stronger role as coordinator of social protection policy. Additionally, the regulatory framework did not mandate using the RUP for all social programs, thus leaving the possibility of targeting some social programs d with methods other than the RUP. 19 Policy Tool 1: Unique Registry of Beneficiaries 3.23 The project supported the development of a tool for targeting social benefits. Along with the United Nations Economic Commission for Latin America and the Caribbean, the project supported the design of the RUP.8 Regarding this policy tool, the project results framework had two intermediate outcome indicators: (i) “Normative framework for the RUP approved by the SPSU Intersectoral Committee,” and (ii) “Percentage of Comunidades Solidarias program participants registered in the RUP.” The first intermediate outcome indicator was achieved in March 2013 when the SPSU Intersectoral Committee approved the RUP operations manual. Regarding the second intermediate outcome indicator, the project fell short of its 75 percent target because only 66.7 percent of Comunidades Solidarias program participants were registered in the RUP. In interviews with STP staff, IEG asked about the reasons for this shortfall. Staff acknowledged that the project financed only the conceptualization and design of the RUP but did not include financing for data collection, and the government did not allocate resources for this endeavor. To address this lack of funding, the STP sought funding from the U.S. Agency for International Development and the European Union, both of which supported data collection in 56 rural municipalities and 25 urban municipalities, that is, nearly one-third of the 262 municipalities (World Bank 2017). When these funds were exhausted, the STP partnered with the Ministry of Health, which was implementing a program of mobile health brigades to collect health records in rural communities. Together with health surveys, these mobile health brigades applied a reduced version of the RUP questionnaire.9 By 2017, the RUP had increased its coverage to 180 municipalities through this partnership. 3.24 IEG fieldwork identified several challenges that require attention. The RUP requires additional financing for conducting data collection exercises in urban municipalities where coverage remains low. Additionally, the regulatory framework does not make it mandatory to use the RUP for all social programs, which constrains its usefulness. IEG fieldwork found that STP staffing for the RUP is at risk. As of November 2019, four staff members form the RUP team, of which two have temporary positions financed through funds provided by the Spanish Cooperation Agency. Interviews with STP staff revealed that these funds would be exhausted soon. Policy Tool 2: Monitoring and Evaluation System 3.25 The project results framework lacked indicators to measure achievements regarding this policy tool, but there is evidence that can be used to ascertain capacity improvements. Specifically, the project supported training and financed the hiring of personnel for the STP M&E unit created in 2010. Additionally, World Bank staff guided and supported the M&E unit in managing and supervising the PATI program impact evaluation, thus increasing the unit’s capacity and allowing it to later produce and 20 supervise rigorous evaluations. Using this newly built capacity, the STP subsequently conducted the evaluation of other strategic social programs and developed a repository of evaluations, which included 31 evaluation reports at project closing (World Bank 2017a). In interviews with IEG, STP staff noted that the improvement of the STP’s evaluation and monitoring capacity made it possible to promote social programs in general because the STP could show the impacts of various social programs. For example, stakeholders noted that STP evaluation of the PATI program was instrumental in developing the new government program PJCT. 3.26 Despite this good progress, IEG fieldwork found several shortcomings and challenges with the M&E system. The public cannot access the repository of evaluations through the STP website, and some development partners mentioned that they were unaware of this resource. Additionally, the overall capacity created within the STP for M&E has waned, and sustainability of the achievements in this area remained uncertain. In interviews with IEG, STP staff revealed that the number of staff in the unit was reduced because of changing priorities, and most of the unit’s efforts were shifted away from strategic evaluations of social programs and redirected toward monitoring the National Development Plan 2014–19 (El Salvador 2015). IEG interviews also found that the M&E system lacked an adequate follow-up mechanism to ensure uptake of evaluation findings and recommendations by implementing agencies. Policy Tool 3: Social Information System 3.27 The project made no progress in developing a social information system because of disagreements about the system’s purpose. At appraisal, the system’s purpose was not specified clearly.10 Interviews with STP staff confirmed that there never was a clear agreement on the system’s purpose. For some stakeholders, the system would provide the STP with information on the beneficiaries of all social programs to assess the adequacy of the programs’ scope and the beneficiaries’ eligibility criteria. For others, the system’s purpose was to provide citizens with direct access to customized information on the programs and benefits for which they were eligible based on their socioeconomic characteristics. Additionally, IEG interviews revealed that the system’s design and implementation had a division of labor between the United Nations Development Programme and the World Bank that complicated execution.11 Although the system was not implemented, the STP coordinated with different social programs to obtain access to information that could later feed into the social information system. By the end of the project, only 25 percent of the Comunidades Solidarias social programs had shared information that could feed into the future social information system, thus not meeting the target of 75 percent for intermediate outcome indicator 14, percentage of Comunidades Solidarias that incorporated their programmatic offerings into the social information system of the SPSU (World Bank 2017a, vi). 21 3.28 The project achieved the enactment of a regulatory framework for the SPSU, but this objective is rated modest considering the following: (i) Although the RUP is in place, its status as a unique registry remains unclear, and there are concerns about its coverage, quality of the data, and sustainability of the RUP team; (ii) the capacity of the M&E system has deteriorated, and its sustainability is uncertain; and (iii) there was no progress on implementing a social information system. 4. Efficiency 4.1 The economic analysis conducted at appraisal showed that the project was worth the investment. At appraisal, the project team conducted a cost-benefit analysis to estimate the PATI program’s impact (component 1, 75 percent of the IBRD loan at appraisal). The analysis assumed that the project would have three benefits: (i) PATI income transfers, (ii) increased income, and (iii) higher chances of employment. The amount of PATI transfers was estimated at $24 million (assuming 40,000 recipients would receive $600 each). To estimate the other two benefits, the analysis assumed that the intervention would lead to a 6 percent increase in the likelihood of being employed during the four years after the training, and that PATI participants would command a permanent income premium of 5 percent. 12 The analysis did not consider other potential private and public benefits, such as reduced costs related to crime or poverty reduction and enhanced equity.13 Regarding costs, the cost of implementing component 1 was identified as the main cost to be factored into the analysis ($37.3 million at appraisal). Using these benefits and costs, the analysis estimated a $41.5 million net social revenue, assuming a 20-year time horizon (10 percent internal rate of return). 4.2 At closing, the project updated the economic analysis with new data and corroborated that benefits outweighed costs, thus, that the investment was efficient. After what was done at appraisal, the ICR conducted a cost-benefit analysis of the PATI program (component 1, 81 percent of IBRD loan at closing) but focusing on the long- term gains of increased employability and earnings that were attributable to the program as identified in the impact evaluation (FUSADES 2014). Specifically, the impact evaluation found that the likelihood of being employed in the medium term was 5.5 percent higher for PATI beneficiaries than for nonbeneficiaries; in the medium term, the average wage of PATI participants was 13 percent higher than the average wage of nonbeneficiaries. Regarding costs, the analysis considered the financing of training and stipends for PATI beneficiaries and the cost of operating the program. Using several tenable assumptions,14 the analysis estimated the internal rate of return of the flow of costs and benefits and the present value of benefits using three discount rates (3, 6, and 12 percent). The analysis estimated the internal rate of return at 11 percent, and the estimation of the present value of benefits showed that for each dollar invested per 22 beneficiary, the program obtained a benefit in wage gains equal to $3.70 (3 percent discount rate), $2.60 (6 percent discount rate), and $1.50 (12 percent discount rate). Overall, the analyses at appraisal and at closing indicate that the project benefits outweighed its costs, thus the investment was efficient. 4.3 Implementation had some minor shortcomings that diminished efficiency. Some of these shortcomings were outside of the project’s control, such as gang violence that affected the mobility of program personnel and participants. The ICR reports that a small number of beneficiaries were unable to attend PATI trainings because of gang violence, though it does not provide the exact number. Another outside factor that negatively affected efficiency was a delay in project effectiveness of more than one year caused by lack of congressional approval. The delay negatively affected the implementation and disbursement schedule. Regarding factors within the project’s control, the original implementation arrangements placed fiduciary responsibilities under the FISDL for component 2, but this arrangement overstretched FISDL’s capabilities, resulting in significant delays and slow disbursements for component 2. The third restructuring (July 2013) removed this bottleneck by placing fiduciary responsibilities under the MTPS. The change brought the project back on track, thus accelerating the implementation pace and disbursements for component 2. 4.4 Despite the minor implementation shortcomings that diminished efficiency, overall efficiency is rated substantial given the favorable returns identified through the economic and financial analysis and proactivity in removing bottlenecks to implementation efficiency that were under the project’s control. 5. Ratings Outcome 5.1 The relevance of objectives and project design is rated substantial. Efficacy and efficiency are both rated substantial. Considered together, these ratings lead to an overall outcome rating of satisfactory. Risk to Development Outcome Objective 1 5.2 Some of the institutional strengthening outcomes associated with the implementation of the temporary income program might not be maintained. Even though PATI was conceived as a temporary program, it was expected to create the knowledge and capacity to respond to future negative economic shocks. A PATI-like 23 program has not been rolled out to address the effects of another emergency like the 2009 crisis, but some programs have adopted PATI design elements, learned from its evaluations’ findings, and absorbed some of the former PATI team members.15 Additionally, the project strengthened FISDL’s fiduciary, technical, operational, and supervisory capacity and the INSAFORP’s capacity to provide tailored training for urban vulnerable youth. Overall, these capacity gains are likely to be maintained. However, the capacity generated in municipalities to design, implement, and supervise subprojects has a significant risk of being lost because of likely staff turnover that typically accompanies changes in elected municipal authorities. The overall likelihood that institutional strengthening outcomes will be reversed is moderate. Objective 2 5.3 Maintaining the coverage achieved in labor intermediation services and the increased MTPS institutional capacity depends on continued budget allocations and collaboration with municipalities in operating employment offices. The MTPS partnered with municipalities to open decentralized employment offices in each of the country’s 14 departments. This increased coverage faces moderate political risk because the polarized political environment could lead some municipalities to cancel their agreements with the MTPS, disrupting the provision of intermediation services.16 Regarding institutional strengthening outcomes, MTPS gained solid fiduciary and operational experience with project support, and the ministry absorbed some of the staff financed by the project, which provides some continuation. By contrast, the LMO faced staffing issues that persist to date. Current support by the U.S. Department of Labor that builds on the conceptualization and capacity created by the project creates the possibility of future consolidation of the observatory. However, the consolidation of the observatory would be contingent on stronger political will and an accompanying budget allocation.17 Overall, this assessment found that the risk to the sustainability of the outcomes associated with this objective is moderate. Objective 3 5.4 The government’s important institutional gains toward developing an integrated social protection system risk stalling without a renewed political push to expand the regulatory framework’s scope and strengthen the role of the entity behind social protection policy coordination, and if no additional budgetary support is provided to continue developing key policy tools for targeting, M&E, and information systems for social programs. The project’s contribution to passing El Salvador’s Social Protection and Development Law was a highly important achievement, and although it is unlikely the principles stated in the law will be reversed, continued implementation requires refinements and clarifications.18 Without continued political and budgetary support, the 24 institutional capacity temporarily gained within the STP—particularly in the development of the policy tools—could be lost. STP staff and other development partners concurred that the STP needs additional resources to retain personnel with the institutional knowledge and experience to implement key policy tools and, more important, the RUP. Overall, this assessment found that the risk to the sustainability of the outcomes associated with this objective is high. 5.5 Overall, the risk to development outcome is rated substantial, driven by the high likelihood that the progress made under objective 3 will not be maintained. Bank Performance Quality at Entry 5.6 The World Bank developed an adequate technical design for the project that aligned with the country context and priorities, and was based on lessons learned from similar projects in Latin America and on existing analytical work. The project appropriately aimed at addressing the short-term effects of the crisis, that is, protecting the urban poor from falling deeper into poverty as they faced a decline in jobs and remittances. At the same time, the project incorporated medium-term interventions that would increase the employability of the urban vulnerable poor and pave the way for the development of a universal social protection system. Project design built on the World Bank’s previous experience with the Red Solidarias program and on existing analytical work, notably the urban poverty maps prepared by the Latin American Social Sciences Institute and the United Nations Development Programme. The World Bank also incorporated lessons from similar projects in the Region, including the need to use evaluation findings to improve project design, promote the involvement of stakeholders to build political consensus, consider the country implementation capacities to avoid overcomplex projects, and use communication strategies effectively. 5.7 The World Bank identified most project risks and elaborated adequate mitigation measures. The project team appropriately identified the risk of delays in project approval by the National Assembly, delays in implementation caused by institutional complexity and capacity of municipalities, and the risk of political capture of the PATI program, among others. Generally, mitigation measures were appropriate though not always sufficient. For example, to mitigate the risk of a delayed approval, the project engaged political stakeholders in policy dialogue and in project design and consultations. Despite these efforts, project approval was delayed for more than one year. Although the project failed to anticipate that crime and gang activities would interfere with project implementation, it developed adequate mitigation measures throughout implementation. 25 5.8 The World Bank adequately assessed the implementing agencies’ institutional capacity and designed implementation arrangements that were in line with this capacity, though with minor shortcomings. After the World Bank’s assessments, the project assigned the MTPS with overall responsibility for implementing component 2, but it gave fiduciary responsibilities to the FISDL. At the time, the decision to give fiduciary responsibilities to the FISDL was a sensible choice because the MTPS lacked adequate capacity. However, through project implementation, the arrangement proved impractical because it overstretched FISDL’s capabilities, resulting in significant delays and slow disbursements for component 2. The project eventually brought fiduciary responsibilities back to the MTPS after the project strengthened the MTPS’ fiduciary capacity. In hindsight, FISDL and MTPS staff noted that the project should have taken this latter action at the beginning of project implementation. Despite this minor shortcoming in the design of the implementation arrangements, quality at entry is rated satisfactory. Quality of Supervision 5.9 World Bank supervision efforts were adequate and proactive, according to project documents and stakeholders’ views. The World Bank proactively adjusted the interagency arrangements between INSAFORP and FISDL to allow for an output-based disbursement mechanism, even before the project reached effectiveness in December 2010. Similarly, the World Bank closely supported the STP in conceptualizing and preparing the social protection law and the RUP. Overall, key government officials involved in the project design and implementation praised the World Bank for its flexibility and adaptability to the country’s circumstances and political process. At the third restructuring in August 2013, the World Bank took a calculated risk by giving the MTPS fiduciary responsibilities over component 2. MTPS’ commitment, along with close World Bank support in fiduciary and financial management issues, paid off and resulted in improved disbursement rates and attainment of project results. A minor shortcoming was that the World Bank flagged issues with the implementation of environmental guidelines for PATI community subprojects only at the midterm review stage. However, the issues were resolved after World Bank recommendations were adopted, as confirmed by subsequent World Bank supervision of subprojects. 5.10 Additionally, the World Bank made several course corrections during implementation that considered the government’s changing priorities and incorporated lessons learned from evaluations and ongoing implementation feedback to maximize development impact. Priorities changed as the Sánchez Cerén administration started its tenure in 2014 because the incoming government emphasized youth development and crime reduction in its national strategy. In response, the World Bank extended the project’s closing date to include activities supporting the preparation of a youth 26 employment strategy and to add another round of PATI that would concentrate on youth and high-crime areas. In addition, the World Bank used the findings of the process, impact, and qualitative evaluations to adjust PATI’s design. For example, based on the qualitative evaluation, the World Bank increased the number of technical training hours because they were considered insufficient. Quality of supervision is rated satisfactory. 5.11 Overall, Bank performance is rated satisfactory. Borrower Performance Government Performance 5.12 Generally, the government showed ownership and commitment to achieving development objectives. The government was highly committed to implementing the PATI program (first project objective) and took proactive measures when obstacles arose. For example, because project effectiveness was delayed by lack of National Assembly approval, the government started implementation of PATI pilots in two municipalities using its own resources. Moreover, at appraisal, the government was highly committed to strengthening the SPSU. To this end, it took proactive measures to ensure progress toward this objective, such as making STP the strategic coordinator of social protection policy and later passing a social development law. 5.13 However, commitment to some project objectives wavered throughout implementation. In June 2014, the Sánchez Cerén administration took office and introduced changes in priorities that resulted in decreased government commitment to the third project objective (improve the institutional capacity of the borrower to develop an integrated social protection system). Specifically, the changes made the STP the coordinator for national strategic planning, thus diverting attention away from its role as main coordinator of the SPSU. In addition, the envisioned government funding for the collection of household data for the RUP was reallocated to the new administration’s priorities and, although a new approach was implemented by STP and partnered with the Ministry of Health to collect information, the government could not fully overcome RUP’s implementation delays (World Bank 2017a, 8). 5.14 Government performance is rated moderately satisfactory. Implementing Agency Performance 5.15 The project had complex implementation arrangements involving several agencies that performed adequately. The project included five implementing agencies: FISDL, INSAFORP, MTPS, municipalities, and STP. Generally, these agencies showed 27 commitment to achieving development results and proactivity in addressing implementation bottlenecks. For example, several key project staff highlighted FISDL’s role in dealing with attempted political interference with the PATI program. To address this issue, FISDL took proactive measures to protect the program from political manipulation. Key project staff also praised the municipalities’ role in mitigating the effects of gang violence on the PATI program and the MTPS’ commitment to development results by taking the challenge of assuming fiduciary responsibilities for component 2. The implementing agencies also proactively adjusted the project design based on feedback from monitoring systems and evaluation studies. For example, in response to findings from a qualitative evaluation showing that training duration was too short, the INSAFORP increased the length of the training to 80 hours in the second PATI round and to 160 hours in the last PATI round. 5.16 There were shortcomings regarding implementation readiness for one agency and some inadequate coordination on topics that required the involvement of more than one agency. IEG interviews with key project staff revealed that the MTPS team involved in project design lacked knowledge and understanding of the MTPS’ Employment Department’s challenges and limitations because members of this department were not part of the project team. The team composition was adjusted only after a new Minister took office, and attention was given to implementation delays, which led to greater efforts to implement and disburse assigned funds and ultimately prompted the ministry’s request to the World Bank to assign fiduciary responsibility for component 2 to the MTPS. Another shortcoming was inadequate coordination between the FISDL and the MTPS for the interoperability of their M&E systems (see the Monitoring and Evaluation section for a description of this problem). Despite these shortcomings, the implementing agency performance is rated satisfactory. 5.17 Overall, borrower performance is rated moderately satisfactory. Monitoring and Evaluation Design 5.18 The M&E framework had an output orientation that was generally adequate for measuring the formulated project development objectives, but there were some shortcomings. Given the development trajectory of El Salvador’s social protection system, the formulation of objectives had an output orientation. For this reason, the results framework included outcome and intermediate outcome indicators that were output oriented and adequate for measuring the objectives. However, several shortcomings are notable. The intermediate outcome indicators chosen for the third objective (improve the institutional capacity of the borrower to develop an integrated 28 social protection system) did not fully reflect all the activities that the project supported. For example, there were no indicators to measure the project support to the creation of an M&E system. The results framework also included intermediate outcome indicators for the third objective (percentage of Comunidades Solidarias participants registered in the RUP) that were outside of the project’s control, thus making the project accountable for intermediate outcomes for which the project could not have any effect. 5.19 The results framework had an output orientation consistent with the project objectives, but the overall M&E arrangements included several management and information systems and evaluation studies that together generated additional information on critical longer-term outcomes implicit in the project’s theory of change. The M&E design planned for the implementation of a system to monitor PATI program performance (Sistema de Información de PATI; SIPATI). This system would monitor, among other things, attendance to community subprojects and trainings under the PATI program. The FISDL coordinated the system, which relied on information provided by the INSAFORP and municipalities on attendance records. In addition, municipalities were also responsible for generating information on PATI subprojects. To monitor labor intermediation outcomes, the MTPS collected data on the number of individuals who found a job through the employment offices. These data were collected through the system for employment intermediation. The overall M&E arrangements included qualitative and quantitative impact evaluations and process evaluations of the PATI program. The purpose of the process evaluations was to help project management understand the operational aspects and possible bottlenecks in the PATI program, whereas the quantitative impact evaluation assessed income and employability outcomes that were not measured in the results framework. The quantitative impact evaluation also measured effects on outcomes where the PATI program was expected to have indirect effects, such as empowerment and levels of violence. The qualitative evaluation’s purpose was to understand participants’ perceptions and opinions about PATI better. Implementation 5.20 Generally, implementation of the M&E arrangements was adequate, though it had minor shortcomings. Specifically, SIPATI was not implemented until 2014. Before implementation, FISDL staff collected data on PATI manually using Microsoft Excel forms. SIPATI automated data collection and integrated with the management and information system from INSAFORP to monitor individuals’ participation in required training. Additionally, SIPATI implementation did not adequately coordinate with the MTPS system for employment intermediation—for this reason, it was unable to capture information on labor intermediation and employment outcomes for PATI participants. SIPATI monitored individuals through the code of the individuals’ contracts signed 29 between the PATI program and each participant, whereas the system for employment intermediation used the National Unique Identification Document (Documento Único de Identidad). Without a common identifier, SIPATI was unable to retrieve data on employment outcomes of PATI participants from the system for employment intermediation. Despite these shortcomings, the implementation of the rest of the M&E system was satisfactory and adequate for generating the information needed for assessing outcomes. Use 5.21 Overall, evidence shows that relevant stakeholders discussed M&E findings and that they were used to adjust implementation and inform subsequent interventions. The ICR reports that the project team always had evaluation meetings at the end of each PATI round, which it used to discuss implementation adjustments based on evaluation findings (World Bank 2017a, 40). The evaluation findings led to concrete PATI program adjustments. For example, the qualitative evaluation found that participants perceived the original 64 hours training duration as too short, and thus the length of the training was increased. The results of the evaluation studies were used to inform the PJCT program’s design, and it became the government’s main youth employment program. 5.22 Despite the minor shortcomings in the results framework and implementation of the M&E system, the overall project M&E is rated substantial. Among other things, the substantial rating is attributable to the rigorous impact evaluation the project team conducted and the evidence showing that the project team used evaluation results to maximize development impact through program adjustments. 6. Lessons 6.1 A crisis period can create opportunities for broader reform of an existing social protection system through projects that adequately balance short and long-term objectives.. The onset of the global economic crisis in El Salvador revealed the social protection system’s limitations. The government’s desire to protect affected groups quickly, coupled with its limited fiscal space, showed the need for coherent and sustainable social programs that are cost-effective and reach the neediest. Project design supported the PATI while incorporating longer-term support to build a universal social protection system. Additionally, the reforms of the social protection system initiated with the project provided the government with the institutional framework and tools that, if sustained, would enable a more rapid response in the face of future shocks. 6.2 Ensuring high participation and higher incomes for vulnerable groups requires interventions that are specifically tailored to address the needs of these groups. The 30 project included women and youth as part of its categorical targeting, but both faced barriers to participation in the program. For example, youth faced barriers to participation and completion, specifically the risk of violence and coercion because they had to move across different gang territories to attend training sessions and community activities. Women’s participation could have been higher if the program had arranged for childcare. Additionally, evidence indicates that women and youth also faced several barriers to paid employment such as traditional gender roles and employers refusing to hire youth from high-crime areas. These findings call for rejecting one-size-fits-all approaches in favor of interventions specifically tailored to the needs of each vulnerable group. 6.3 In an emergency context, a project can be implemented rapidly and effectively using the country’s existing capacity when the project relies on an implementing agency with a proven track record and is accompanied by close World Bank supervision. The project relied on FISDL for implementing the PATI program because the agency had experience working in similar settings, including engaging with municipalities and providing monetary transfers based on targeting mechanisms. Proactive and flexible World Bank supervision complemented this preexisting capacity. Consequently, the government started piloting PATI before effectiveness, and once the project started disbursing, it reached 56 percent of beneficiaries within the first two years, despite being a new and complex program targeting urban high-crime areas. 6.4 The support of knowledgeable local actors is critical for the successful implementation of an intervention in areas affected by high levels of crime and violence. The territorial division of gangs in El Salvador limited the mobility of PATI beneficiaries and program staff by creating borders that, though invisible, could not be crossed without facing coercion and violence. These invisible borders are mostly well-known to locals but unknown to outsiders. Municipalities brought deep contextual knowledge of crime and violence dynamics that were highly instrumental to maneuvering in a dangerous environment. 1World Bank Open Data based on UN Office on Drugs and Crime's International Homicide Statistics database: https://data.worldbank.org/indicator/VC.IHR.PSRC.P5?end=2015&locations=SV&start=2006&vie w=chart 2A manufacturing firm is defined as a maquila when it operates within a fiscal regime that allows it to import intermediate goods on a duty-free or tariff-free basis, process or assemble 31 them (labor valueadded), and then reexport the final good, usually to the originating country (de Hoyos and others 2008). 3 The components’ actual costs are as reported in the World Bank’s operations portal. 4According to Independent Evaluation Group (IEG) interviews with Ministry of Labor (MTPS) staff, public employment offices focused their outreach efforts on small and medium enterprises because larger firms use the services of private recruiting companies. 5According to project documents and IEG interviews with MTPS staff, the MTPS signed agreements with some municipalities in which they committed to hosting intermediation offices in their own premises (El Salvador 2014b; World Bank 2014c). Through these agreements, municipalities financed the use of space, facilities, and the cost of utilities, including internet. The ministry used project funds to pay the salaries of the intermediation office personnel and buy the equipment needed. 6The MTPS staff reported to IEG that the job placement rate for 2009 was 12 percent (of 21,834 individuals who registered to receive intermediation services). In 2018, the placement rate was 52 percent (of 28,195 individuals registered). 7IEG asked MTPS staff about the labor intermediation costs per person and how this compared with other similar intermediation services in the region. However, the MTPS staff lacked information in this regard. 8According to IEG interviews with MTPS staff, the project supported the elaboration of a conceptual document, a methodology for prioritization of participants, definition of thresholds, and an operational manual. 9In interviews with IEG, the Technical and Planning Secretariat of the Presidency staff recognized that this partnership has been critical for continuing the data collection effort, but they also raised concerns on the quality of information gathered by health brigades because their data collection focus was different from that of the Unique Registry of Beneficiaries. 10The project appraisal document noted that the social information system would be a “system of public information on all social programs, allowing citizens to access all the necessary information of this offer” (World Bank 2017a, 52) 11Under this division of labor, the United Nations Development Programme would conduct a conceptual design of the system with financing support from the European Union while the World Bank project would finance the software design and rollout. The lack of a clear agreement on the system’s purpose led to delays in the United Nations Development Programme hiring a consultant and the European Union withdrawing the funds, after which this activity remained unfinished. 12To justify the 5 percent permanent income premium and 6 percent increase in the probability of being employed during four years after training, the project appraisal document noted that these figures were chosen in similar analyses done in other similar countries, but it did not reference these studies specifically. 32 13The project team used household survey data to simulate the effect of increased income on poverty and inequality outcomes before and after the Temporary Income Support Program (PATI). The analysis showed that urban poverty would decline by 5 percent because of the income support, and urban inequality would also decline (a 1.7 percent decline in urban Gini coefficient). These poverty and equity effects were not monetized and therefore not included in the cost-benefit analysis. 14To estimate the future cash flow of program benefits, the analysis included several assumptions (World Bank 2017a, 41). It assumed that the average wage gain remains throughout beneficiaries’ active economic life and that the period of active economic life was 31 years, given that that the average age of PATI beneficiaries was 34 years and that the retirement age in El Salvador was 65 years old. The analysis also assumed that the average wage gap between PATI and non-PATI beneficiaries increases and then decreases over time (inverted U shape). Additionally it assumed that PATI beneficiaries enter the labor market after graduation from PATI and that they remain active until retirement, thus the wage of beneficiaries is multiplied by the probability of being employed at each age. The analysis estimated this probability using 2015 household survey data (Encuesta de Hogares de Propósitos Múltiples). The final assumption is that the benefit of the program starts at age 35 (rather than 34) to be consistent with the impact evaluation findings showing that benefits start in the medium term. 15Like PATI, the Youth with Everything Program (PJCT) provides transfers conditioned to beneficiaries’ participation in training activities and entails interinstitutional coordination among the National Institute for Youth, MTPS, and the Salvadoran Institute of Professional Training. Learning from PATI’s experience, the PJCT program adopted a design focused on youth that emphasizes employability and entrepreneurship elements. The PJCT program increased the number of technical training hours from 80 to 250 and program duration from six months to up to two years). Additionally, the PJCT program incorporated a training module on soft skills and dropped the subproject component that characterized the PATI program. 16In interviews with IEG, MTPS staff reported that the problem of municipalities canceling their agreements with MTPS arose in the past, prompting the MTPS to seek partnerships with other local institutions. 17The economic risk is also moderate because of the persistence of the fiscal deficit and the possible impacts it could have on the financing of social programs and public services in general. A fiscal agreement among the country’s political stakeholders that would ensure funding for existing social programs and key functions is expected to be discussed when the newly elected government starts its tenure in June 2019. Although political and fiscal risks are present, the ministry’s hiring of additional personnel and expansion of partnerships somewhat mitigates the possible detrimental impacts. 18The Social Protection and Development Law did not explicitly designate the Technical and Planning Secretariat of the Presidency as the agency in charge of coordinating social protection policy while explicitly naming a subset of social programs for which the law applied, leaving open interpretation on whether new programs or other existing programs are also subject to the principles of the law, including targeting. 33 Bibliography de Hoyos, Rafael E.; Bussolo, Maurizio; Núñez, Oscar. 2008. Can Maquila Booms Reduce Poverty? Evidence from Honduras. Policy Research Working Paper; No. 4789. Washington, DC: World Bank. El Salvador, Government of). 2010. “Five-Year Plan for Development 2010–14.” Government of El Salvador, San Salvador. ———. 2013. Manual Operativo del RUP. San Salvador: Secretaría Técnica de Planificación de la Presidencia. ———. 2014a. “Ley de Desarrollo y Protección Social.” Decree No. 647, Asamblea Legislativa de la República de El Salvador, San Salvador. ———. 2014b. Sistematización de la Experiencia PATI y su Vinculación con la Inserción Productiva y Convivencia Social Comunitaria. San Salvador: Fondo de Inversión Social para el Desarrollo Local. ———. 2015. El Salvador—Building on Strengths for a New Generation. Systematic Country Diagnostic. Washington, DC: World Bank. ———. 2015. Plan Quinquenal de Desarrollo 2014–2019: El Salvador Productivo, Educado y Seguro. San Salvador: Secretaría Técnica de Planificación de la Presidencia. ———. 2016a. “Reglamento de la Ley de Desarrollo y Protección Social. ” Decree No.43, President of the Republic of El Salvador, San Salvador. ———. 2016b. Informe de Labores 2015–16. San Salvador: Ministerio de Trabajo y Previsión Social. ———. 2016c. Informe Final: Proyecto Protección a los Ingresos y Empleabilidad. San Salvador: Secretaría Técnica de Planificación de la Presidencia. ———. 2016d. Fortalecimiento del Servicio Público de Empleo a través del proyecto de préstamo Banco Mundial, Presentación Final. San Salvador: Ministerio de Trabajo y Previsión Social. ———. 2018. Informe de Labores Junio 2017–Mayo 2018. San Salvador: Ministerio de Trabajo y Previsión Social. FUSADES (Fundación Salvadoreña para el Desarrollo Económico y Social). 2014. Evaluación de Impacto del Programa de Apoyo Temporal al Ingreso (PATI). San Salvador: FUSADES. ———. 2015. Qualitative Complement: Evaluating the Impact of the PATI Program. San Salvador: FUSADES. Ocampo, José Antonio, and Gómez-Arteaga, Natalie. 2016. “Social Protection Systems in Latin America: An Assessment.” ILO Working Papers 994902513402676, International Labour Organization, Geneva. UNODC (United Nations Office on Drugs and Crime). 2013. Global Study on Homicide 2013. Vienna: UNODC. World Bank. 2009a. “El Salvador—Income Support and Employability Project.” Project Appraisal Document 50885-SV, World Bank, Washington, DC. ———. 2009b. El Salvador—Country Partnership Strategy, FY10–12. Washington, DC: World Bank. 34 ———. 2010. El Salvador—Loan 7811-SV Conformed Loan Agreement. Washington, DC: World Bank. ———. 2011a. Crime and Violence in Central America: A Development Challenge, Main Report. Washington, DC: World Bank. ———. 2011b. “El Salvador—Income Support and Employability Project.” Implementation Status and Results Report ISR3977, World Bank, Washington, DC. ———. 2012. “El Salvador—Income Support and Employability Project.” Implementation Status and Results Report ISR5773. World Bank, Washington, DC. ———. 2014a. Programa de Apoyo Temporal al Ingreso—ISPA Report. Inter-Agency Social Protection Assessment Initiative, Washington, DC: World Bank. ———. 2014b. “Proyecto de Apoyo al Ingreso y la Empleabilidad, Misión de Apoyo a la Implementación de Actividades del Proyecto, Ayuda Memoria.” World Bank, Washington, DC. ———. 2015. El Salvador—Country Partnership Framework, FY16–19. Washington, DC: World Bank. ———. 2017a. “El Salvador—Income Support and Employability Project.” Implementation Completion and Results Report ICR3976, World Bank, Washington, DC. ———. 2017b. “El Salvador—Income Support and Employability Project.” Independent Evaluation Group, Implementation Completion and Results Report Review ICRR0020665, World Bank, Washington, DC. 35 Appendix A. Basic Data Sheet Income Support and Employability Project (Loan No. 7811-SV; P117440) Table A.1. Key Project Data Actual or Current Appraisal Estimate Estimate Actual as Percent of Financing ($, millions) ($, millions) Appraisal Estimate Total project costs 54.00 Not reported Not reported Loan amount 50.00 49.30 99 Cofinancing – – – Cancellation – 0.70 – Table A.2. Cumulative Estimated and Actual Disbursements Disbursements FY11 FY12 FY13 FY14 FY15 FY16 FY17 Appraisal estimate ($, millions) 8.80 21.20 17.10 1.90 1.00 – – Actual ($, millions) 4.10 14.50 10.20 9.80 8.80 1.40 0.40 Actual as percent of appraisal 47 69 60 518 876 – – Date of final disbursement November 15, 2016 Table A.3. Project Dates Event Original Actual Concept review 08/06/2009 08/06/2009 Negotiations 10/19/2009 10/19/2009 Board approval 11/24/2009 11/24/2009 Signing – 07/19/2010 Effectiveness 02/15/2010 01/18/2011 Closing date 12/31/2014 08/31/2016 37 Table A.4. Staff Time and Cost World Bank Budget Only Staff time Costa Stage of Project Cycle (no. weeks) ($, thousands) Lending FY10 66.13 310,301.51 Total 66.13 Supervision or ICR FY10 7.11 45,679.27 FY11 32.27 169,641.29 FY12 44.43 166,749.01 FY13 44.08 153,210.03 FY14 33.60 126,767.82 FY15 30.75 124,146.31 FY16 26.47 101,262.38 FY17 8.74 30,856.12 Total 227.45 918,312.23 Note: ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. 38 Appendix B. Theory of Change Table B.1. IEG Reconstructed Theory of Change Project Development Objective Intermediation Outcome Final Outcomes Implicit Assumptions Outputs Indicator Indicators (IEG identified) Objective 1: Provide temporary income support to the urban vulnerable poor Temporary income support for Number of individuals Percentage of individuals 16 years Protect income to allow for PATI-targeted groups do not PATI beneficiaries covered by PATI and older in targeted precarious consumption smoothing and face obstacles to enrollment urban settlements covered prevent falling deeper into and completion of the program Training for PATI beneficiaries poverty Percentage of women and youth Food prices remain stable Institutional strengthening of the covered Increase employability and Social Investment Fund for Local income in the medium term Skills gained through Development and municipalities Percentage of PATI participants who participation in PATI subprojects completed training increase employability Economic opportunities available for disadvantaged groups Objective 2: Improve the coverage of labor intermediation and training services to the urban vulnerable population poor Support for expanding coverage of Number of individuals ages Increased percentage of women and Increase employability Economic opportunities local employment offices (Bolsas 18–25 years enrolled in the individuals enrolled who have not available for disadvantaged de Empleo) Red Nacional de Empleo completed secondary education groups Number of new employers enrolled Vulnerable groups do not face Financing activities to improve the obstacles to accessing quality of labor intermediation Number of employment fairs intermediation and training services: organized services 39 (i) Technical assistance for Number of job seekers participating Intermediation and training assessing the role and services in fairs services are of good quality and provided by the local employment relevant to labor market needs offices; Labor Market Monitoring Unit up and running (ii) Training for local employment offices’ personnel for improving the adequacy and efficacy of the services provided to the urban vulnerable; and (iii) Establishment of coordination mechanisms with municipalities and the private sector for better identifying employment opportunities Objective 3: Improve the institutional capacity of the borrower to develop an integrated social protection system Financing activities to strengthen Normative framework for RUP Developed social protection Government continued the institutional capacity of the the Universal Social Operations manual approved system commitment to social Technical Secretariat of the Protection System protection Presidency approved Percentage of Comunidades Solidarias Programs registered in the Government continued Financing the development of key RUP commitment to use the RUP as tools for social protection policy: a targeting mechanism (i) a unified registration system of Percentage of Comunidades participants of all social programs Solidarias Programs with their (the Unique Registry of programmatic offer in the RUP Beneficiaries; RUP); (ii) a monitoring and evaluation system; and (iii) a social information system for citizens Note: IEG = Independent Evaluation Group; PATI = Temporary Income Support Program; RUP = Unique Registry of Beneficiaries. 40 Appendix C. The Temporary Income Support Program The Temporary Income Support Program (PATI) was implemented as a workfare program within the strategy of Comunidades Solidarias Urbanas to mitigate the effects of the 2009 crisis. The program provided income support ($100 per month for a maximum of six months) to poor and vulnerable individuals in urban areas, conditional on individuals’ participation in community subprojects training activities. Overall, the program had two objectives: to mitigate the effects from the 2009 crisis through the income transfers and thus prevent individuals from falling deeper into poverty, and enhance individuals’ employability through participation in trainings and a labor market orientation course. The Social Investment Fund for Local Development (FISDL) and the municipalities acted as main implementing institutions of PATI, and the Salvadoran Institute of Professional Training (INSAFORP) was responsible for designing and providing training activities. The program sought to reach 43,000 individuals, representing more than 30 percent of households living under extremely precarious conditions in the 43 selected municipalities. Income Support Level and Conditionalities To avoid labor market distortions, specifically disincentives to labor supply, PATI sat the level of monthly cash transfers at $100, below the lower minimum wage (at $173 in 2009) and the urban poverty line ($165.70). Municipalities were responsible for monitoring participation in community subprojects, and INSAFORP was responsible for monitoring participation in training activities. FISDL carried out spot checks to ensure compliance with conditionality requirements. The project implemented eight PATI rounds in which 44,239 PATI participants received a transfer of $100 a month for six months. The total amount spent in transfers was $26.49 million. Community Subprojects Considering that PATI was meant to provide rapid support during a six-month period and that participants would be mainly unskilled labor, community projects were required to be small in scale, simple, labor intensive, and suitable for unskilled labor. Community subprojects had to include between 30 and 50 participants and be implemented over six months. The project operational manual provided a positive list of eligible activities and excluded any activities that would require displacement of people or that could have a negative environmental impact. The positive list included, among others, community services (such as basic sanitation and small sidewalks), art-related projects (such as murals and acting), productive activities (such as tilapia farming and 41 vegetable gardens), and training in trades (such as baking and electrical work). Eligible municipalities were responsible for submitting proposals describing the nature of the community activity, the number of participants, expected duration, and their coparticipation. The FISDL was responsible for appraising and approving subproject proposals, and it supported municipalities in the formulation, design, and implementation of the subprojects ($2,500 per subproject). Training Activities As part of the conditionalities, participants were expected to take part in specific trainings to enhance their employability after leaving the PATI program. Each participant received training in the following areas: (i) basic technical training provided for 64 hours originally, increased to 80 hours in the second round, and then duplicated to 160 hours during the las PATI round; (ii) labor orientation training for an additional 16 hours (6 hours training on job searching and 10 hours training on self-employment orientation), originally delivered by the INSAFORP, but from the fourth round on delivered by the MTPS; and (iii) life skills training, which was introduced in the last PATI round for an additional 64 hours. INSAFORP was responsible for providing the technical training after matching participants’ demand for training with the local private sector’s demand for specific profiles and the jobs required at the municipal level. To conduct this matching, INSAFORP carried out three diagnostics for each PATI round: • Diagnosis of the training needs (Diagnóstico de Necesidades de Capacitación). This diagnosis would gather information about the sociodemographic profile of participants and their expectations and preferences for trainings. • Diagnosis of the employment and business opportunities (Diagnóstico de Oportunidades de Negocios y Empleo). Through this diagnostic exercise, INSAFORP would convene municipalities, businesses, local leaders, and other relevant stakeholders to analyze the municipal development opportunities and the jobs profiles needed. • Diagnosis of the potential job supply (Diagnóstico de la Oferta Potencial de Empleo de las Empresas). This diagnostic tool surveyed local businesses to learn about their labor needs. Generally, there was a great mismatch between participants’ preference for training and the job opportunity identified through the diagnostic tool (for example, bakery would usually be at the top of participants’ preferences but at the bottom of businesses’ needs). 42 Targeting PATI targeting involved four steps. First, the program chose the most poor and violent urban municipalities using an urban poverty map elaborated by the Latin American Social Sciences Institute (Facultad Latinoamericana de Ciencias Sociales) and criminality index prepared by the Ministry of Justice and Public Safety (Ministerio de Justicia y Seguridad Publica). Municipalities were eligible if they had a high incidence of precarious urban settlements (asentamientos urbanos precarios) or extreme poverty, and if they were identified as high priority according to the criminality index. This geographical approach provided the first step to targeting households that could become potential participants of the PATI program. Second, potential participants self- selected into the program in each municipality. As part of the eligibility requirements, the program stipulated that individuals must be older than age 16. This age requirement changed through the several PATI implementation rounds. For example, during the seventh round, this requirement was changed to ages 16–40, and in the last round, ages 16–29 were prioritized. Additional requirements included residing in the precarious urban settlement for at least six months, being unemployed, and not attending school or a training program full time during working days. Third, the FISDL assessed participants based on a quantitative scoring mechanism reflecting individuals and households’ socioeconomic conditions. The scoring prioritized women heads of households, individuals ages 16–24; and other socioeconomic variables related to income. Finally, the municipalities validated the list of potential participants generated through the scoring mechanisms through on-site visits to verify socioeconomic conditions. 43 Appendix D. Impact Evaluation: Temporary Income Support Program Purpose The evaluation had a fourfold purpose: (i) to assess the short- and medium-term impact of the Temporary Income Support Program (PATI) beneficiaries’ income; (ii) to assess the impact of PATI on beneficiaries’ employability, which was understood as the likelihood of being employed and prepared for the labor (iii) to understand PATI’s effect on other variables such as empowerment (understood in terms of self-esteem and household decision making); and (iv) to understand PATI’s indirect effect on variables such as community participation, violence, and victimization rates. Although these were not among PATI’s explicit objectives, it was expected that the program would influence them. Finally, the evaluation also sought to identify the impact on these variables by age (ages 16 to 24 versus 25 to 64), gender (women versus men), and education level (less than a ninth-grade education versus ninth grade or more). Methodology The methodological strategy relied on propensity score matching to create a comparison group, and on a difference-in-differences approach to estimate the program’s impact. Propensity score matching is a nonexperimental approach that builds a comparison group based on observed variables that are correlated with the likelihood of participating in the program. The technique assumes no selection bias based on unobserved characteristics. This assumption cannot be verified, and for this reason, it is common practice to combine propensity score matching with another technique that controls for unobserved heterogeneity. In this case, the evaluation study complemented the propensity score matching with a difference-in-differences approach, which controls for any possible time-invariant, unobserved differences between PATI participants and nonparticipants. In addition, the study also conducted a triple differences estimation to control for any possible time-variant, unobserved factors that affect both participants and nonparticipants in the same area (precarious urban settlement or municipality, in this case). The evaluation study conducted three comparisons: 1. Comparison 1: The study compared individuals who registered to take part in the PATI program (T1) with individuals living in the same precarious urban settlements who were eligible to take part but did not register (C1). 2. Comparison 2: The study compared individuals who registered to take part (T2, identical to T1) with individuals living in similar precarious urban settlements 44 not covered by PATI who would be eligible to take part based on observable characteristics (C2). 3. Comparison 3: The study compared eligible individuals who resided in precarious urban settlements covered by PATI (T3) with individuals living in similar precarious urban settlements not covered by PATI who would be eligible to take part based on observable characteristics (C3, identical to C2). The treatment group (T3) for this third comparison included eligible individuals who registered (T1) and those living in the same precarious urban settlement who did not register to take part in PATI (C1). For each of these three comparisons, the study estimated three impacts: the average treatment effect on the treated, the average treatment effect on the intent-to-treat, and the average treatment effect. Sampling Design The study decided to sample the individuals who took part in PATI’s second round that covered 412 precarious urban settlements in 25 municipalities between November 2011 and April 2012. The treatment group comprised 19 municipalities that had only precarious urban settlements participating in the second round of the program. To build the comparison group, the study considered only municipalities that were geographically close to the treated municipalities. Within these treated and nontreated municipalities, the study identified a pool of precarious urban settlements that had at least a minimum number of people who would meet the sample size needed for each. Subsequently, the study randomly chose a pool of precarious urban settlements. In summary, the sample design first randomly chose municipalities, then precarious urban settlements, and then individuals. The sample included both individuals who registered and those who did not register for PATI. The sampling frame for the treatment groups (T1/T2) included all registered individuals for the PATI second round who the FISDL later deemed eligible. Because there was no information on the individuals living in precarious urban settlements, the study had to conduct censuses to build a sampling frame for the control group. FISDL carried out this task between May and June 2011. The sampling frame included all individuals between ages 16 and 64 and who were eligible to take part in PATI. Surveys The study conducted a baseline (September–November 2011) and two follow-up surveys (May–July 2012 and May–July 2013). The baseline survey collected data on 3,804 individuals, of which 1,231 were individuals registered in the PATI program (T1/T2); 45 1,201 were individuals not registered but eligible to take part and living in PATI areas (C1), and 1,372 were individuals living in similar areas not covered by PATI but would be eligible to take part based on observable characteristics (C2/C3). Between the first and second follow-up, attrition decreased the sample size from 3,656 to 3,436 individuals. Table D.1 shows sample sizes by treatment and control group and attrition rates. The study did not find systematic different rationales for attrition among the groups. Table D.1. Sample Sizes and Attrition Rates PATI Municipalities Other Municipalities Registered Not registered Eligible to register Survey (T1/T2) (C1) (C2/C3) Total Baseline (no.) 1,231 1,201 1,372 3,804 First follow-up (no.) 1,190 1,149 1,317 3,656 Second follow-up (no.) 1,134 1,090 1,212 3,436 Attrition (no.) 97 111 160 368 Attrition (percent) 7.9 9.2 11.7 9.7 Source: FUSADES 2014. Note: C1 = individuals living in the same precarious urban settlements as T1 participants who were eligible to take part but did not register; C2 or C3 = individuals living in similar precarious urban settlements to those in T2 that are not covered by PATI who would be eligible to take part based on observable characteristics; PATI = Temporary Income Support Program; T1 or T2 = individuals iving in precarious urban settlements who registered to take part in the PATI program. Baseline Characteristics The study shows that for the second PATI round, 18,654 individuals registered to take part in PATI, and 16,053 were prioritized to receive treatment based on the scoring index carried out by FISDL. Of the 16,053, 71 percent were women and more than one-third (34 percent) youth (ages 16–24). The sample showed that people who register for PATI tend to be, on average, younger than those who do not register in the same precarious urban settlements. Generally, both treatment and comparison group have low levels of human capital, though it is slightly lower for individuals who registered to take part. Findings This section summarizes the findings on the average treatment effect on the treated for comparison 1 only using a difference-in-differences approach. The section presents results for the overall sample and by age, gender, and education level. Results reported are significant at a 10 percent confidence level. 46 Labor Market Outcomes The theory of change postulated that basic technical training (80 hours for round two) and the labor orientation training (16 hours) would increase PATI participants’ employability prospects. The study shows that only 76 percent of the participants mentioned receiving the basic technical training, and only 40 percent noted receiving the labor orientation training. FISDL and INSAFORP administrative data corroborate this suboptimal treatment uptake. Moreover, the study interviewed those who took the labor orientation training and found that very few of them could mention the steps that are needed to search for a job or start a business. Overall, very few could recall the training content, thus showing that the intervention did not work. The study found that the percentage of PATI participants who mentioned that they would elaborate a curriculum as one of the steps for finding a job increased from 17 percent in the baseline to 32 percent in the first follow-up. There were no gender differences in this regard, but youth (ages 16–24) mentioned this at a higher rate than adults. The main difference is when results are analyzed by age group: people with a higher level of education (ninth- grade or more education) mentioned this at a higher rate. Generally, very few interviewees noted that they would take part in job fairs. Perceived job search readiness. To assess whether the labor orientation training module was useful, the study asked participants to what extent they felt prepared for conducting a job search. The study found that receiving the labor orientation training module is associated with an increased perceived readiness of 6.6 percentage points. This positive impact is observed in women (8.4 percentage points), adults (9.2 percentage points), and individuals with less than ninth-grade (7.6 percentage points). The positive effects for these groups are observed only in the short term (first follow-up) and they disappear in the medium term (second follow-up). Perceived self-employment readiness. The study found that the labor orientation training module is associated with an increase in the perceived readiness to start self- employment by 5.6 percentage points in the short term. This positive impact is observed for women (5.6 percentage points), youth (ages 16–24) (10.3 percentage points), and individuals with a ninth-grade education or more (9.6 percentage points). These positive effects disappear in the medium term. Labor force participation. The study found that PATI participation is associated with increased labor market participation (either being employed or searching for a job) in the short term (4.6 percentage points) and medium term (5.5 percentage points). This increased labor force participation is observed in women (5.9 percentage points short term and 6.6 percentage points medium term), youth (9 percentage points short term and 8.4 percentage points medium term), and in individuals with a ninth-grade 47 education or more (9.0 percentage points short term and 10.9 percentage points medium term). Employment. The study found that PATI participation is associated with increased employment (either salaried, not salaried, or self-employment) in the medium term (5.5 percentage points). The study did not find short-term impact. The positive impact on employment in the medium term is observed for women (6 percentage points), youth (6.2 percentage points), and individuals with a ninth-grade education or more (8.5 percentage points). Poverty Outcomes Employment income. The study found that PATI participation is associated with an average $17 increase in individuals’ monthly employment income one year after program completion (second follow-up). This positive impact is observed for men ($56.70), adults ($27.40), and individuals with less than a ninth-grade education ($24.60). The study did not find impact on average monthly employment income for women and youth, and individuals with a ninth-grade education or more. Overall, PATI met its objective of protecting the income of poor individuals in urban areas, but it did not influence the groups (within the poor) that were a specific PATI priority: youth and women. Extreme poverty. The study found that PATI participation is associated with a decrease in extreme poverty. After a year of completion (second follow-up), the share of PATI participants who lived in households with extreme poverty decreased by 8 percentage points. This decrease is also observed for men (11.2 percentage points), youth (9.8 percentage points), adults (7.8 percentage points), and individuals with less than a ninth-grade education (9.4 percentage points). By contrast, the study did not find any impact for women and individuals with a ninth-grade education or more. Poverty. According to study results, PATI participation is not associated with a decrease in poverty. Specifically, after a year of completion, PATI participation did not have an impact on the share of participants who lived in households with poverty. Disaggregation by groups show an unexpected impact for women—the share of PATI female participants living in households with poverty increased by 4.7 percentage points after a year of completing the program. Empowerment Outcomes Self-esteem. The study found no impact on self-esteem measures. The study used the Rosenberg Self-Esteem Scale developed by Dr. Morris Rosenberg to measure self-esteem. 48 Mood. However, the study found a positive impact on mood measures. The study used the Center for Epidemiologic Studies Depression Scale developed by the U.S. National Institute of Mental Health for measuring mood. Overall, participation in the program is associated with a short-term improvement in mood measures. However, the positive effect disappears in the medium term. Disaggregation by groups showed that mood measures improve in the short term for women, adults, and individuals with less than a ninth-grade education. Household decision making. To measure improved empowerment in relation to household decision making, the study asked individuals whether they were responsible for making decisions alone regarding how to spend household income. The study did not find any short-term impact in this regard for the entire sample or for any of the studied groups. However, the study found that PATI participation is associated with an increase empowerment in the medium term for men (10.7 percentage points) and individuals with a ninth-grade education or more (10.5 percentage points). There is no medium-term impact for women and youth. The study also asked participants if they made decisions jointly with other household members and found that PATI participation is associated with an increase in this measure (6 percentage points) in the medium term. Disaggregation by groups shows medium-term improvements for men and individuals with a ninth-grade education or more. Violence Outcomes The theory of change postulated that civic participation, trust, and satisfaction with community life would increase through PATI participation, which would lead to a reduction in the levels of violence. Civic participation. The study found that the program is associated with a short-term participation increase in civil society organizations (3 percentage points). The effect is sustained in the medium term. The positive effect is observed in the short term for women and men (3.3 percentage points), youth (1.7 percentage points), adults (4.5 percentage points), and individuals with less than a ninth-grade education (3.9 percentage points). In the medium term, these positive effects disappeared for all groups except for individuals with less than a ninth-grade education (3.7 percentage points). Trust. To measure trust, the study asked individuals whether they felt that they could trust most of their community peers. Contrary to what was expected, the study found that trust diminished in the short term (5.5 percentage points) and medium term (6.9 percentage points). These decreasing effects are observed for women in the short 49 term (−5.2 percentage points) and medium term (−5.9 percentage points), for youth in the medium term (−5.7 percentage points), adults in the short term (−5.5 percentage points) and medium term (−6.1 pp), and for individuals with less than a ninth-grade education in the short term (−5.2 percentage points) and medium term (−6.1 percentage points). The study did not find any effect for men and individuals with a ninth-grade education or more. Satisfaction with community life. To assess satisfaction with community life, the study elaborated a composite indicator capturing individuals’ perceptions of community areas that contribute to peaceful coexistence, such as parks, leisure areas, street cleanliness, street safety, and city ornaments. The study found that PATI participation is associated with a small increase in satisfaction with community life in the short and medium terms (0.3 percentage points). The study observed this positive effect for men in the short term (0.6 percentage points) and medium term (0.7 percentage points), and youth in the short term (0.3 percentage points) and medium term (0.4 percentage points). Individuals with less than a ninth-grade education show only a positive effect in the short term (0.3 percentage points). Perceived safety from violence. The evaluation team considered a decrease in victimization rates an indirect result of the program. The hypothesis was that the program decreases victimization rates by decreasing youth risky behaviors. In addition, the study also hypothesized that money transfer could increase violence in the precarious urban settlements. For this reason, the evaluation assigned high priority to measuring perceived safety from violence and victimization rates. To measure perceived safety, the study asked individuals to rate how safe they felt community parks on a scale from 0 to 10 (with 0 meaning highly unsafe and 10 meaning highly safe). Overall, the study did not find any short- or medium-term positive impacts on perceived safety. In fact, a disaggregation by groups shows that perceived safety had small medium-term decreases for women (−0.6 percentage points), youth (−0.5 percentage points), and individuals with a ninth-grade education or more (−0.6 percentage points). Victimization rate. To measure victimization, the study asked individuals to report on whether they or somebody else in the household suffered from personal or property crime. The study found that PATI participation is associated with increased victimization in the medium term (4.4 percentage points). Disaggregation by groups shows that victimization increased in the medium term for men (9 percentage points), adults (5.1 percentage points), and individuals with a ninth-grade education or more (6 percentage points). 50 Appendix E. Qualitative Evaluation Temporary Income Support Program The evaluation’s main purpose was to explore the perceptions and opinions regarding the Temporary Income Support Program’s (PATI) impact on the agency and employability of participants to understand the main conclusions of the quantitative impact evaluation better. In social science, agency is the capacity of individuals to act independently and to make their own free choices. The evaluation sought the views of both PATI participants and nonparticipants and had specific areas of inquiry for each group. For participants, the evaluation gathered their views on the following six areas: (i) the program’s principal impacts, (ii) the longer-term impacts on employability and poverty reduction, (iii) the different program components’ influence and value-added on the participant’s employability and agency, (iv) the influence of the program and its components on the various elements of agency (self-esteem, self- efficacy, self-control, intrinsic motivations, meaning and purpose, depression, family dynamics, and decision making), (v) the influence of the program and its components on participants’ aspirations, mainly those related to their labor market participation in the short, medium, and long term, and (vi) the program’s influence on participants’ sense of social support, confidence, and sense of security in their communities. For people who did not take part in the PATI program, the evaluation sought their views and perceptions on the following six areas: (i) their knowledge of the program, including its components, the recruitment process, and the selection criteria; (ii) the program characteristics that made their participation less attractive; (iii) the influence of PATI and its components on beneficiaries’ labor market participation; (iv) their employment situation and its influence on their level of poverty; (v) their aspirations and the various elements of agency (self-esteem, self-efficacy, self-control, intrinsic motivations, meaning and purpose, depression, family dynamics, and decision making); and (vi) the program’s influence on participants’ sense of social support and their confidence and sense of security in their communities. In addition, the evaluation sought to explore the possible influence on program results of contextual factors, such as local conditions, leadership, formal and informal support networks, programs and strategies in the community that complemented or competed with PATI, and links with other programs, among others. Finally, the evaluation explored the perceptions and opinions about the evaluation’s assumptions with institutional stakeholders involved in the program’s implementation, such as personnel from the FISDL and municipalities, among others. 51 Methodology The evaluation did not seek to produce an analysis of magnitudes and frequencies but rather sought to understand the reasons behind the findings of the quantitative evaluation through the protagonists’ own expressions and perceptions. The qualitative evaluation used the findings of the quantitative evaluation as a starting point, becoming the study’s assumptions. The evaluation gathered views of participants and nonparticipants through a combination of qualitative techniques, such as semistructured interviews, focus groups, and life stories. The evaluation used semistructured interviews to gather the views of institutional stakeholders. Guides were prepared for all interviews and focus groups. In addition, the evaluation used the line of relationships establishment instrument that allowed the mapping of individuals’ labor profiles and community engagement characteristics before, during, and after PATI. Sampling Design The study used purposive sampling to identify individuals who would be interviewed or take part in the focus groups. Furthermore, the study determined profiles based on education levels, age, and gender, and whether the person took part in the program. A quota for each profile was established to gather information from the different types of PATI beneficiaries. The municipalities were selected based on three criteria: The perception of the local development advisers regarding program performance, the precarious urban settlement participation in recent PATI rounds (not just in the second round), and the identification of enough people in the municipality for the different participant profiles. Based on these criteria, three municipalities were selected: Acajutla, Ahuachapán, and Ciudad Arce. Additionally, four more municipalities were included to add more geographic variability: Jiquilisco, Nahuizalco, San Vicente, and Zacatecoluca. The individuals selected for interviews and focus groups were drawn from the respondents to the quantitative evaluation’s survey, within the municipalities of focus. The focus groups were formed based on the list of individuals that took part in that survey, taking care to group people in the municipalities based on the established profiles. The preliminary list of participants would include at least 15 people, aiming to have at least eight individuals per group. The focus group took place in a centric and recognizable place for individuals, and in some municipalities, the study offered to cover their transportation costs. Sixty-four interviews and six focus groups were completed (with the participation of 34 individuals). Table E.1. shows the sample distribution for all the profiles and the technique used. 52 Table E.1. Sample Distribution of Participants and Nonparticipants Women Men Participants Nonparticipants Participants Nonparticipants Ages Ages Ages 25– Ages Ages Ages Ages 25– Ages 16–24 25–60 60 16–24 16–24 25–60 60 16–24 Technique (Youth) (Adult) (Adult) (Youth) (Youth) (Adult) (Adult) (Youth) Semi- Education structured less than 4 4 4 4 4 4 4 4 interviews sixth grade Education greater 4 4 4 4 4 4 4 4 than ninth grade Focus Education groups less than sixth grade 4 10 6 3 5 6 Education greater than ninth grade Source: FUSADES 2015. As shown in table E.2, 16 individuals from the institutions involved in the implementation of PATI were interviewed. Three interviews were in person, and six interviews were conducted in groups. Table E.2. Sample of Institutional Stakeholders Interviewees Institution (no.) Social Investment Fund for Local Development 4 Municipalities 2 Ministry of Labor 3 Salvadoran Institute of Professional Training 7 Total 16 Source: FUSADES 2015. Findings Recruitment and Selection Process The evaluation found that young individuals were not interested in participating in PATI because they thought the monetary transfer was too low and did not cover their 53 needs. The study also confirmed that the individuals who were drawn to take part were those who needed the transfer the most and could not obtain income from another source. Moreover, they were expecting this support would be permanent. The transfer was what attracted people the most to the program, followed by the interest in learning a new trade and thus becoming able to attain employment. The program design did not have a counter-reference mechanism. For this reason, individuals who registered but were not selected to take part in PATI did not find out the reasons behind this decision or the selection criteria. Training Activities The technical training activities were considered too basic and of short duration. Moreover, they did not focus on the most relevel local employment opportunities. Considering that the duration of PATI was preestablished, the training did not cover the material in depth. The mismatch between the training and the local labor needs could have been caused by the lack of coordination between the municipalities and the Salvadoran Institute of Professional Training. This made participants enroll in training that might not have been of interest or relevance to them only because they needed the transfer. The evaluation found that people rarely enrolled in a training that was interesting to them. Moreover, participants who had low education levels (or were even illiterate) had difficulty understanding the training content. The study concluded that different teaching methodologies should be used for the different types of participants. Main Program Impacts The main program impacts were participants’ labor market participation and improvements in the community spaces. Both impacts were considered temporary, perceptible only in the months after their participation in the program. According to nonparticipants, PATI positively affected the personal development of individuals with fewer opportunities to join the labor market, such as the elderly and single women. They also believed that more comprehensive support to young participants could become an important opportunity for further development and labor market participation. Labor Market Participation and Income Neither the quantitative nor the qualitative evaluation found evidence to ascertain that women and youth improved their employment opportunities because of the program. The study found that even though participants had positive expectations for their prospects to join the labor market, they realized that they need formal education and experience besides vocational training if they wanted to get a job. Those who wanted to open a business 54 realized they also needed the required inputs and access to credit. For this reason, the perception of improvement, or rather the enthusiasm that they experienced when they finished the training, diminished with time. The evaluation found that even though there is the perception that receiving vocational training would create job opportunities for participants who had more schooling, there is no evidence that these individuals joined the labor market in the areas in which they received training through PATI. Regarding the program impacts on income, the evaluation found that it was not possible to conclude whether people thought PATI contributed to improvements in the labor income of youth, men, and individuals with higher education levels. The majority of young individuals interviewed considered their economic conditions the same or worse than they were two years before. For women, labor participation and the resulting income increase depended on whether they continued being the main caregivers in their households. People, especially youth, believed there were no job opportunities. The restrictions were not only because of lack of formal education and experience but also because of insecurity, which limited their free movement in the communities and beyond. This situation restricted their ability to respond to labor demand in other areas, which made it even more important to adapt the training to respond to job opportunities in their own territory. Agency The evaluation did not find that people improved their self-esteem or self-control because of the program. For women, there was no evidence that the training contributed to their empowerment. Women considered household chores and caregiving their sole responsibility and did not perceive the need to change this situation. Interpersonal and Community Relationships The study found that the small increase in beneficiaries’ participation in community spaces was motivated by their expectation that, by being more involved, they would have greater chances to participating in other PATI rounds as well. However, the participation does not reflect a sustained interest or conviction regarding the importance of taking part in these organizations. Moreover, there was no perceived change in community leadership motivated by PATI, as they had remained unchanged. Another finding was that PATI contributed to better relationships in the community, but only in an apparent and temporary way—participants’ cordiality with other community member did not evolve into more connections that are permanent. Moreover, people felt safer the least they interacted with their neighbors, which also discourages them from using public and recreational spaces. In addition, the improvements in public areas that resulted from PATI’s community subprojects were temporary because of the lack of maintenance after the project concluded. 55 Appendix F. List of Persons Met World Bank Oscar Avalle Country Manager, World Bank Jania Ibarra Operations Officer, World Bank Edmundo Murrugarra Senior Social Protection Economist, World Bank Nancy Banegas Social Protection Specialist, World Bank Claudia Rodriguez Alas Social Protection Specialist, World Bank Marcela Ines Salvador Sr Social Protection Specialist, World Bank Government Fanny Martínez Head of the Human Capital Development Department, Social Investment Fund for Local Development (FISDL) Melissa Alvarenga Head of the Registry and Transfer Department, FISDL Roberto Molina Office of Information and Response, FISDL Carolina Avalos Former President, FISDL Claudia Ramírez Youth with Everything Program Coordinator, National Institute for Youth. Former Territorial Coordinator, FISDL Suyapa Marroquín Initial Training Manager, Salvadoran Institute of Professional Training (INSAFORP) Carlos Gómez Executive Director, INSAFORP Lesly Cervellón Head of the National Employment Department, Ministry of Labor (MTPS) Angel Ernesto Córdova Coordinator of the Labor Market Observatory, MTPS César Pineda Head of the Employment Offices Section of San Salvador, MTPS Luis Mario Flores Guillén Head of the Institutional Financial Unit, MTPS Jesús Valencia Technical Specialist, Directorate of Management and Coordination of Social Policy, Technical and Planning Secretariat of the Presidency (STP) Clotilde Barrios Falcon de Santamaria Technical Specialist, Directorate of Monitoring and Evaluation, Technical and Planning Secretariat of the Presidency (STP) Juan Francisco Grande Technical Specialist, Directorate of Strategic Programs, Technical and Planning Secretariat of the Presidency (STP) Martha Evelyn González Technical Specialist, Directorate of Management and Coordination of Social Policy, Technical and Planning Secretariat of the Presidency (STP) Susana Lobato Technical Specialist, Directorate of Management and Coordination of Social Policy, Technical and Planning Secretariat of the Presidency (STP) Manuel Delgado Technical Specialist, General Directorate of Government Coordination and International Cooperation, Technical and Planning Secretariat of the Presidency (STP) 56 Rebeca Sánchez Technical Specialist, Directorate of Management and Coordination of Social Policy, Technical and Planning Secretariat of the Presidency (STP) Claudia Sandoval Technical Specialist, Directorate of Strategic Programs, Technical and Planning Secretariat of the Presidency (STP) Juan Meléndez Director of Strategic Programs, Technical and Planning Secretariat of the Presidency (STP) Aída Argüello Former Coordinator of Social Policy, Technical and Planning Secretariat of the Presidency (STP) Private Sector Jorge Roberto Guzmán Head of the Human Resources Department, Industrias Plásticas S. A de C. V Fabiola Estefani Contreras Head of Talent Selection, Grupo Lorena Mario Magaña Director of Economic Affairs, Salvadoran Chamber of Commerce Other Organizations Alicia Louro Cooperation Section of the European Union Delegation to El Salvador Natalia Otamendi Vallet Responsible of Cooperation Programs, Fight against Poverty, Sustainable Development and Humanitarian Action, Spanish Cooperation Agency (AECID) Jimmy Vásquez Social Policy Specialist, United Nations Children’s Fund (UNICEF) Margarita Beneke de Sanfeliú Director of the Research and Statistics Center, Salvadoran Foundation for Economic and Social Development (FUSADES) 57