Document of The World Bank Report No: ICR00001470 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73470) ON A LOAN IN THE AMOUNT OF US$240 MILLION TO THE UNITED MEXICAN STATES IN SUPPORT OF THE FIRST PHASE OF THE SCHOOL-BASED MANAGEMENT PROJECT June 28, 2010 Human Development Sector Management Unit Colombia and Mexico Country Management Unit Latin America and the Caribbean Regional Office CURRENCY EQUIVALENTS (Exchange Rate Effective 06/18/2010) Currency Unit = Peso (MXN$) MXN$1.00 = 0.08 US$ 1.00 US$ = 12.58 MXN$1.00 FISCAL YEAR January 1 ­ December 31 ABBREVIATIONS AND ACRONYMS AEE State Education Authorities (Autoridades Educativas Estatales) AGEB Basic Geo-Statistical Area (Área Geo-Estadística Básica) of CONAPO ANMEB National Agreement for Basic Education Modernization (Acuerdo Nacional para la Modernización de la Educación Básica) APL Adaptable Program Loan CAS Country Assistance Strategy CDI National Commission for the Development of Indigenous Peoples (Comisión Nacional para el Desarrollo de los Pueblos Indígenas) CEPS School Councils (Consejos Escolares de Participación Social) CEPSE State Education Council for Social Participation (Consejo Estatal de Participación Social en la Educación CGEIB General Coordination for Intercultural and Bilingual Education (Coordinación General para la Educación Intercultural Bilingüe) CGEPEC State Coordination of the PEC Program (Coordinación General Estatal del PEC CNPEC National Coordination of the PEC Program (Coordinación Nacional del PEC) at SEP CONAFE National Council for Educational Development (Consejo Nacional de Fomento Educativo) of SEP CONALITEG National Commission for Free Textbooks (Comisión Nacional de Libros de Texto Guatuitos) CONAPO National Population Council (Consejo Nacional de Población) DGAPRF General Directorate for Budget and Financial Resource Administration (Dirección General de Administración Presupuestal y Recursos Financieros) at SEP DGE General Directorate for Evaluation (Dirección General de Evaluación) at SEP DGEI General Directorate for Indigenous Education (Dirección General de Educación Indigena) at SEP DGPPP General Directorate for Planning, Programming, and Budgeting (Dirección General de Planeación, Programación y Presupuesto) at SEP DGRMS General Directorate for Material and Service Resources (Dirección General de Recursos Materiales y Servicios) at SEP DGTI General Directorate for Information Technology (Dirección General de Tecnología de la Información) at SEP FAEB Contribution Fund for Basic Education (Fondo de Aportaciones para la Educación Básica) FEEC State Trust Fund for the PEC (Fideicomiso Nacional para Escuelas de Calidad) FISM Fund for Social Municipal Infrastructure (Fondo de Infraestructura Social Municipal) FM Financial Management (Gestión Financiera) FMR Financial Monitoring Report (Informe de Monitoreo Financiero) FNEC National Trust Fund for the PEC (Fideicomiso Nacional para Escuelas de Calidad) IADB Inter-American Development Bank INEE National Institute for Education Evaluation (Instituto Nacional para la Evaluación Educativa) INEGI National Institute of Statistics, Geography and Informatics (Instituto Nacional de Estadística, Geografía e Informática) IPDP Indigenous Peoples Development Plan LGE General Education Law (Ley General de Educación) MET Technical Memorandum of Understanding on Audit (Memorando Técnico de Entendimiento sobre Auditorías) MIS Management Information System (Memorando Técnico de Entndimiento sobre Auditorías) NAFIN National Financing Agency (Nacional Financiera, S.N.C.) NGO Non-Governmental Organization OECD Organization for Economic Cooperation and Development OM Operational Manual PAD Project Appraisal Document PARE Primary Education Project (Programa para Abatir el Rezago Educativo) (Loan 3407-ME, 1991) PAREB Second Primary Education Project (Programa para Abatir el Rezago en Educación Básica) (Loan 4722-ME, 1994) PAREIB-I Basic Education Development Project, APL I (Programa para Abatir el Rezago en Educación Inicial y Básica) (Loan 4333-ME, 1998) PAREIB-II Basic Education Development Project, APL II (Programa para Abatir el Rezago en Educación Inicial y Básica) (Loan 7108-ME-ME, 1998) PAREIB-III Basic Education Development Project, APL III (Programa para Abatir el Rezago en Educación Inicial y Básica) PAT Annual Work Program (Programa Anual de Trabajo) of PEC schools PEC Quality Schools Program (Programa Escuelas de Calidad) PETE Strategic School Transformation Plan (Plan Estratégico de Transformación Escolar) of PEC schools PIARE Integral Program of Education (Programa Integral para Reducir el Rezago Educativo) financed by IADB PNE National Education Program 2001-2006 (Programa Nacional de Educación 2001-2006) PRODEI Initial Education Project (Proyecto para el Desarrollo de la Educación Inicial) (Loan 3518-ME, 1992) PROSSE Program of Essential Social Services (Programa de Servicios Sociales Esenciales) (Loan 3913-ME) SBM School-Based Management SEB Under-Secretariat of Basic Education (Subsecretaría de Educación Básica) at SEP SEGOB Secretariat of Internal Affairs (Secretaría de Gobernación) SEP Secretariat of Public Education (Secretaría de Educación Pública) SFP Secretariat for Administrative Development (Secretaría de la Función Pública) SHCP Secretariat of Finance and Public Credit (Secretaría de Hacienda y Crédito Público) SIPEC Information System for the Quality Schools Program (Sistema de Información del Programa Escuelas de Calidad) SOE Statement of Expenditure SOSEDF Under-Secretariat of Educational Services for the Federal District (Subsecretaría de Operación de los Servicios Educativos para el Distrito Federal) at SEP Telesecundaria Television-Based Lower Secondary Education Program TESOFE National Treasury Under-Secretariat (Tesorería de la Federación) at SHCP Vice President: Pamela Cox Country Director: Gloria Grandolini Sector Manager: Chingboon Lee Project Team Leader: Ricardo Silveira/Erik Bloom ICR Primary Author: Suzana de Campos Abbott MEXICO School-Based Management Project, First Phase CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 3. Assessment of Outcomes .......................................................................................... 14 4. Assessment of Risk to Development Outcome......................................................... 18 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned........................................................................................................ 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........... 23 No additional comments received. Annex 1. Project Costs and Financing .................. 24 Annex 1. Project Costs and Financing.......................................................................... 25 Annex 2. Outputs by Component.................................................................................. 26 Annex 4. Beneficiary Survey Results ........................................................................... 31 Annex 5. Stakeholder Workshop Report and Results................................................... 32 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 33 Informal translation of Borrower's comments (Letter dated June 15, 2010) ............... 33 Annex 7. List of Supporting Documents ..................................................................... 37 A. Basic Information Mexico Education Country: Mexico Project Name: Quality Project ID: P088728 L/C/TF Number(s): IBRD-73470 ICR Date: 06/29/2010 ICR Type: Core ICR SECRETARIAT OF Lending Instrument: APL Borrower: PUBLIC EDUCATION Original Total USD 240.0M Disbursed Amount: USD 240.0M Commitment: Revised Amount: USD 240.0M Environmental Category: C Implementing Agencies: Secretariat of Public Education Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 03/16/2005 Effectiveness: 11/17/2006 11/17/2006 Appraisal: 10/13/2005 Restructuring(s): Approval: 12/13/2005 Mid-term Review: 07/31/2007 09/13/2007 Closing: 07/31/2009 12/31/2009 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 2 2 Pre-primary education 18 18 Primary education 40 40 Secondary education 40 40 Theme Code (as % of total Bank financing) Access to urban services and housing 20 Education for all 40 50 Managing for development results 20 20 Participation and civic engagement 20 30 E. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox Pamela Cox Country Director: Gloria M. Grandolini David N. Sislen Sector Manager: Chingboon Lee Evangeline Javier Project Team Leader: Ricardo Rocha Silveira Harry Anthony Patrinos ICR Team Leader: Ricardo Rocha Silveira ICR Primary Author: Suzana Nagele de Campos Abbott F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The proposed project aims to improve public school management through strengthening and expanding the Government's Quality Schools Program (Programa Escuelas de Calidad, PEC) now being implemented on a pilot scale by SEP. PEC is designed to ii empower the school community, and promote school autonomy and accountability, encouraging parents, teachers and directors to design and carryout school improvement plans that respond to the needs of the school and its students. The PEC awards a grant to finance the improvement plans in selected schools; these grants are jointly financed by SEP and state governments. The program is open to all public basic education schools that apply on a voluntarily basis, but priority is given to disadvantaged schools located in poor urban areas, as these schools have the most need and also tend to benefit more from the program. The federal investment in the program leverages approximately half as much in state contributions and in donations from local school communities, where schools mobilize parents, municipal governments and private organizations to support their school improvement plans. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years (a) Number of schools participating in PEC as percentage of the total number of Indicator 1 : basic education schools Value quantitative or 10.3% (2005/2006) 15.8% 20.35% Qualitative) Date achieved 11/14/2005 07/31/2009 12/30/2009 Comments (incl. % achievement) Indicator 2 : Number of basic education schools participating in PEC Value quantitative or 21,500 (2005/2006) 33,087 40,790 Qualitative) Date achieved 11/14/2005 07/31/2009 12/30/2009 Comments (incl. % achievement) Commitment to the goals and activities of the PETE as expressed by local Indicator 3 : financial contributions to the schools (Mexican pesos/school) Value quantitative or 30,168 34,968 8,170 Qualitative) Date achieved 11/14/2005 07/31/2009 12/10/2009 Comments As explained in the text, this target was not met due to PEC's focus on (incl. % incorporating poorer and smaller schools. This target will be reformulated for achievement) PEC II, taking into account PEC's strategy of focusing on the poorest schools. iii (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Percentage of community members that know PETE Value (quantitative 62 71 76.8% or Qualitative) Date achieved 11/14/2005 07/31/2009 12/10/2009 Comments (incl. % achievement) Percentage of community members that observe participatory decision-making Indicator 2 : between parents, teachers and principal about school affairs Value (quantitative 82 87 90.2% or Qualitative) Date achieved 11/14/2005 07/31/2009 12/10/2009 Comments (incl. % achievement) Percentage of community members that observe teachers encouraging and Indicator 3 : supporting student performance Value (quantitative 95 95 66.9 or Qualitative) Date achieved 11/14/2005 07/31/2009 12/10/2009 Comments There was a cahnge in definition since the baseline was established, leading to a (incl. % more conservative estimate of community observation of teachers. The new achievement) Indicator try to ensure the actual observation of positive practices by teachers. Number of PEC schools located in areas of medium, high and very high Indicator 4 : marginalization levels (Total) medium 3559 Total 15,068 Value high 5092 Target established Medium 5,153 (quantitative very high 799 at appraisal High 4,635 or Qualitative) Total 9450 Very High 5,280 Date achieved 11/14/2005 07/31/2009 12/10/2009 Comments (incl. % achievement) Percentage of states that report on time the information of the Program through Indicator 5 : SIPEC Value Base line was not 80% 75% (quantitative avaiable at appraisal iv or Qualitative) Date achieved 11/14/2005 07/31/2009 12/31/2009 Comments As explained in the text, PEC is changing its MIS requirements and the final (incl. % indicator reflects this, including states using SIPEC and acceptable, comparable achievement) systems. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/29/2005 Satisfactory Satisfactory 0.00 2 05/05/2006 Satisfactory Satisfactory 0.00 3 10/31/2006 Satisfactory Satisfactory 0.00 Moderately 4 05/12/2007 Satisfactory 115.16 Unsatisfactory Moderately 5 12/03/2007 Satisfactory 126.42 Unsatisfactory 6 04/27/2008 Moderately Satisfactory Moderately Satisfactory 180.30 7 12/14/2008 Moderately Satisfactory Moderately Satisfactory 207.42 8 05/16/2009 Moderately Satisfactory Moderately Satisfactory 235.23 9 12/04/2009 Moderately Satisfactory Moderately Satisfactory 237.79 10 02/19/2010 Moderately Satisfactory Satisfactory 240.00 H. Restructuring (if any) Not Applicable I. Disbursement Profile v 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal By the 2005 appraisal of the School-Based Management Project (the Project) in support of the First Phase of the School-Based Management Program (the Program), Mexico had made significant progress in expanding access to education. By 2005, 93 percent of school-aged children were attending primary school, with a completion rate of 89 percent. Despite this progress, however, universal coverage remained a challenge, particularly in rural areas, indigenous communities, and among migrant students where dropout and repetition rates remained high. Overall, the main issues facing Mexico's education system were: (i) inequality in access to learning; (ii) low access to preschool, lower secondary, and upper secondary education; (iii) low internal efficiency; and (iv) low overall learning achievement levels throughout the education system. In contrast to the high enrollments at the primary level, net enrollment in upper secondary education was only 53 percent in 2004, which is particularly low compared to similar economies in Latin America, such as Argentina and Chile. The quality of education was a particular concern. The overall learning achievement levels were low. School quality had not kept pace with enrollment increases and many children, especially poor children, were educated in low quality schools. In 2006, Mexico's results on the international PISA assessment showed that one percent of 15 year olds were at the highest level of performance in Reading and Math, compared to an OECD average of 7.6 percent. 21 percent of Mexican students were below Level One-- the lowest level of performance--in Spanish, and 28 percent were below Level One in Math. These last two results were significantly worse than the corresponding OECD average of only seven and eight percent, respectively. Mexico's 2001 National Education Development Program proposed initiatives at all levels of government to address issues of education equity and quality. Analysis in Bank sector work identified school autonomy, accountability and assessment as factors that would help improve the quality of education in Mexico.1 In 2001, the Government introduced the Programa Escuelas de Calidad (Quality Schools Program, PEC), a school-based management program that aimed to promote community participation in schools. PEC was the first federal program in Mexico aimed at transforming schools by making them the center of decision making. Stakeholders from the school community included school directors, teachers, parents, and community members that form the School Council of Social Participation. The council designed a School Transformation Strategic Plan (Plan Estratégico de Transformación Escolar, PETE) and an Annual Work Program (Programa Anual de Trabajo, PAT). The PETE was then submitted to state officials, who would evaluate the plan and the accepted ones 1 Garcia, V., Knaul, F. and Patrinos, H.A., Returns to Education and Quality of Education in Mexico, 2005, World Bank (Processed) 1 would receive resources for up to five years (renewable each year, based on satisfactory performance). The Federal Government funded 75 percent of the total cost of the Program, while states financed the remaining 25 percent. Schools could raise further resources from the school community, including Non-Governmental Organizations and the private sector. Additional PEC rules required that the school community be involved in carrying out the PETE. Parents have a direct participation as they are custodians of the funds and must verify the purchases and contracts made using PEC resources. Participation by the schools in PEC was voluntary. States selected schools to participate in PEC following Federal Government rules that required a competitive review of the PETEs, while prioritizing disadvantaged schools, particularly indigenous schools. PEC had been operating for about four years when the Mexican Government in 2005 requested the Bank to provide support to the Program to help improve learning outcomes. Support for the Project was fully consistent with the objectives of the Bank's Country Partnership Strategy (CAS) for Mexico because PEC contributed to improving the quality of basic education services, ensuring the sustainability of the ongoing efforts in basic education, and, at a higher level, reducing poverty in the longer run. Improving the quality of education was seen in the CAS as critical for Mexico to become competitive in the knowledge economy. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Project aimed to improve public school management through strengthening and expanding the Secretariat of Public Education's (Secretaría de Educación Pública, SEP) PEC Program. PEC was designed to empower the school community, and promote school autonomy and accountability. The long-term development objective of the three-phase APL for the School-Based Management Program was to improve the quality of education as measured by coverage, social participation and educational outcomes, as follows: ¾ Program schools as a percentage of all public basic education schools; ¾ Commitment to the goals and activities of the PETEs or its state equivalent as expressed by local financial contributions to the schools; and ¾ Average test scores in Math and Spanish for students in participating PEC schools. This indicator was to be reported by type of school (primary, secondary) and would be evaluated during the Third Phase of the Project. The First Phase of the APL was to support fine-tuning of the model and lead to increased levels of participation and school autonomy, and a modest expansion with robust accountability measures and increased use of assessments. The First Phase was to support the federal share of the Program and consist of: (i) school grants; (ii) monitoring and Program oversight; and (iii) policy development and evaluation, including specific interventions to encourage the participation of schools serving the most disadvantaged urban areas where the Program was expected to be expanded in subsequent Phases. 2 Triggers for moving forward with preparation of the Second Phase, which were similar to the outcome indicators in the Project Appraisal Document's (PAD's) Results Framework, included: ¾ Number of schools participating in the Program (including schools that graduate and no longer receive financial support) will increase by 50 percent; ¾ Program schools (including schools that graduate) as a percentage of all public basic education schools will increase to 15 percent; ¾ Completion of baseline data collection for impact evaluation; and ¾ Government approval of the Program expansion plan with specific targets for expansion. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDOs and Key Indicators were not revised during implementation. 1.4 Main Beneficiaries The Project's main beneficiaries were primary and lower secondary school-aged children enrolled in disadvantaged areas and communities. At the time of appraisal, about 55 percent of the schools in PEC were located in small towns with less than 15,000 people, and 15 percent were located in medium-sized cities with less than 100,000 inhabitants. Focusing on smaller communities was expected to facilitate participation and increase oversight. Only 4.5 percent of PEC schools were located in urban areas classified as highly marginal. Since participation in PEC was voluntary, schools with higher human and social capital were seen to be in a better position to participate in the Program. Participation would likely be considerably lower in schools in marginal urban neighborhoods since the level of parents' education in those areas tended to be lower. To deal with this issue, a compensatory promotion strategy was to be included in the Project. The selection criteria for participation in PEC were: (i) preference was to be given to schools that had participated in the Program in prior years---up to a maximum of five years---and continued to fulfill Program requirements; (ii) among schools applying to the Program for the first time, preference was to be given to qualified schools located in urban areas (AGEBs) of medium, high and very high marginality; and (iii) further, a state could use additional school priority selection criteria, as long as those criteria were applied uniformly to all public basic education schools in the state and did not contradict the PEC National Operating Rules. 1.5 Original Components (as approved) The Project comprised three components, as described below. 3 Component 1: School Grants (US$324.7 million including contingencies, equivalent to 94 percent of project costs; US$234.49 Bank financing) This Component would finance the federal share of the PEC school grant program that was transferred each year by SEP from the PEC National Trust Fund to the PEC State Trust Funds of participating states. The states would complement federal funds in accordance with a financing-share schedule and would be responsible for the Program's implementation in their jurisdictions. At the local level, the Program would be carried out by the State Education Authorities (AEEs) and the Sub-Secretariat of Educational Services for the Federal District (SOSEDF). The AEEs would transfer PEC grants from the State Trust Funds directly to the bank accounts of the participating schools, after completing a process of Program dissemination, submission of applications by interested schools, technical assistance to schools for preparation of grant proposals, qualification of proposals, and selection of schools for grant award. This process was to be repeated each year throughout the country. To qualify for the Program, the schools would need to establish participatory school management and present satisfactory PETE proposals. States would give priority to qualified schools in low-income urban neighborhoods. Under the Project, an estimated 33,000 public schools were expected to benefit from one or more of the following types of PEC grants: (a) Initial PEC grant, renewable for a maximum of five years. An initial grant was to be awarded to all schools selected (from the schools that voluntarily chose to participate and submitted plans) to participate in the Program in a given year, and could be renewed for a maximum of five years. Each year, the schools that wished to continue in the Program would need to re-apply for the grant, meet the selection criteria, and be chosen by the state selection committee to receive a grant. After the five-year cycle, the schools would graduate from the Program. The amount of this grant would vary according to the requirement of each PETE, but would not exceed MXN$50,000 (US$4,600 at the time of appraisal) per year, per school. The AEEs were to transfer the grant funds to the bank accounts of the schools in a single installment in the first half of the school year, before the end of the fourth month of classes. (b) Supplemental matching grant, renewable for a maximum of five years. A supplemental matching grant was to be awarded to PEC schools that were implementing PETEs with funds from the initial PEC grant, but that had also mobilized additional local resources for their plan. The PEC schools eligible for this type of grant would be those that sought and obtained donations in cash and/or in kind from parents, municipal governments, civil organizations and/or private enterprises. The supplemental grant would match the amount of local resources raised by the school, but was not to exceed MXN$50,000 per year, per school. Schools could apply and receive additional supplemental grants during the five-year Program cycle, provided they continued to meet the eligibility criteria. The AEEs were to transfer the supplemental grant 4 funds to the bank accounts of the schools in the second half of the school year, before the end of the ninth month of classes; and (c) Special matching grant, with no renewal restrictions. This special type of grant was to be targeted to disadvantaged basic education public schools that wanted to continue to carry out their PETE after graduating from the five-year PEC Program cycle, but required financial support to do so. To be eligible, a school needed to meet the following criteria: (i) be located in lower-income urban neighborhoods; (ii) be graduated from the regular PEC Program; (iii) meet Program requirements; and (iv) have raised additional local resources in support of their PETE. The PEC Program's Operating Rules (Reglas de Operación) were published by SEP every year since PEC's inception; they specify the amount to be invested in the Program each year by the Federal Government and the states, and confirm the criteria for the selection of schools and the arrangements for the Program's implementation at the national and state levels. These rules are an official annex to the Project's Operations Manual. Rules are changed on an annual basis, which does not necessarily led to a change in the Operations Manual. During PEC's implementation, the AEEs could introduce additional procedures to operate the Program in their jurisdiction, provided they complied fully with the Program's Operating Rules. This Component also financed the banking needs for the administration of the PEC federal trust fund through counterpart funding. Component 2: Program Monitoring and Oversight (US$15.1 million including contingencies, equivalent to 4 percent of project costs; no Bank financing) This Component would finance Program monitoring, oversight and information dissemination by SEP with the objective of ensuring compliance with Program Operating rules throughout the country and providing ample information to the public about the Program's operations and results. It was to be carried out through three Subcomponents. Subcomponent 2.1: Program monitoring would finance the continuous operation, maintenance and upgrading of the PEC national management information system (Sistema de Información del Programa Escuelas de Calidad---SIPEC). The SIPEC was operational, and online at appraisal, and would be continuously updated. It serves to: (i) record and systematize information on the operation of PEC at school, state and federal levels; (ii) record key Program outcomes, such as the use schools gave to the grant funds and the local funds mobilized; and (iii) flag shortcomings that needed to be addressed by state agencies for the Program's efficient operation. SIPEC would be enhanced through the design, test and operation of a customized user support module that would be made available to participating states and schools, enabling them to maintain the appropriate flow of information to the national office. The institutional management of the SIPEC would also be improved, especially regarding the linkage between the flagging of a problem and the corresponding remedial action. This Subcomponent would finance 5 consultants' services for system development. SEP would finance the procurement of additional computer equipment with other sources of funds. Subcomponent 2.2: Support for program implementation would assist the PEC national coordination office in providing technical and operational support to the AEEs to ensure smooth Program implementation in accordance with its Operating Rules. Regular SEP staff assigned to PEC by the Under-Secretariat of Basic Education would be used to carry out this Subcomponent. SEP's team consisted of experts in program operations and basic education that would carry out the following activities: (i) visit each state several times a year following an intensive supervision program; (ii) organize seminars and workshops to promote the continuous development of technical staff working on the Program at the state level; and (iii) coordinate regional and national meetings and the PEC annual Congress, attended by federal and state authorities. Through these activities, Program innovations would be discussed, best practices shared, and the Program Operating Rules updated. Funds allocated to this Subcomponent were to be funded entirely by the Government to finance travel, workshops, meetings, including hotel accommodations, meals, speakers, facilitator and materials. Technical support to schools for the preparation of their PETEs was not included under this Subcomponent since this assistance was to be provided by the AEEs and funded by a 20 percent share of state funds deposited in the PEC state trust funds. Subcomponent 2.3: Support for Program dissemination would assist to disseminate information on Program objectives, activities and results with the aim of promoting the Program and guaranteeing full accountability and transparency. Television, radio, newspapers and special publications would be used throughout the year to disseminate messages tailored to specific audiences of parents, teachers, directors, AEEs' technical staff and society at large. The National Free Textbooks Commission (Comisión Nacional de Libros de Textos Gratuitos-CONALITEG) would continue to publish a Program magazine, Educare, and distribute it three times per year across Mexico. SEP would continue collaborating with the Secretariat of Internal Affairs (Secretaría de Gobernación--SEGOB) to gain access to the official television network and other media for PEC's messages. This Subcomponent would finance the production of television, radio and newspaper messages, the design and editing of periodicals and pamphlets and the printing and distribution of materials, all with Government funding. Component 3: Policy Development and Evaluation (US$5.8 million including contingencies, equivalent to 2 percent of project costs; US$4.91 Bank financing) This Component would finance the development of basic education policy based on the results of policy studies, Program evaluation activities and pilot experiments designed to test possible future adjustments to the Program's design and financing. It was to be carried out through two Subcomponents. Subcomponent 3.1: Program evaluation would support the design and implementation of program evaluation activities that were to be carried out throughout the APL Program's implementation in order to create a strong analytical basis for its further 6 development. The following types of evaluation activities would be financed over the three phases of the APL Program: (i) external evaluation of the Program's performance and preparation of the Annual Report to the National Congress; (ii) qualitative evaluation to determine the non-quantifiable benefits accruing from the Program; (iii) student learning achievements using National Standards, and (iv) impact evaluation by an external, independent evaluation using rigorous methodology. Under the Project (i.e., the first phase of the APL Program), impact evaluation activities were to comprise: (i) the design and field test of evaluation methodology and data collection instruments; (ii) a baseline survey, and (iii) preliminary data analysis. Repeated surveys to measure the impact of the Program and further analysis and dissemination of results would be financed under subsequent phases of the APL Program. This Subcomponent would finance consultants' services provided by individuals, firms and national academic institutions. Subcomponent 3.2: Policy studies would support the development of national basic education policy, including the lower-secondary education reform that was currently in progress at the time of appraisal, through the analysis of national and international assessments of student learning achievement, evaluations of the Program's outcomes, and special studies, all addressing the following themes, among others: · Development and pilot experiment of strategies to encourage applications to participate in PEC from schools located in lower-income urban areas, and to identify the best way to provide technical assistance to help those schools prepare PETEs; · Adjustments to the PEC design and financing aimed to strengthen its impact and expand its coverage; and · PEC expansion plan for Phases II and III of the proposed APL Program. 1.6 Revised Components The components were not revised during implementation 1.7 Other significant changes The loan's July 31, 2009 closing date was extended by five months to December 31, 2009 on April 6, 2009. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Soundness of the Background Analysis. The Program's objectives fit well with the Bank's analysis of what is required to improve learning outcomes, and with lessons learned from similar school-based management programs mostly in Latin America, but in other regions as well. The Project's preparation followed extensive, programmatic sector work on the quality of education in Mexico that had identified school autonomy and school 7 environment as important factors that affected learning outcomes. It also built on regional sector work that showed the importance of schools in "producing" education quality and highlighted the opportunity to use participation and assessments to improve quality2. The review of World Bank projects supporting similar programs at the time showed that those projects often did not have the improvement of learning outcomes as their primary goal, and therefore, did not include indicators to measure those outcomes as it was difficult to see any changes during the life span of a project. Still, the review did show that the grant mechanism was an enabling instrument for creating conditions that could lead to a responsive education system, acting as a catalyst for change and empowerment. There have been few concrete evaluations of the impact of school-based management on learning outcomes. Perhaps the best-known school-based management program at the time, the EDUCO program in El Salvador, which the Bank had been supporting for several years, had been evaluated in terms of learning outcomes as well as participation. That project's impact evaluation found that greater participation by parents in schools, as measured by the number of visits to classrooms by a parent association member, was significantly and positively correlated with higher test scores in math and language. PEC authorities are committed to carry out an impact evaluation of the program during Phase II. A baseline study was conducted during Phase I. Assessment of the Project's Design. The Project's design was responsive to the needs of a program like PEC. Its technical design drew on several years of experience in the implementation of PEC and other SEP programs. There was extensive technical work in Mexico that guided the Project's design that focused on: (i) improving the quality of schooling; (ii) setting internal and international benchmarks on key indicators; (iii) evaluating the effectiveness of public programs designed to improve school quality; and (iv) designing improvements to policies and programs. Based on this analysis, a bottom- up approach that empowered the school community was seen as the best option. The Program also incorporated baseline and follow-up data for qualitative and quantitative analysis of operational processes and outcomes. The choice of an Adaptable Program Loan (APL) instrument was appropriate for a program requiring sustained long-term investments that introduced innovations in the way education services were delivered. The APL instrument was seen as providing flexibility to permit adjustment to the Program's design and financing over time, as country conditions changed and partnerships evolved. Also an APL responded to the need to incorporate lessons of experience through a process of learning-by-doing, in which the needs of different school communities and the realities of working in a federal structure could be addressed. The Project (Phase I of the APL) was seen as fine-tuning the model and leading to increased levels of participation and school autonomy, with robust accountability measures and increased use of assessments and impact evaluations. Phase II would consolidate gains and make appropriate improvements to the model and 2 World Bank (2005). Determinants of Learning Policy Note. Latin America and the Caribbean Region, Human Development Department. 8 expand to more disadvantaged schools, with the goal of seeing improvements in intermediate outcomes such as retention and dropout rates. Phase III would lead to improvements in learning outcomes. The Project's objectives, intermediate outcome indicators and triggers for moving ahead to Phase II of the APL were succinctly defined. Based on the experience of similar projects, the Program's outcomes in terms of learning achievements in Math and Spanish would only be evaluated during Phase III of the APL Adequacy of the Government's Commitment. The PEC Program had been in place since 2001, and the Government sought Bank involvement to help consolidate the gains that had by then so far been achieved, and to expand the Program while contributing to its long-term sustainability. The Government requested Bank support after four years of the Program's operation, when concerns about its impact and sustainability surfaced. Building upon the Bank's earlier support to Mexico's education sector, the Government felt that the Bank's presence would consolidate gains and provide crucial support to expand the Program while strengthening its fiduciary norms. The Government's commitment to the Program's and, consequently, the Project's objectives was strong. The Government's commitment was expressed in a Policy Letter on the Program that was submitted to the Bank during preparation. Assessment of the Risks. The Project's assessment of risks was extensive, and identified the overall risk rating of low. Several key risks were not identified and subsequently materialized. First, the disbursement of grants under Component 1 was much faster than the technical assistance under Components 2 and 3. Second, some states had difficulties in adopting and utilizing SIPEC. Consultative Processes. The social assessment carried out during project preparation involved a stakeholder analysis focused on capturing: (i) the Program's existing and potential beneficiaries; (ii) actual experiences of beneficiaries and other actors involved in the Program; (iii) expectations held by the relevant stakeholders; (iv) levels of social capital and potential for participation; and (v) impact of the Program on the lives of participants. The analysis was based on interviews with the Program's coordinators at the state level and with a sample of school directors, complemented by several existing studies. It provided specific recommendations that were addressed in the Project's design, especially with respect to the need for targeted communications and information dissemination and strategies to incorporate broader community participation in the definition of specific school priorities. 2.2 Implementation The Bank's loan for the Project was approved by the Board on December 13, 2005, signed on April 7, 2006 and became effective on November 17, 2006. A Mid-Term Review was held on September 13, 2007. The following issues impacted the Project's implementation: Project Design. As mentioned in Section 2.1, the Project's design was very responsive to the needs of the Program. One issue that surfaced was the rapid disbursement of Bank 9 finance for school grants (under Component 1) that outpaced PEC's capacity to monitor and evaluate the Project's progress, which was supported by technical strengthening under other components. This was not expected given delays in effectiveness and the provision for US$24 million in retroactive financing, almost immediately after effectiveness, by mid-2007, nearly half of the loan was disbursed. At that time, since the Project was new, the monitoring of output indicators was limited and the financial information system provided by SEP did not provide comfort to proceed with disbursement of grants. In response, SHCP informally suspended disbursements against school grants to allow PEC to strengthen its financial management, through an Action Plan. Institutional Capacity of Federal Entities and Schools. As any program implemented in a large, federal country like Mexico, the institutional capacity of the decentralized implementing agencies, in this case 32 AEEs and thousands of schools, varies greatly. PEC originally adopted an almost "cookie-cutter" approach that required undifferentiated application of the Program's systems and processes across all levels. In the roll out and adoption of SIPEC to all the AEEs, it soon became apparent that their capacities varied greatly and there would be a need to adjust the one-size-fits-all approach. This would allow those states with limited capacity to respond to the objective of providing reliable financial and monitoring information through their own systems. Financial Management and Monitoring. Because of the varying institutional capacity issues described above, the Project faced issues with the timely receipt of financial and other information. This was eventually addressed by allowing them to provide this information through their existing or new information systems, and at the same time developing an accreditation system for state-level MIS that would ensure minimum requirements were met with respect to school-based reporting. Social Participation. The success of the school-based management model depends on mobilizing community and parental support for the schools' programs. As the Project was implemented, it became clear that social participation lagged behind original expectations, in part because of the complexity of the PETE and the lack of understanding of the process by stakeholders. This issue is being addressed in the context of Phase II of the APL (Section 2.5). Supervision Missions. During the implementation of Phase I, the World Bank carried out eight supervision missions in order to monitor the advances in project's implementation and the challenges it was facing. This was crucial to identify areas in which improvements were possible. The Government was open to these missions and worked closely with the Bank in improving and adjusting the project. During the missions, the Bank visited schools and state authorities in different parts of the country (Chihuahua, Colima, Guanajuato Mexico, Mexico City, Morelos, Oaxaca, Puebla, Tamaulipas, and Zacatecas). 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 10 M&E Design. In the Project's design, monitoring and evaluation were seen as an integral element to inform policy so that objectives and targets could be adjusted in light of evidence on their effectiveness. The Project's Component 2 and 3 included technical assistance activities that were designed to strengthen the PEC's national management information system, SIPEC, and the design and implementation of the Program's evaluation activities that were expected to begin in Phase I of the APL. M&E Implementation and Utilization. At the program level, PEC has a strong system to monitor its performance. It receives detailed financial information on the transfer of resources to schools and how those resources are spent. For the reasons described in Section 2.2 above, practically from the start PEC's project unit was trying to improve its capacity to obtain data from the AEEs and schools to provide necessary fiduciary data. Initially PEC's ability to monitor progress at the school level was weak, but with Bank support and supervision a concerted effort was made to improve data collection. From the start, SIPEC encountered problems related to technological issues and overall implementation capacity. SEP eventually presented a proposal to the Bank to allow states to use existing or develop new alternative school-based information systems, and contracted a consultant to design the accreditation system for state-level MIS. In the end, SIPEC has been fully adopted by 19 states, partially adopted in 8 states and not adopted in 5states. The Bank was satisfied that the fiduciary and monitoring data obtained through alternative means was of satisfactory quality, and has adopted this more flexible, tailored approach to financial and monitoring systems for Phase II of the APL. PEC carried out a number of activities to gather and analyze information about the effectiveness of the Project. Some studies were used in the design of the Program's next phase and will lead to the development of a detailed impact evaluation of the Program. This information has been utilized to evaluate level of social participation to measure the impact of PEC's ability to reach poor and indigenous communities, to measure PEC's effect on social communication, pedagogical practices, and accountability, and to assess efficiency in monitoring of PEC schools. These activities include: (i) continued monitoring of information received from schools through the AEEs, (ii) collection of baseline data for the impact evaluation that focuses on intermediate indicators such as social participation and school management, (iii) a study of social participation in PEC schools, and (iv) an assessment of other SEP programs for possible linkage with PEC. 2.4 Safeguard and Fiduciary Compliance Safeguards. The only Bank safeguard policy that applied to the Project was OD 4.20, Indigenous People3. The Indigenous Peoples' Development Plan prepared for the Project identified the main issues that affected the indigenous population's access to basic education and incorporated their priorities and expectations identified during the 3 The Project was approved before the current OP 4.10, Indigenous Peoples, was introduced. 11 consultation process. The Project exceeded the targets in the Indigenous Peoples Development Plan for incorporating PEC in indigenous schools: over seven percent of PEC schools are indigenous, against a target of four percent (see also Section 3.5.a). The Project supported an evaluation of PEC's implementation in indigenous schools that included consultation with children, parents, teachers, directors, and community stakeholders in indigenous schools. . The main recommendation were to: (i) develop a promotion strategy specifically for indigenous communities; (ii) give greater focus on schools in poorer and more marginalized communities and increase the transparency of the state-designed school selection criteria; (iii) design special training modules for parents and other stakeholders in indigenous communities on the preparation of participation instruments; (iv) reinforce engagement of education authorities in the states with the highest concentration of indigenous people, including enhanced dissemination, provision of technical assistance to schools for PETE preparation, and selection of schools; and (v) develop partnerships with other federal programs also targeting poor, marginalized population, including indigenous peoples. Fiduciary. PEC involved a considerable level of financial management complexity, including transfers from the Federal Government to state governments to schools, and the related record-keeping and flow of financial information back from the schools, through the states, to the project unit in PEC. It involved financial management of several dispersed spending units with a large number of small transactions. Under the Project, the Sistema de Información del Programa Escuelas de Calidad (SIPEC), which was online and operational at the time of loan approval, was to be updated and expanded to monitor and administer expenditures at the state level. . Almost from the start there were issues with the roll out of SIPEC to states and schools, as described previously. Despite the early agreement upon an Action Plan to improve financial management and strengthened supervision by the Bank at the state level, efforts to roll-out SIPEC were slow, and the Government proposed to allow states to use existing or develop new school-based information systems. The development of an accreditation system for individual state-level MIS was financed to ensure that these alternative systems ensured minimum requirements with respect to school-based reporting. Despite minor delays in the submission of project audits, the only issue raised in those reports was that PEC continued to submit financial management reports with information sent via parallel systems due to deficiencies in SIPEC. Procurement under the Project was also relatively complex since it involved procurement of small goods, works and training at the level of thousands of school units under the supervision of the state education authorities. The procurement of these small items was mostly through shopping or price quotations from local suppliers. Procurement proceeded relatively smoothly. An in-depth procurement review carried out in 12 states by an independent firm under the Project revealed that while PEC schools were complying with the procedures outlined in the Project's Operational Manual, and that expenditures were eligible for financing under the Project, some specific areas for improvement were 12 highlighted. These were successfully addressed under an Action Plan that included recommendations for technical assistance to schools on procurement guidelines. 2.5 Post-completion Operation/Next Phase As the first phase of a three-phase APL ends, the Government has finalized preparation of Phase II for which a Bank loan of US$220 million was approved by the regional Vice President on May 25, 2010 and received final approval on June 17, 2010. All of the triggers specified for moving forward with the APL's Phase II were met as presented in Table 1 below: Table 1: Status of Triggers for Phase I and II of the School-Based Management Program Trigger Status in 2009 Number of schools participating in the Program Exceeded. The number of schools in PEC has will increase by 50 percent increased by 84 percent, from 21,312 (2005/06) to 39,180 (2008/09) Program schools as a percentage of all public Exceeded. In 2009, 18 percent of public basic basic education schools will increase to 15 education schools have participated in PEC. percent Completion of baseline data collection for Met. PEC carried out a baseline survey. impact evaluation Government approval of the Program Met. The Government, through SEP, has expansion plan with specific targets for approved the plan with targets to expand school expansion coverage and learning achievements, among others. The main challenges learned from the implementation of Phase I have been addressed in the design of Phase II, including those relating to: (i) the monitoring system; (ii) social participation; and (iii) coordination with other Government programs. Specifically, with respect to monitoring, Phase II will support states in developing their own monitoring systems compatible with SIPEC instead of requiring them to utilize SIPEC as a requirement for receiving federal funding. With respect to social participation, Phase II will support improvements in mobilizing social participation in decision-making by introducing a simplified school planning instrument accompanied by extra training of school directors and parents, thereby addressing the Program's main criticisms: that it is too complex for many school communities and that social participation is lagging. Finally, Phase II will play a greater role in coordinating different federal government programs at the basic education level, including Oportunidades (a conditional cash transfer program) and CONAFE (which operates a national compensatory education program). In addition, as originally planned, Phase II will support the carrying out of an impact evaluation of the Program, building on the data that was made available under Phase I. Building upon these lessons, Phase II of the APL for Mexico's School-Based Management Program will support the expansion of Program coverage from about 40,000 to 50,000 schools, focusing especially on the most disadvantaged schools in low- income areas, putting greater emphasis on marginalized schools and on the indigenous 13 population, as well as reorient a significant portion of the school grants to improve schools' internal efficiency and learning outcomes. Phase II would also support pilots to identify, design and develop adjustments to adapt the Program for further expansion to schools serving small disadvantaged communities in semi-urban and non-urban areas. In addition, it would introduce a simplified school planning instrument (PETE-Formato Concentrador or PETE-Condensed) accompanied by extra training of school directors and parents, which would address the Program's two main criticisms described above. The Oportunidades program will support PEC's efforts in the poorest areas. Finally, PEC will change its Operating Rules to ensure that a larger share of school grants are used to acquire education inputs rather than infrastructure, and to better link PEC with other education programs. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The Project's objectives continue to be highly relevant to the Government and to the Bank's Country Partnership Strategy (CPS) for Mexico4. Both the Government's Plan Nacional de Desarrollo 2007-2012 (NDP, National Development Plan 2007-2012) and the Bank's CPS prioritize comprehensive assistance to efforts to promote social inclusion and to reduce poverty5. The Government's NDP endorses the Program, and its expansion, under the objective of "Promoting Social Inclusion and Reducing Poverty". The Government's education sector program specifically calls for greater community participation in education. The Project's design and implementation, with a strong bottom-up and participatory approach, through which Bank assistance and support is provided through an APL continues to be relevant for the reasons described in Section 2.1. The Government and the Bank finalized the preparation for Phase II of the APL Program with a US$220 loan. As expected when the APL was designed, this phased approach to project has served the Program well in that Phase II already incorporates several lessons learned during Phase I through adjustments to the PEC's design. 3.2 Achievement of Project Development Objectives Through the implementation of its three components, the Project made important progress towards the achievement of its specified outcome indicators and targets and intermediate results as presented in the PAD's Results Framework. 4 World Bank Group, Country Partnership Strategy for Mexico, Report No. 42846-MX dated March 4, 2008. 5 Poder Ejecutivo Federal (2007), Plan Nacional de Desarrollo, 2007-2012. 14 Number of schools participating in PEC as a percentage of the total number of basic education schools. In 2005/06, 21,500 basic education schools, or 10.3 percent of the total number of such schools in Mexico participated in PEC. By 2009, the Project supported 40,790 basic education schools, representing 20 percent of the total basic education schools. Of these, 27,974, almost 69 percent of PEC schools, were located in areas of medium, high and very high marginalization. This result was achieved through the various activities that SEP implemented under the Project to disseminate the Program, to provide school grants through the institutional procedures for the Program, and to support AEEs in the Program's implementation (Annex 2). Participation by communities in the planning, accountability and decision making at the school level increased steadily under the Project. With respect to the school planning processes, at the time of appraisal, only 62 percent of community members in PEC schools knew what their respective PETE was, and only 51 percent had actually participated in the design or adjustment of the PETE. By project completion, those figures had increased, based on the evaluation of social participation in PEC, to 76.8 percent and 74.2 percent respectively. Similarly, indicators of PEC's accountability within communities showed steady albeit smaller increases. At the time of appraisal, 96 percent of community members observed that parents were informed about student performance, and 82.8 percent of community members observed participatory decision- making among parents, teachers and principals about school affairs. By project completion, the respective figures had increased to 99 percent and 90.2 percent, respectively. The Project was less successful in promoting perceived improvements in teachers' pedagogical skills in PEC schools, although there is some question regarding the baseline of this intermediate results indicator and the consistency of its application over the period, especially given the relatively large decline in community members' observations. A total of 95 percent of community members observed teachers encouraging and supporting student performance at the time of appraisal, but this had dropped to 72 percent at completion. Similarly, a somewhat correlated indicator, percentage of community members that observe teachers encouraging the active participation of the students, dropped from 93 percent at appraisal to 66.9 percent at completion. Scores in Enlace (achievement test at national level) have been increasing faster in PEC schools than in non-PEC schools, which is clear evidence that the program has been impacting in learning quality. Commitment to the goals and activities of the PETE as expressed by local financial contributions to the schools. Progress towards achievement of this outcome was significantly lower than expected, as measured by this indicator. At appraisal, a total of MX$30,168 was mobilized per participating school; this figure dropped significantly to MX$8,170 (based on partial data) at project completion. In retrospect, this indicator was 15 problematic. First, its baseline was established on the basis of contributions from a total of schools under what was then a much smaller Program. As the Program expanded rapidly to a large number of schools, including a large number of smaller schools in more marginalized areas, it would have been nearly impossible to maintain the average level of contribution per school. With smaller schools, fewer total school contributions should have been expected, and with more schools, the denominator increased rapidly. Furthermore it is likely that the economic downturn that has affected Mexico and led to a 6.8 percent contraction in GDP in 2009, undoubtedly affected families' disposable income, in the aggregate, thereby reducing their ability to make contributions to their child's school. Because of the problems with its formulation, this indicator has been replaced for Phase II by the following: commitment to the goals and activities of the PETE as expressed by parental participation in PETE design or adjustment (as a percentage of total parents). By using this revised indicator as a proxy for community commitment to the goals and activities of the PETE, it is clear that the Project was successful in meeting the objective of increased community involvement. As described above, the Project was very successful in increasing community participation in both school planning and accountability in decision-making. 3.3 Efficiency An economic analysis was not prepared in the context of this ICR. Being a three-phase APL for an important and innovative program that will represent a long-term commitment to school-based management, a full economic analysis should most likely be prepared for the ICR of the final phase. The Project's ex-ante economic analysis in the PAD justified the Project's viability on the basis of expected school and out-of-school benefits to be generated by the Project, most importantly, improvements in community participation measured by the amount of funds contributed by the school community to the Program, and efficiency gains or savings that would accrue to the education system as a result of lower repetition and dropout rates produced by the Program. On the cost side, the analysis noted that the Program would become more cost-effective because public unit costs per student would decrease. Local financial contributions to the schools actually decreased in the first phase, although the expectation that these might increase concomitantly with the Program's expansion to marginal areas in retrospect was optimistic. It is too early to evaluate possible efficiency gains due to lower repetition and dropout rates produced by the Project. Still, several recent studies reveal promising results on PEC's impact. A study by Shapiro and Skoufias (2005) found that participation in PEC reduced dropout rates (by 0.24 percentage points), failure rates (by 0.24 percentage points) and repetition rates (by 0.31 percentage points)6. A second evaluation conducted by Murnane et al (2006) found that 6 Shapiro, J., and E. Skoufias. 2005. "The Pitfalls of Evaluating a School Grants Program Using Nonexperimental Data." Mimeo, World Bank. 16 PEC lowered dropout rates, but not failure rates.7 Studies of other school-based programs in Mexico8 have found that they improve school participation and communication, motivate teachers, reduce teacher absenteeism, and increase student attendance and parents' participation. Early indications are that, over time, PEC, and the early stages of the Program that the Project supported, will lead to increased efficiency in the delivery of quality education programs in Mexico. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The Project's Overall Outcome Rating is considered Satisfactory. This rating is based on the Project's and the Program's continued high relevance to the Government and to the Bank's Country Assistance Strategy. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The Project was national in scope and did not focus its efforts exclusively on states with higher levels of poverty. Rather, its design recognized that disadvantaged schools and students exist in every state, including in urban areas, and that high performing schools exist even in poor states. The Project adopted a bottom-up approach that empowered the school communities to determine whether or not to opt to participate in the Program. This was an articulated position, presented during preparation, where a specific decision regarding whether to focus exclusively on rural and disadvantaged schools was decided based on several factors, including that those schools were supported by Mexico's compensatory education program and that indigenous students existed in urban areas The Project would serve to fine tune the Program's approach and pilot its expansion to more disadvantaged schools in subsequent phases of the APL. While PEC has made a strong effort to reach poor schools through its focus on targeting, it did not directly address the issue of equity during Phase I. A total of 68.58 percent of PEC schools are in highly marginalized or marginalized localities compared to 77.37 percent of schools as a whole. Likewise, 31.42 percent of PEC schools are located in areas with low or very low levels of marginalization, compared to 22.63 percent of all schools. However, when analyzing this data on a per-student (as opposed to school) basis, the distribution of students in PEC and non-PEC schools becomes quite similar 7 Murnane, R. J., J. B. Willet, and S. Cardenas, 2006. "Did Participation of Schools in Programa Escuelas de Calidad (PEC) Influence Student Outcomes?" Working Paper, Harvard University Graduate School of Education, Cambridge, MA. 8 The School Management Support Program (Apoyo a la Gestión Escolar, AGE), created in 1996, similar in design to PEC but focusing on smaller rural schools and operating through Parents' Associations. 17 because schools in poor areas tend to be substantially smaller than schools in other areas, reducing concerns about equity. Although the Project did not specifically target the indigenous, it exceeded the goals specified in the IPDP. At the time PEC was launched in 2001, there were only 27 indigenous centers (schools with 70 percent or more of indigenous students) participating in the Program, out of a total of 2,240 PEC schools. The IPDP aimed that the PEC's indigenous centers would comprise 4 percent of PEC schools. By 2009, PEC covered a total of 40,790 schools, of which 2,950 were indigenous centers, equivalent to seven percent of the total PEC schools. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) PEC was an established program that was launched five years before the appraisal. The staff of SEP that is directly involved with managing the Program have acquired significant experience in its overall administration, which is based on established procedures. PEC's Operating Rules, which are issued annually by SEP and published in the Government's Diario Oficial (the official gazette), govern the Program's implementation activities, including the eligibility and selection criteria for schools to enter the Program. PEC funds are allocated to the states, under a transparent formula matching the amount of funds provided by states at a rate of three-to-one. The AEEs have also developed capacities, at varying levels, to disseminate the program, provide technical assistance to schools for preparation of their PETEs, analyze the proposals received and select the schools that will participate. These processes are set out in the form of an established government program, and with a sound institutional backing to support the Program's implementation in Mexico's federal structure, and through schools and communities. (c) Other Unintended Outcomes and Impacts (positive or negative) There were no unintended outcomes and impacts associated with the project. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops There were no specific beneficiary surveys or stakeholder workshops commissioned under the Project, however PEC does monitor parent participation in the Program. In 2009, 77 percent of parents in PEC schools are aware of the PETE. Around 70 percent of parents have observed that teachers are actively engaging the participation of students in the school. 4. Assessment of Risk to Development Outcome Rating: Negligible The Risk to Development Outcome for the Project is considered Negligible. There is continued strong support to the Project and the Program at all levels within Mexico: the 18 Federal Government, the State Governments and local communities. At the federal level, the Government continues to prioritize the importance of school-based management as a means to improve the quality of education. This public awareness has been a decisive factor for the support provided at all levels for the operation of the Program. Beneficiary ownership has been growing rapidly, as the demand from schools to participate in the Program has increased steadily. The institutional arrangements for the Project and the Program's continued operation are in place, and are expected to continue with the Bank's financial and advisory support for their strengthening and refinement in the context of Phase II of the APL. Finally, despite a severe economic slowdown, the Government has protected the budget of the education sector. The 2010 Federal Budget shows that the education sector (in particular, PEC) is spared from cuts that have affected other sectors. Once early results of what are expected to be significant impacts on learning outcomes become available, support for the Project, and the Program more broadly should only continue to strengthen. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory The Bank's Performance in Ensuring Quality at Entry is rated Moderately Satisfactory. The Project's preparation counted on extensive analytical work, the incorporation of lessons learned from the implementation of several school-based management projects in countries in Latin America and elsewhere, a carefully selected lending instrument, and tailoring the objectives of the APL Program's three phases for which especially the Project could be held accountable. The Project's implementation arrangements were challenging due to the national focus of the Project. Monitoring and evaluation arrangements were also carefully planned and staged so that the APL Program will develop an impact evaluation before its completion. One issue in the Project's design had not been contemplated was the large financing for school grants in the first six months of the project with nearly half of the loan had been disbursed by mid-2007. Loan disbursements were out of sync with the PEC's capacity to monitor and evaluate the Project's progress. The definition of the Project's outcome indicator related to commitment to the goals and activities of the PETE, using as a proxy the amount of local financial contributions, were not realistic especially given the Project's goal to increase its focus on poorer schools. (b) Quality of Supervision Rating: Satisfactory. 19 The Bank's Quality of Supervision is rated Satisfactory. The Bank's supervision effort was focused on keeping the Project on track. Supervision was steady, responsive and pragmatic, and supervision reporting was candid, identifying issues as they arose. The Bank quickly identified the issue that upon effectiveness, disbursements were out-pacing PEC's ability to monitor progress, and worked closely with the project unit and SHCP to slow down disbursements for school grants while agreeing upon changes internally in PEC's management structure and a plan to bring monitoring back on track in an accelerated manner. Similarly, when it became clear that many states were not able to adopt SIPEC, the Bank sought, and achieved a pragmatic solution in a manner that responded appropriately to the Bank's fiduciary responsibilities. More importantly, however, Bank supervision was forward looking while never losing track of learning from experience. The Bank supervision made both small and large adjustments in the Program's design to ensure that the Project would come to a satisfactory conclusion and that the APL Program as a whole would as well. The supervision team carried forward these lessons into the design of Phase II of the APL. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory Overall Bank Performance is rated Satisfactory based on similar ratings for Ensuring Quality at Entry and Quality of Supervision. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory The Government's Performance is rated Satisfactory. The Government displayed commitment to the Program and the Project, and to their objectives. The overall enabling environment was supportive, both institutionally, and in terms of financial allocations to the Project and the Program. As mentioned in Section 4 above, despite serious fiscal constraints, the Government has protected the budget allocation to the education sector as a whole and to the Program in particular, and is proceeding with original plans to gradually expand the number of PEC schools. (b) Implementing Agency or Agencies Performance Rating: Satisfactory The Implementing Agencies' Performance is rated Satisfactory. In this respect, implementing agencies is taken to mean SEP, and specifically SEP's staff directly managing the Program, as well as at the state level, the AEEs, and individual schools. 20 For SEP, the challenge of working with the AEEs, which in turn worked through thousands of schools, each with widely varying institutional capacity was formidable. Throughout, SEP remained committed to the Project's and the Program's objectives, despite some initial delays in getting things underway and on track, especially financial management and monitoring activities. When supervision missions identified these issues, the team in SEP invariably worked to address identified issues, and subsequent missions were generally reporting progress. As had been planned given the Program's decentralized implementation, throughout, SEP carried out several beneficiary/stakeholder consultations to obtain their feedback and perceptions of PEC. SEP staff responsible for PEC worked continuously in the transition from Phase I to Phase II of the APL Program. States demonstrated varying capacity for implementation, especially with respect to fiduciary aspects such as financial management and procurement. These issues did not in the end affect project outcomes, but have been taken into consideration in the design of Phase II. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory The Overall Borrower Performance is rated Satisfactory based on the ratings for Government Performance and the Implementing Agencies' Performance 6. Lessons Learned Project-Specific Lessons Learned. There are several project-specific lessons learned that have been incorporated into the design of Phase II of the APL. These are: ¾ Top-down solutions are often inefficient and do not work on the ground. This was a central observation made during the Program's design, based on the international literature. However, some elements of PEC were still top-down and did not work well. One clear example was PEC's information system, SIPEC, which was developed centrally. For Phase II, PEC is developing a more flexible information system that will respond to the Program's broader information needs, while being more responsive to the varying capabilities of states and schools, allowing them to customize their own information systems within a general framework. ¾ Targeting is complicated, given the multiple objectives of PEC and political realities. PEC has a transparent formula that allocates resources across states on a per capita basis using the total number of children between the ages of 4 and 14. As a result, well-off states effectively receive a higher allocation of resources for their poor population while poorer states receive fewer resources for their poor students. The Government and the Bank are working to adjust the targeting of resources both at the federal level (improving equity) and at the state level (ensuring that disadvantaged schools are more likely to participate). 21 ¾ Creating legal autonomy does not automatically lead to changes in school management. By law, community participation plays a role in all public schools in Mexico. PEC was designed to empower communities to carry out this responsibility effectively. However, qualitative studies and the Bank's observation show that community participation has been mixed. In many cases, the school director effectively runs PEC with community members often playing a passive or consultative role. The studies found that in many cases community participation existed in paper but not in reality. Real community participation requires cultural changes, incentives, and capacity building. Phase II will support enhanced capacity building for school directors and community groups and will more actively promote greater social participation. Broader Lessons Learned. There are also several broader lessons learned with respect to the design and implementation of the Project that apply equally to other projects, although some of these may sound redundant. ¾ Government commitment to the implementation of a Project or Program is perhaps the best indicator of its overall success. ¾ An APL is a useful instrument to support implementation of a government program where a long-term involvement may be necessary to produce expected outcomes. It is important when designing an APL to build in the processes and methods by which the program's and the APL's expected outcomes will be measured over the longer term, while cautiously defining only those outputs and indicators by which the progress of earlier phases can realistically be measured. Also, the APL instrument is well suited to the needs of programs in early stages of development and establishment where it is important to incorporate the lessons of earlier phases and implementation into the design of the program through minor adjustments. ¾ When designing projects and corresponding Bank loans with both relatively fast disbursing components and slower technical assistance type activities, it is desirable to build in mechanisms that pace the disbursements of fast disbursing components to progress in the implementation of technical assistance for institutional strengthening type activities. This is especially important in the case of projects where there was a large amount of financing that disbursed shortly after effectiveness. One possibility would be to build milestones into the disbursement of fast disbursing activities that would pace disbursements for those activities reflecting progress with implementation of technical assistance or specific institutional outputs or actions. ¾ When supervising projects it is important to be pragmatic and not lose sight of the broad objectives. When it became clear that the SIPEC was too complex for implementation in several states and schools, Bank supervision immediately stood back and identified the next best solution with which the same objective, 22 (i.e., establishment of a monitoring system) could be accomplished. Progress towards the objective was accomplished, but the specific process or instrument through which it is accomplished is different than originally planned, adjusted to meet institutional realities and needs, albeit no less responsive to the original objective. ¾ Care is needed in the design of a project's monitoring framework and indicators to avoid individual indicators that are not feasible given the changes brought about by the Project. For example, the project has two main indicators: (i) number of schools participating in PEC as a percentage of the total number of basic education schools, and (ii) commitment to the goals of the PETE as expressed by local financial contribution to the schools conflicted. As PEC expanded, and increased the number of schools into marginalized areas, family contributions actually went down, impacting the achievement of the latter indicator. This indicator, in and of itself, is also likely subject to exogenous factors, not the least of which is the downturn in the economy that undoubtedly affected families' ability to increase or even maintain a fixed level of contribution to their children's' schools. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The implementing agency, PEC, has prepared comments on the World Bank's financing for this Implementation Completion Report. A copy of this letter with a translation is contained in Annex 7 of this ICR. PEC was grateful for the support that it has received from the World Bank, which has helped to improve the impact and the financial management of the program. PEC is one of the leading federal education programs and its relationship with the Bank has strengthened its position within Mexico. The Bank has played an important role in improving PEC's financial management. The initial high level of disbursement can be explained the nature of the school grants that can be claimed and disbursed much faster than the Bank had originally planned. At the same, the original design of an information system (SISPEC) was not realistic and the second phase of the Program will be more flexible. Likewise, the initial proposal for evaluation was felt to be unrealistic given government rules and capacity. (b) Cofinanciers 23 The World Bank was the only external financier involved with the implementation of PEC. (c) Other partners and stakeholders No additional comments received. 24 Annex 1. Project Costs and Financing The Project supported three components, with both Government and IBRD financing. Component 1, School Grants, is the center of PEC and accounted for an estimated 93.8 percent of the project cost and 97.7 percent of IBRD financing. Component 2, Project Oversight, accounts for 4.4 percent of the project cost and was to be entirely financed by the Government. Component 3, Monitoring and Evaluation, accounted for 1.7 percent of the total cost of the project. Following standard procedures in Mexico, the IBRD loan did not provide any additional financing to the Mexican budget. PEC authorities only tracked IBRD spending on project activities and this is reflected in Table 1. Component 2 was fully implemented by the Government with an adequate budget. The main difference between the appraised estimates and actual budgeted amount is in component 3.1, which was originally budgeted for US$ 4.23 million. A baseline survey was carried out at a cost of $80,000. The remaining resources were reallocated to Component 1. (a) Project Cost by Component (in US$ Million equivalent) Total Cost in Total IBRD Cost in Actual IBRD Actual IBRD as Components Appraisal Appraisal financing percentage of (US$ million) (US$ million) (US$ million) appraisal amount School Grants 324.70 234.49 238.75 101.82% Program Monitoring and 0.00 15.15 0.00 n/a Oversight Policy Development and 4.91 5.80 0.65 13.24% Evaluation Total Baseline Cost 345.65 239.40 239.40 100% Physical Contingencies 0.00 0.00 0.00 n/a Price Contingencies 0.00 0.00 0.00 n/a Total Project Costs 345.65 239.40 239.40 100% Front-end fee PPF 0.00 0.00 0.00 n/a/ Front-end fee IBRD 0.60 0.60 0.60 100% Total Financing Required 346.25 240.00 240.00 100% (b) Financing Appraisal Type of Actual Percentage of Source of Funds Estimate Cofinancing (USD millions) Appraisal (USD millions) Borrower 106.25 n/a n/a International Bank for Reconstruction 240.00 240.00 100% and Development 25 Annex 2. Outputs by Component The following table outlines the activities and outputs by component during the Project. Intermediate Results Target Actual Component 1, School Grants Increased community 71% of community members know PETE 79.7% of community members know participation, planning and PETE accountability in the decision- 60% of community members participate in the making at school design or adjustment of PETE 66.9% of community members participate in the design or adjust of PETE 96% of community members observe that parents are informed about student performance 98.4% of community members observe that parents are informed about student 87.7% of community members observe performance participatory decision-making between parents, teachers, and principal about school affairs 92.6% of community members observe participatory decision-making between parents, teachers, and principal about school affairs Improved teacher's pedagogical 95% of community members observe teachers 97.4% of community members observe skills in PEC schools encouraging and supporting student performance teachers encouraging and supporting student performance 93% of community members observe teachers encouraging the active participation of the 97.4% of community members observe students teachers encouraging the active participation of the students Improved access for schools in 9,450 PEC schools located in areas of medium, 27,974 PEC schools located in areas of low income areas to actions that high, and very high marginalization levels (by medium, high, and very high strengthen School Based 2005) marginalization levels (by 2009) Management Component 2, Program Monitoring and Oversight Improved and more efficient PEC 80% of states (25 states) report on time the 15 states report on time the information of operation information of the Program through SIPEC the Program through SIPEC Component 3, Policy Development and Evaluation Improvement in the quality of Baseline for impact evaluation completed Baseline for impact evaluation completed PEC evaluations 1 study per year Numerous studies of different types (technical, legal, administrative) were carried out by PEC with IBRD support 26 The table below provides information on the number of schools participating in the PEC and the total spending on PEC per state, in Mexican pesos. Average No of Average No of No of PEC Total Amount No of School Federal Total School PEC schools in of School PEC Grant per Entity Basic Grant schools poor areas Grants students student Schools (MX$) (MX$ ) Aguascalientes 1,489 360 128 18,359,131 50,998 63,562 288.84 Baja 2,954 1,022 424 44,437,084 43,481 205,141 216.62 California Baja 882 330 134 20,581,750 62,369 60,995 337.43 California Sur Campeche 1,756 401 354 23,703,326 59,111 69,245 342.31 Coahuila 3,845 848 341 33,590,716 39,612 194,852 172.39 Colima 977 102 60 10,269,905 100,685 22,262 461.32 Chiapas 17,350 826 810 28,500,000 34,504 122,606 232.45 Chihuahua 5,487 1,062 464 44,751,576 42,139 224,477 199.36 Distrito 4,660 2,368 657 146,139,469 61,714 788,683 185.30 Federal Durango 5,219 1,614 941 27,115,387 16,800 180,698 150.06 Guanajuato 9,704 2,253 1,820 79,052,373 35,088 276,939 285.45 Guerrero 10,348 2,460 2,352 106,175,191 43,161 272,524 389.60 Hidalgo 7,196 947 737 37,224,906 39,308 146,759 253.65 Jalisco 11,453 1,928 1,201 97,441,093 50,540 314,958 309.38 México 15,662 5,526 4,318 205,471,014 37,183 1,055,571 194.65 Michoacán 10,440 2,482 2,026 116,898,561 47,099 277,395 421.42 Morelos 1,972 400 299 33,074,267 82,686 71,450 462.90 Nayarit 2,757 351 194 14,341,503 40,859 59,040 242.91 Nuevo León 5,601 2,107 754 76,482,362 36,299 349,226 219.01 Oaxaca 11,938 1,475 1,440 63,043,938 42,742 124,507 506.35 Puebla 10,091 2,326 2,233 84,726,452 36,426 264,097 320.82 Querétaro 2,941 545 431 27,253,782 50,007 89,380 304.92 Quintana Roo 1,579 332 263 20,196,546 60,833 68,475 294.95 San Luis 7,674 672 560 38,597,242 57,436 111,367 346.58 Potosí Sinaloa 5,902 1,635 786 51,128,050 31,271 294,644 173.52 Sonora 3,838 1,547 822 61,055,430 39,467 263,586 231.63 Tabasco 4,798 634 481 34,055,766 53,716 97,080 350.80 Tamaulipas 4,819 1,610 829 98,124,740 60,947 345,678 283.86 Tlaxcala 1,713 400 319 23,026,176 57,565 84,625 272.10 Veracruz- 19,560 1,026 888 45,446,404 44,295 141,933 320.20 Llave Yucatán 2,767 743 631 25,910,079 34,872 140,963 183.81 Zacatecas 4,799 458 277 27,187,488 59,361 65,306 416.31 TOTAL 202,171 40,790 27,974 1,763,361,707 43,230 6,848,024 257.50 27 28 Annex 3. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Erik A. Bloom Senior Economist LCSHE Ricardo Rocha Silveira Senior Operations Officer HDNGA Maria E. Castro-Munoz Sr. Social Scientist LCSSO Maria E. Colchao Senior Program Assistant LCSHE Isy Faingold Vigil E T Consultant LCSHE Ruben Fernandez Garcia Temporary LCC1C Vicente A. Garcia Moreno Consultant LCSHE Alina Garduno Lozano Temporary LCSHD Dmitri Gourfinkel Financial Management Analyst LCSFM Mark V. Hagerstrom Lead Operations Officer LCC1C Martha Belem Hernandez Junior Professional Associate LCSHD Peter Anthony Holland Education Spec. LCSHE Efraim Jimenez Consultant EAPCO Claudia Macias Operations Officer LCSHH Mauricio Ondarreta Huerta Consultant LCSHE Victor Manuel Ordonez Conde Financial Management Specialist CTRLP Harry Anthony Patrinos Lead Education Economist HDNED Gabriel Penaloza Procurement Analyst LCSPT Anna Maria Sant'Anna Consultant LCSHE Juan Carlos Serrano-Machorro Financial Management Specialist LCSFM Emmanuel Skoufias Lead Economist PRMPR Sandra Monica Tambucho Perez Team Leader CTRDM Miguel Angel Vargas Consultant LCSHE Mary A. Dowling Program Assistant LCSHE 29 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY05 46 158.36 FY06 47 174.89 FY07 0.00 FY08 0.00 Total: 93 333.25 Supervision/ICR FY05 0.00 FY06 13 53.19 FY07 29 143.47 FY08 26 124.09 FY09 37 0.00 Total: 105 320.75 30 Annex 4. Beneficiary Survey Results (if any) N/A 31 Annex 5. Stakeholder Workshop Report and Results (if any) N/A 32 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR Informal translation of Borrower's comments (Letter dated June 15, 2010) Dr. Evangeline Javier Director Human Development Department Latin American Region The World Bank Ref: Quality Schools Program (PEC) - Phase I - 73 470 Loan - ME Government's comments on Phase I We would like to offer some comments on the School-Based Management Project (PEC) which was approved on December 13, 2005 and was declared effective on November 17, 2006. This World Bank's project provided financial and technical support to PEC, which the Federal Government of Mexico established in 2001 with the aim of transforming Mexican schools through improved decision-making process, among other benefits. This positions PEC as an international example of improving the quality of education. In general, we are pleased to have worked with the World Bank during the project implementation. During the execution of the Loan, the program has shown its impact on beneficiaries, which generates a positive impact on basic education in Mexico. The World Bank has been a key partner in this operation, supporting the PEC team in its efforts to improve program and financial management. At the same time, the Government and the World Bank have worked together to improve the equity of the program's impact. The Government and the Bank identified potential activities to improve equity in resources distribution and are making efforts in that direction. This does not necessarily require distributing more resources to poorer states, but a redistribution of resources within each state. An important issue in the relationship between the World Bank and PEC is the area of financial management (FM). It was not easy to meet simultaneously with the requirements of the federal law and the World Bank regulations. Obviously, each state has a different situation and capacity; however, Bank supervision found in PEC financial management the capacity to obtain the required financial information. Disbursements under Category 1 (transfers to schools), because of their characteristics, are susceptible to rapid disbursement, which impacts the amount allocated to Loan's Category 1. Thus the Bank initially found a level of disbursement than expected. The Government and the World Bank worked together to streamline disbursements over the remaining term of the project. We consider that the transfers to state governments and from them to the schools are appropriate and that adequate financial information has been collected. The complexity 33 lies in obtaining the information records from schools to the states and to transfer them to PEC National Coordination. Originally, a single information system (SIPEC) was proposed to coordinate all the information on PEC. For the second phase, it was agreed that states may use or develop their own monitoring and information systems that are consistent with Federal Government requirements and PEC will provide the necessary assistance to those states that want to develop their own information system. Those that do not choose this option will use the system proposed by PEC National Coordination. In original proposal from the first phase was to develop an ambitious and rigorous evaluation for PEC. In our opinion, this proposal did not take into account the Government's capacity and resources nor the public sector's administrative requirements. That caused a delay in the implementation of the evaluation that was not expected in the PAD. However, the Government is committed to undertake the PEC's "Impact Evaluation", which is expected to be accomplished during Phase II. During Phase I the collection of the baseline was started. The program's efforts will only be successful with the participation of all educational stakeholders, including the parents, who are key allies in the implementation of the program. Hence, the Social Participation School Councils are crucial for the project. It is important to highlight the existence of the Annual Work Program (PAT), which is complementary to the PETEs and allows the annual review of actions implemented by The school. I take this opportunity to send a cordial greeting. Best Regards. ENGINEER DANIEL HERNANDEZ NATIONAL COORDINATOR Quality Schools Program 34 35 36 Annex 7. List of Supporting Documents Garcia, V., Knaul, F. and Patrinos, H.A., Returns to Education and Quality of Education in Mexico, 2005, World Bank (Processed). Murnane, R. J., J. B. Willet, and S. Cardenas, 2006. "Did Participation of Schools in Programa Escuelas de Calidad (PEC) Influence Student Outcomes?" Working Paper, Harvard University Graduate School of Education, Cambridge, MA. Shapiro, J., and E. Skoufias. 2005. "The Pitfalls of Evaluating a School Grants Program Using Nonexperimental Data." Mimeo, World Bank. World Bank (2005). Determinants of Learning Policy Note. Latin America and the Caribbean Region, Human Development Department. World Bank Group, Country Partnership Strategy for Mexico, CAS Report No. 42846- MX, March 4, 2008. 37 38