Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) Report Number : ICRR0020365 1. Project Data Project ID Project Name P106589 MX IT Industry Development Project Country Practice Area(Lead) Mexico Transport & ICT L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-75710 30-Jun-2013 80,000,000.00 Bank Approval Date Closing Date (Actual) 10-Jul-2008 31-Dec-2015 IBRD/IDA (USD) Grants (USD) Original Commitment 80,000,000.00 0.00 Revised Commitment 73,148,488.84 0.00 Actual 73,148,488.84 0.00 Sector(s) ICT Services(41%):Public Administration - Information and Communications Technologies(38%):Other Information and Communications Technologies(21%) Theme(s) Other Private Sector Development(45%):Micro, Small and Medium Enterprise support(11%):Other Financial Sector Development(44%) Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Krishnamani George T. K. Pitman Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The Project Development Objective (PDO) as stated in the Loan Agreement (Schedule 1, page 6) and in the Project Appraisal Document (PAD, page 8) was similar. “To assist the Government of Mexico to implement an alternative strategy for Development Program for the Software Industry (PROSOFT) in order to foster the creation of jobs in Information Technology (IT) companies in Mexico by improving their competitiveness and efficiency through access to: (i) a large supply of trained personnel. (ii) Technologies, quality standards and global marketing networks of multinational companies. and, (iii) private debt finance. Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Components There were seven components. Component A. Human Skills Development. Appraisal estimate US$38.30 million. Actual cost US$48.14 million. This component aimed at helping Mexico’s Information Technology (IT) and Information Technology Enabled Service (ITES) industry to compete globally by increasing its skilled manpower. (The IT services include application development, software design, systems integration, IT consulting, Research and Development (R&D) services, hardware and software maintenance, network administration and help desk services. ITES services (also called as Business Process Outsourcing (BPD)) could be remotely delivered using Information Communication Technologies (ICTs), such as, call centers, data processing, reservation and information services, accounting services, human resource management, sales contact centers, medical transcription services, telemarketing, etc.). Activities included providing training grants for technical, managerial and English courses, certifications, seminars, workshops for faculty, students, industry professionals and potential recruits for the IT industry. Component B. Strengthening of IT Clusters and Selected State Agencies. Appraisal estimate US$9.0 million. Actual cost US$13.16 million. This component aimed at helping existing IT clusters and associations in states having potential in the IT/ ITES sector. Activities included: (i) Providing access to technologies, processes and markets by linking local IT companies with global companies. (ii) Designing strategies for developing the IT/ITES sector. (iii) Raising awareness among local business on the benefits of outsourcing and using IT in business and production processes, and (iv) Awarding certification grants through the Development Program for the Software Industry (PROSOFT) fund. Component C. Financing of the IT Industry. Appraisal estimate US$2.90 million. Actual cost US$1.31 million. This component provided technical assistance for implementing an accreditation program for improving IT companies access to finance. Activities included: (i) Technical assistance for designing the Financing of IT Companies Accreditation (FIT) program. (ii) Accreditation Grants to IT companies, and (iii) Technical assistance to banks to appraise IT projects and companies. Component D. Supporting Infrastructure. Appraisal estimate US$10.10 million. Actual cost US$0.44 million This component supported the IT parks to promote agglomeration and clustering of companies in close proximity to research centers, universities, financial institutions and incubators. Activities included: (i) Technical assistance for preparing feasibility studies, design and bidding specifications for IT parks based on a Public-Private Partnership (PPP) model, and (ii) Financing the portion of the resources that the Federal Government would contribute to each PPP. Component E. Outsourcing of Government Services. Appraisal estimate US$2.70 million. Actual cost US$0.73 million. This component planned to execute subprojects at the Federal, State and Municipal levels using outsourcing and PPP approaches. Activities creation of an Integration Competency Center and setting up cost-effective technology escrow arrangements for improving the prospects of Small and Medium Enterprises (SMEs) in securing business within and outside Government. Component F. Strengthening of the Legal and Regulatory Framework and Institutions. Appraisal estimate US$6.10 million. Actual cost US$2.05 million. Activities included: (i) Technical assistance for drafting changes and improvements to the legal and regulatory framework. (ii) Awareness raising campaigns for improving sector laws and regulations. (iii) Tailor made professional training for legislators, judges, public officers, law enforcement officers and technical experts in trials. (iv) Creation of a center for disputes settlement within the Mexican Software Consortium, and (v) creation of a local master’s program in IT law, IT regulations and enforcement of best practices. Component G. Strengthening the Program for the Development of the Software Industry (the Spanish acronym - PROSOFT) and Project Management. Appraisal estimate US$3.60 million. Actual cost US$7.11 million. Activities included: (i) Creation and operation of an International Consultative Commission in PROSOFT to advise the Ministry of Economy (the Spanish acronym – SE) on new developments in the global IT industry. and, (ii) Support to the SE to manage, implement, monitor and evaluate project results. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. Appraisal estimate (including total baseline cost, costs associated with physical contingencies and Front-end Fee International Bank for Reconstruction and Development) US$80.00 million. There was reallocation of funding between project components with increase in allocations for component A, B, C and G activities and reduction in allocations for component D, E and G activities following the first project restructuring (discussed below). Actual cost at closure US$73.15 million. Project Financing. The project was financed through an IBRD loan. Appraisal estimate US$80.00 million. Amount disbursed US$73.15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) million.The amount disbursed was lower than the appraisal estimate due a combination of factors including, exchange rate fluctuations between the Mexican Peso and the US$ and cancellation of some subprojects by the Board of PROSOFT during implementation. Borrower Contribution. None was planned. Dates. The project was restructured twice. The first restructuring was after the Mid Term Review on 03/12/2012. As of 2010, only 4% of the loan was disbursed and many milestones had not been achieved due to a combination of factors including changes in implementation arrangements (discussed in Section 10b ) and delays in the initial years of the project. Following this restructuring, although the PDO remained unchanged, new Monitoring and Evaluation (M&E) indicators were added, target values were revised and funding was reallocated between components. Following this restructuring, a two-phase extension of the project closing date was envisaged. The first phase extended the project closing date by 18 months from 06/30/2013 to December 31, 2014, and a further extension of 12 months was envisaged contingent on implementation performance till then. The following changes were made through the second restructuring on 09/04/2014. There was a reallocation of funding between components and in view of the satisfactory implementation performance following the first restructuring, the project closing date was extended by 12 months for completing ongoing activities. The project closed two and half years behind schedule on December 31, 2015. 3. Relevance of Objectives & Design a. Relevance of Objectives Given its inherent advantages such as geographical proximity to USA, its large labor pool, presence of many multinational companies and membership of the North American Free Trade Agreement (NAFTA), although Mexico had a distinct opportunity to be a leading hub for IT/ITES industry in Latin America, Mexican firms did not use IT extensively in their business processes, in the years before appraisal. The sector's contribution to Gross Domestic Product (GDP) was only 3%, as compared to 5% in Latin America, 7% in high income countries and 9% in USA. In 2004, the government launched the Development Program for the Software Industry (PROSOFT in Spanish) aimed at promoting the ICT sector and this program among other things in a limited scale, established a guarantee fund for IT companies, developed Mexican certification standards, supported the development of a national level cluster to foster linkages between IT companies, a strategy for managing a federal fund for encouraging Public-Private Partnerships (PPPs) at the state level to foster the development of the IT industry. In 2006, the Government followed this up through a decree mandating federal government agencies to purchase IT services (as opposed to IT equipment), and thereby encourage the development of PPPs for providing government services. Although the statement of the PDO was convoluted and a simpler, clearer PDO would have helped in providing clarity, the PDO of improving competitiveness and efficiency of the IT/ITES industry was highly relevant to the Government strategy articulated in the National Development Plan (NDP) for the 2007-2012 period. Of the five pillars articulated in the plan, the second pillar highlighted the need for building economic competitiveness and generating employment. The NDP for the 2008-2013 period specifically identified the Information and Communication Technology (ICT) sector as key enablers for public modernization, social inclusion, improvements in productivity and high quality education. The project’s current relevance was reinforced by Mexico’s first Digital National Strategy (DNS) launched in 2013. The second objective of this strategy highlighted the need for fostering competitiveness and innovation in the ICT industry and integrating IT industry with other sectors. Further, while the previous focus of the PROSOFT program was on developing the IT industry itself, the new strategy and Plan for the 2014- 2024 period articulated in the Sectoral Agenda for the Development of Information Technologies in Mexico 2014-2024, emphasized on the application of information technologies to foster innovation and greater productivity in other sectors of the economy. The PDO continues to be relevant to the Bank strategy for Mexico. The Country Partnership Strategy (CPS) for the 2008-2013 period supported the objective of developing startup firms in businesses outsourcing and other IT related services. The PDO was also relevant to the Bank’s current CPS for the 2013-2018 period. This CPS highlighted the need for improving competitiveness through improving human skills, infrastructure, links between local and global companies and improving the financial, legal and regulatory frameworks in the IT sector. Rating High b. Relevance of Design The design was comprehensive and targeted key aspects of the ICT sector’s competitiveness and efficiency including through building skills, international certifications, access to financial services, regulatory and legal framework, awareness raising amongst policy makers, collaboration between IT firms offering complementary services, infrastructure and government as anchor client for outsourcing ICT services. The causal links between the project activities, their outputs and outcomes were clear and the intended outcomes were measurable. Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) Component A activities could be expected to increase the quality and quantity of skilled manpower and this in turn can be expected to aid in fostering jobs in IT companies. Component B activities (strengthening of IT clusters) in conjunction with Components D and F activities (supporting infrastructure such as IT parks and strengthening the legal and regulatory framework and institutions) can be expected to improve the competitiveness of IT companies. And component C activities (improving IT companies access to finance) in conjunction with components E and G activities (outsourcing government services and strengthening the Development Program for the Software Industry (PROSOFT) can be expected to improving the efficiency of IT companies. The multiplicity of project components at design – with seven components- led to initial delays and inefficiencies due to the inadequate capacity of the beneficiaries. However, given that none of the components were dropped indicates that the original comprehensive design was appropriate. Rating Substantial 4. Achievement of Objectives (Efficacy) PHREVISEDTBL Objective 1 Objective To foster the creation of Jobs in IT companies. Rationale Outputs. • 102,527 persons were certified by the Mexico Federal Institute for Remote Services and Technology (MexicoFIRST- an entity created to close the gap between academic capacity building and private sector needs). This exceeded both the original and revised targets of 30,000 and 64,293 respectively. Outcomes. • A total of 40,918 new jobs were created by the IT industry as compared to the target of 20,000 jobs at project closure. • Based on satisfaction surveys, 87% of the candidates expressed satisfaction with the effectiveness of Mexico First Training Program at project closure. This exceeded the revised target of 80%. There was no original target for this indicator as the indicator was added following the restructuring of the project. Rating Substantial PHREVISEDTBL Objective 2 Objective To improve the competitiveness and efficiency of IT companies. Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) Rationale Outputs. • 100 IT companies participated in the Financing of IT accreditation program at project closure as per the revised target and as compared to the original target of 250. • Six IT parks were supported by technical assistance and infrastructure as per the revised target. This exceeded the original target of four. • 15 government agencies were trained for outsourcing Government services at project closure. This exceeded both the original and revised targets of 12 and 10 respectively. Six feasibility studies on outsourcing Government services was completed at project closure. This exceeded both the revised and original targets of five and four respectively. • 796 public officials were trained on IT sector Legal Framework as compared to the target of 450. • Awareness raising activities were carried among local businessmen in several states on the benefits of outsourcing and the usage of IT in business and production processes, as targeted. • 16 IT clusters were supported by the PROSOFT program as compared to the revised target of 14. This was a new indicator added after restructuring. Outcomes. • 701 IT companies received certifications proving that they and their staff had the required capacities for competing in international markets at project closure. This exceeded the target of 390 companies. (The ICR (page 15) reports that many international clients will not engage with an IT company unless it has an internationally recognized certification). • The debt portfolio of IT companies increased from US$5.8 million at the baseline to US$47.20 million at project closure. This exceeded both the original and revised targets of US$20.00 million and US$36.00 million respectively. Rating Substantial 5. Efficiency Economic analysis. An economic analysis was conducted both at appraisal and at closure. The methodology entailed calculating potential project benefits which were assumed to come from the increase in employment in the IT sector and the downstream multiplier effects on Gross Domestic Product (GDP) due to the new income generated by the IT sector. The Net Present Value (NPV) at 15% discount rate in five years was US$152.5 million at closure as compared to the NPV of US$17.20 million estimated at appraisal. The ex post Economic Internal Rate of Return (EIRR) was 725% as compared to the ex-ante EIRR of 52%. The e post EIRR was substantially higher than the ex-ante EIRR as the number of IT jobs created was more than four times than estimated at appraisal. The other project benefits which were identified but not included in the economic included, benefits due to knowledge spillovers increase in productivity across other industries and citizen satisfaction. Administrative and Operational Issues. There were delays in establishing the Project Implementation Unit (PIU) in view of the changes in implementation arrangements and this delayed implementation. The loan was declared effective in April 2010, 21 months after board approval. Even with this delay, the rate of disbursement remained low until the 2012 restructuring due to a combination of factors including, the time needed by the participating entities to familiarize with Bank procedures, delays associated with implementation of key PROSOFT programs and procurement delays. However, following the first project restructuring, implementation accelerated with faster disbursements. The project did not incur cost overruns and project results were attained with a project budget lower than planned (US$73.5 million as compared to the budget of US$80.00 million). Despite initial delays, all project activities were implemented by the extended project closing date and the project leveraged funds State Governments and from IT companies that participated in the project activities. Capacity building and certification activities were provided with an actual cost about 30% lower than the market price due to economies of scale, as MexicoFIRST negotiated prices in bulk directly with international certification providers to get a 30% discount. Efficiency Rating Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  52.00 Not Applicable 0 ICR Estimate  725.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objective for Government and for the Bank strategy for Mexico was rated as High and relevance of design was rated as Substantial. Efficacy of the two sub-objectives was rated as Substantial. The revised targets were either realized or surpassed. Efficiency was rated as Substantial. The ex post was higher than the ex ante EIRR. Despite initial delays and time overruns, there were no cost overruns and all activities were implemented by the extended project closing date. The project was able to leverage funds from state governments and from IT companies that participated in the project and also generated cost savings due to economies of scale. a. Outcome Rating Satisfactory 7. Rationale for Risk to Development Outcome Rating Government Commitment. The risk to development outcome is rated as Low to Negligible. Government commitment to ICT sector through the PROSOFT program remains high as evidenced by the new strategy and plan for the 2014-2024 period. While the previous focus of PROSOFT programs was on development of the IT industry, the new strategy focuses on application of Information Technologies to foster innovation and greater productivity on other sectors of the economy. The ICR (page 20) reports that the PROSOFT program is receiving adequate budget allocations and given the wide base of ownership of this program which includes both public and private sector stakeholders, it is likely that the program will be adequately funded in the coming years. a. Risk to Development Outcome Rating Negligible 8. Assessment of Bank Performance a. Quality-at-Entry The project design was based on lessons from prior Bank financed ICT projects in the IT/ITES sector in countries like India, China, Malaysia and the Philippines and activities in this project were complementary and mutually reinforced another ongoing Bank financed project in Mexico (Innovation for Competitiveness Project). Risks were identified. Several risk mitigation measures were incorporated and the overall project risk was rated as Moderate at appraisal. Adequate measures were taken at appraisal for safeguards compliance. (discussed in section 11). The project design underestimated the capacity of the beneficiary agencies to manage financial management issues. The original design envisaged a financial management oriented plan to support MexicoFirst. However, given that there were many projects components and funds needed to be transferred to many beneficiary institutions, the design did not include capacity building activities to other beneficiary Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) institutions and this contributed to the delays in the initial years of the project. Quality-at-Entry Rating Satisfactory b. Quality of supervision Supervision missions were approximately twice a year, with 14 supervision missions between loan approval and closing. The consistent engagement of the senior Bank specialists in providing technical assistance on substantive matters aided in adjusting implementation to new circumstances and challenges. The supervision team was proactive in pursuing solutions to expedite project implementation, particularly after the recommendations of the Mid Term Review and the first project restructuring which entailed revision of targets. Tjis aided in speeding implementation since then. • The procurement delays caused by low prior review ceilings and delays in granting non-objections contributed to procurement delays. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The Government Commitment at preparation was demonstrated by providing a strong team and creating a new entity – MexicoFIRST- for developing the skills development program. The Government’s decision taken unilaterally to transfer the project management and the PROSOFT program from the original agency – The Directorate for Domestic Trade and Digital Economy (DGCIED) of the Ministry of Economy (SE) – to the Business Development and Opportunities General Directorate of SE, soon after Bank approval hindered project implementation for nearly three years (including a 21 months effectiveness delay and a further one year delay for revamping implementation). However, once the project was restored to its original unit, the pace of all components accelerated. Following the first restructuring, the Government commitment was further demonstrated through the consistent and direct support from the Deputy Minister and the Director of Digital Economy. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Team in the Directorate General of Domestic Trade and Digital Economy (DGCIED in Spanish) in the Secretary of Economy (SE) was in charge of implementing the project. Having managed the Development Program for the Software Industry (PROSOFT) Fund since 2003, the team had the required implementation capacity. DGCIED provided strong technical and administrative leadership for the project design and approval. The implementing agency was well-staffed, including for addressing the management, technical and procurement functions. The implementing agency was proactive in foreseeing potential shortcomings and proposing solutions (such as organizing capacity building activities for beneficiary entities and proposing a reallocation of funds toward other components when the IT parks lost relevance). Implementing Agency Performance Rating Satisfactory Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design There were three original key outcome indicators - -- IT/ITES jobs created by the IT industry as a result of training programs, overall debt portfolio of IT companies, IT companies satisfaction rating with the IT Links program. Of these three, the last indicator was dropped as the required data could not be collected. Following the project restructuring, key outcome and intermediate indicators were revised to better measure the impacts achieved by the project. The indicators and data collection were integrated into PROSOFT’s M&E system and database, rather than being a stand- alone system for the project. M&E implementation included tools such as comprehensive surveys of MexicoFIRST participants and this aided in tracking parameters such as industry demand trends. b. M&E Implementation M&E data were not reported in reports before 2014, due to the delays in effectiveness in the initial years of the project. c. M&E Utilization The M&E indicators were used for monitoring performance and the ICR (page 10) notes that the information generated by the project and the program’s M&E system, were used for designing PROSOFT 3. M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was classified as a Category B project. Three safeguard policies were triggered: Environmental Assessment (OP/BP 4.01): Involuntary Resettlement (OP/BP 4.12): Indigenous Peoples (OP/BP 4.10). Since the exact location, design, size or extent of the subprojects to be financed under the project (such as construction of Information Technology (IT) parks had not been determined at appraisal, an Environment and Social Management Framework (ESMF) that was consistent with Bank’s standards was prepared at appraisal (PAD, page 87). The issue of resettlement and displacement of indigenous peoples and appropriate measures on how to incorporate resettlement planning in the implementation of subprojects was addressed in the ESMF (PAD, page 89). The ICR (page 11) notes that with reduction of project scope following the 2012 restructuring, activity associated with construction of new IT parks was dropped and the safeguards compliance of the project was deemed to be satisfactory. b. Fiduciary Compliance Although fiduciary performance was affected by the decision to transfer the project to a different unit from the original one in the initial years of the project (discussed in section nine), these were rectified during implementation. Financial Management. The ICR (page 11) reports that the quality of financial reports were deemed to be satisfactory and financial reports and Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) audits were submitted in a timely fashion. Procurement. The ICR (page 11) notes that there were procurement delays and there were numerous instances of consultants selected out of compliance with Bank requirements. The ICR however provides little details of procurement issues during implementation. c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Risk to Development Outcome Negligible Negligible --- Bank Performance Moderately Satisfactory Moderately Satisfactory --- Borrower Performance Moderately Satisfactory Moderately Satisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR draws the following main lessons from implementing this project. (1) Mechanisms for strong private sector involvement throughout the entire project cycle can be very useful ICT projects. In the case of this project, active participation of private sector stakeholders at preparation enabled a comprehensive project design that took into account industry needs. Likewise, the project benefited from active involvement of industry chambers, association and academia, particularly during MTR and the first restructuring of the project which resulted in revising the indicators. (2) The IT/ITES sector relies on the availability of a talent pool that is conversant with English language. In the case of this project, although English skills was not envisaged at project design, it was identified as key for exporting IT services during implementation. This necessitated a component of capacity building for imparting English language skills. (3) A tailored capacity building for procurement for stakeholders can alleviate procurement delays, especially if capacity is build at early stages. In the case of this project with a various stakeholders, organizations such as the IT chambers – although familiar with Government procurement process – had limited knowledge of Bank procedures and this contributed to delays in the early years of the project 14. Assessment Recommended? No Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MX IT Industry Development Project (P106589) 15. Comments on Quality of ICR The ICR provides a detailed overview of the ICT project. The narrative supports the ratings and available evidence. It is candid especially discussing the problems that arose in the early years of the project and is focused on results. The quality of analysis is aligned to the messages outlined in the ICR, although there are some minor shortcomings in the completeness of information. For instance, it would have been useful to provide more information on the methodology followed in administering the satisfaction survey. a. Quality of ICR Rating Substantial