WPS5186 Policy Research Working Paper 5186 Mexico Impact Evaluation of SME Programs Using Panel Firm Data Gladys Lopez-Acevedo Monica Tinajero The World Bank Latin America and the Caribbean Region Poverty and Gender Unit January 2010 Policy Research Working Paper 5186 Abstract Unlike social programs targeting individuals, few data--from the 1994­2005 period--allow the authors to enterprise support programs have been rigorously address selectivity bias and unobserved firm heterogeneity evaluated, and existing evaluations have mostly been by applying a generalization of differences-in-differences done in high-income countries such as the United States models combined with propensity score matching and Europe. Mexico spends a large share of government methods. This study finds evidence that participation in resources on small and medium enterprise programs small and medium enterprise programs is associated with each year. How effective these programs have been in improvements in key variables such as value added, gross achieving their objectives is unclear. In Mexico, impact production, and wages. Furthermore, the study finds evaluations of small and medium enterprise programs evidence that some of the positive effects can take several are rare, and most are qualitative in nature. This is the years to realize. The results also call for streamlining first paper evaluating these programs in Mexico using and greater efficiency in Mexico's small and medium firm-level panel data. The continuous and ten-year panel enterprise programs. This paper--a product of the Poverty and Gender Unit, Latin American and the Caribbean Region--is part of a larger effort in the department to increase knowledge on impact evaluation and on firm's productivity. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at gacevedo@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team MEXICO: IMPACT EVALUATION OF SME PROGRAMS USING PANEL FIRM DATA 1 Gladys Lopez-Acevedo (Senior Economist, LCSPP) Monica Tinajero (Consultant, World Bank) Keywords: small and medium enterprises, program impact evaluation JEL codes: L25, L53, O25, O54 Contact information: gacevedo@worldbank.org 1 This country study was funded by a research grant RF-P105213-RESE-BB from the World Bank's Research Committee for the project "Evaluating Small and Medium Enterprise Support Programs in Latin America" managed by LCSPP and directed by Gladys Lopez-Acevedo and Hong Tan. We acknowledge the close collaboration offered by Abigail Durán and Adriana Ramirez from the headquarters of the Mexican National Statistics Office (Instituto Nacional de Estadistica y Geografia--INEGI). We also acknowledge the contributions and background paper on the "Review of SME Programs" from Jorge Mario Soto, Hugo Fuentes and Victor Aramburu (ITESM). We acknowledge the technical guidance from Hong Tan and his paper on "Evaluating SME Support Programs in Chile," forthcoming as a World Bank Research paper. We thank David McKenzie in particular for his expert guidance on key methodological issues that arose over the course of the regional study. 1.- INTRODUCTION Microenterprises and SMEs make up 99 percent of enterprises in Mexico, employ about 64 percent of the workforce, and account for over 40 percent of GDP. Given the importance of SME in the economy, governments in Mexico over the last 20 years have put in place a wide variety of SME support programs. An inventory of programs--an important contribution of this study, as such information is not generally available--identified 151 SME programs administered by different government agencies. How effective these SME programs have been in achieving their objectives is unclear. In Mexico, impact evaluations of SME programs are rare--most are qualitative in nature and narrow in scope, either measuring beneficiary satisfaction with support services or easily quantified program coverage indicators. None take into account the complicated biases from unobserved firm heterogeneity and self-selection that plague efforts to measure the true impacts of program participation. This paper uses econometric techniques to estimate the net impacts of SME program participation on outcomes. Techniques used include regressions models with program participation indicators and controls for firm characteristics, propensity score matching techniques, and a generalization of difference-in-differences (DID) methods using pre- and post-program panel data adapted to the nature of our panel data. The panel data is created by linking SME program participation information (which identifies the treatment and control groups) to a large panel of annual industrial surveys (1994-2005) maintained by Mexico's National Statistics Office (Instituto Nacional de Estadística y Geografía--INEGI). Impacts are estimated at the agency level (groups of programs run by a given agency or ministry) and in selected cases by program, when data allowed for estimations to be carried out. The paper finds that program participation in certain types of SME programs is associated with higher value added, sales, export, and employment. The positive impact associated with firm participation in SME programs is strongest in the programs of the Ministry of Economy (ME) and the National Science and Technology Council (Consejo Nacional de Ciencia y Tecnología--CONACyT). Our panel data allows us to identify the timing of the effects of program participation on outcomes. We found positive and significant impact for firms participating in SME programs that range from 5 percent in value added to 6 percent in employment, among other outcomes. These results remain robust after trimming the 5 percent of the treatment group for each outcome variable. Some of these positive effects materialize after the third or fourth year after exposure into the program. We found that the programs in the ME and CONACyT have the largest impact on outcomes. CONACyT's fiscal support and technical innovation program had a positive and significant effect, with an estimated increase in value added of 14 percent and an increase in employment of 10 percent. The ME's sectoral promotion program and the Environmental Audit Program also showed positive and statistically significant impacts on firm performance. The paper is organized as follows. Section 2 describes the many SME policies and programs in Mexico. The work presented in this section is unique, since information on SME program budget and beneficiaries is not easily available. Section 3 reviews existing evaluations in Mexico. The evaluations reviewed here are not exhaustive, but they do provide an overview 2 of past evaluations of SME programs. Section 4 presents the data used in the analysis, which comes mainly from establishment surveys and includes information on many of the main programs and policies discussed in Section 2. Section 5 discusses the methodology employed in the paper, which follows the approached used in the regional study suitable for firm panel data. Section 6 presents the results and indicates which programs and agencies have improved the performance of SME programs through their interventions. Section 7 offers conclusions and future lines of work. 2.- SME PROGRAMS From 2001 to 2006, the Government of Mexico invested heavily on SME support programs, providing about US$80 billion to about 3.7 million SME firms (Table 1). As part of this study, an inventory of programs supporting SMEs was undertaken, as consolidated information is not available publicly. The inventory tallied 151 separate programs run by numerous different public agencies, some with overlapping mandates and target groups. Although the econometric study does not attempt to evaluate the impact of all these programs, this section provides an overview of the goals, coverage and budgets of the main programs, as a contribution to assist further analysis and evaluation of Mexico's SME supports. The development bank Nacional Financiera (Nafinsa) contributed more than half of the total resources to SME programs and seemed to have a significant coverage among firms. The Ministry of Economy (ME) comes second in terms of coverage, supporting more than one million firms, but providing less than 1 percent of the total SME resources. Banco Mexicano de Comercio Exterior (Bancomext) ranks second in terms of resources (about 43 percent of the total), however its programs covered less than 1 percent of the firms. The Ministry of Labor (Secretaría de Trabajo y Provisión Social--STPS) supports about 6 percent of SME firms, but with a very low budget. On the other hand, CONACyT programs had a significant budget but limited coverage among firms. An overview of the main characteristics of major SME programs in Mexico is provided below, and full analysis of the rules of operation of the programs is available upon request. 3 Table 1 SME Support Funds and Programs in Mexico: Summary of Results, 2001-2006 Institution Concept 2001 2002 2003 2004 2005 2006 Total Budget Ministry of 94 150 94 91 168 185 782 (Million USD) Economy Firms 70,136 86,142 142,514 286,191 299,731 N/A 884,714 Budget 1,718 3,774 5,939 8,278 10,566 13,137 43,412 Nafinsa (Million USD) Firms 90,574 230,442 353,428 429,136 562,722 877,103 2,543,405 Budget 5,512 6,993 6,177 5,423 5,392 4,952 34,449 Bancomext (Million USD) Firms 1,628 1,807 1,202 751 533 N/A 5,921 Budget 44 64 65 107 304 393 977 Conacyt (Million USD) Firms 150 422 480 583 917 1,146 3,698 Budget Ministry of 19 14 10 10 11 11 75 (Million USD) Labor Firms 60,386 35,376 26,130 33,746 33,770 36,919 226,327 Budget 7,387 10,995 12,285 13,909 16,441 18,678 79,695 Total (Million USD) Firms 222,874 354,189 523,754 750,407 897,673 915,168 3,664,065 *This table does not includes specific interventions of the institutions nor inter-institutional interventions such as the National Guarantee System. Source: Bancomext (2006a and 2006b); Conacyt (2003, 2005, 2006 and 2007); Nacional Financiera (2002, 2003, 2004, 2005, 2006a and 2006b); Secretaría de Economía (2006c); and Secretaría del Trabajo y Previsión Social (2006). Nacional Financiera (Nafinsa) Nafinsa is Mexico's largest development bank. Traditionally focused on large industrial projects with direct subsidies, in the 1990s the bank started to shift increasingly towards smaller firms and second-floor operations, and in 2001 it underwent a deep restructuring to set SME support as its top priority. Nafinsa provides credit and guarantees, training and technical assistance, and Internet-based information services to SME firms. Table 2 Nafinsa: Main results 2001-2006 Private sector credit Firms supported (million USD) Year Business Development Electronic Credit and Program (training and Information Total SME Total guarantees technical assistance) Services 2001 N/A N/A N/A 90,574 1,718 2,684 2002 54,130 83,920 92,392 230,442 3,774 5,066 2003 90,566 158,377 104,485 353,428 5,939 7,615 2004 139,599 211,191 78,346 429,136 8,278 10,309 2005 397,179 99,366 66,177 562,722 10,566 12,886 2006 643,062 166,780 67,261 877,103 13,137 15,640 Total 1,324,536 719,634 408,661 2,543,405 43,413 54,199 Source: Nacional Financiera (2002, 2003, 2004, 2005, 2006a and 2006b) 4 Ministry of Economy From 2000 to 2006, ME substantially increased its budget for SME support programs, from US$22 million to US$185 million, while coverage rose from 13,000 to 300,000 SMEs. Table 3 SME funds and programs from the Ministry of Economy: Main Results 1998-2006 Budget (million Year Firms benefited USD) 1998 9 4,332 1999 13 10,326 2000 22 13,081 2001 94 70,136 2002 150 86,142 2003 94 142,514 2004 91 286,191 2005 168 299,731 2006 185 N/A Total 782 884,714 Source: Secretaría de Economía (2006c). In 2001, ME launched a strategy to promote SME competitiveness using four funds, which in 2004 were merged into a single Support Fund for SMEs (Fondo PYME): · Fund to Promote the Integration of Productive Chains (Fondo de Fomento a la Integración de Cadenas Productivas--FIDECAP) sought to encourage and strengthen vertical and horizontal linkages of SME with other firms through subsidies for productive projects, industrial infrastructure and the formation of Business Linking Centers (Centros de Vinculación Empresarial). From 2001 to 2006, FIDECAP and Fondo PYME supported 2,207 projects that benefited over 597,000 firms, with a total investment of US$387 million. · Support Fund for Micro, Small and Medium Enterprises (Fondo de Apoyo para las Micro, Pequeña y Mediana Empresas--FAMPYME) was designed to increase the competitiveness of SME, with support mechanisms consisting of subsidies for feasibility studies, training and consultancy services, as well as databanks and Business Linking Centers. · Support Fund for Access to Finance (Fondo de Apoyo para el Acceso al Financiamiento--FOAFI) and Fund for the Program of Distribution Centers in the United States (Fondo de Programa de Centros de Distribución en Estados Unidos-- FOCOE) were two small and short-lived funds. The former sought to facilitate access to finance for SME through guarantee funds and financial extension services, while the latter helped SMEs export to United States through training and consultancy services, marketing, databanks, and other services. · Sector Promotion Programs (PROSEC) of the ME provides tax breaks for firms in specific sectors to import certain goods used for manufacturing products approved by the ministry. The imported products should be used for the production of goods regardless of whether they are intended for the internal or external markets. 5 When all of these funds consolidated, Fondo PYME kept most of their support mechanisms, within a single, less complicated operational scheme. Between 2001 and 2006, the total resources allocated to these funds rose from US$41 million to US$205 million, and the number of supported SMEs went from 62,000 to 166,000. Table 4 Funds of the Ministry of Economy: Main Results 2001-2006 2001 2002 2003 Total Fund Million Million Million Million Firms Firms Firms Firms USD USD USD USD FAMPYME 25,692 25 21,141 23 12,745 15 59,578 63 FIDECAP 36,342 16 23,107 60 24,594 125 84,043 201 FOAFI * * 23,362 15 48,917 18 72,279 33 FACOE * * * * 6,606 7 6,606 7 Total 62,034 41 67,610 98 92,862 165 222,506 304 2004 2005 2006 Total Fondo PYME 177,393 105 130,281 168 165,937 205 473,611 478 * The fund did not exist in the specified year. Source: Secretaría de Economía (2006a). ME manages other programs related to SME support, such as the National System of Business Incubators, which had a total of 308 institutions supporting firms, and the Program of Training and Modernization (Programa de Capacitación y Apoyo Empresarial-- PROMODE) to increase the competitiveness of small retailers. Table 5 PROMODE: Main Results 2001-2006 2001 2002 2003 2004 2005 2006 Total Workers trained 91,240 176,506 146,142 156,014 192,875 94,431 857,208 Entrepreneurs trained 9,012 16,907 12,562 6,170 12,581 9,644 66,876 Instructors trained 2,593 1,512 1,492 1,246 1,433 404 8,680 Total individuals trained 102,845 194,925 160,196 163,430 206,889 104,479 932,764 Total SME benefited N/A N/A 16,528 18,889 17,691 19,113 72,221 * Data from 2001 to June 2006. Source: Secretaría de Economía (2007c) Another program run by ME is the Network of Centers for the Development of Business Competitiveness and Regional Centers for Business Competitiveness (CETRO-CRECE). This was created in 1996 to support Mexican SMEs development by identifying their needs and offering them the appropriate tools to become competitive. During its first years CETRO-CRECE was entirely financed with federal funds. However, by 2003, its financial structure had changed to 59 percent federal subsidies, 30 percent from clients, 8 percent from other private entities and 3 percent from state and municipal governments. It is the largest institution in Mexico offering consultancy services and training. Other types of support mechanisms the network provides include elaboration of feasibility studies, marketing strategies, sector and financial analyses, human resource studies, analysis and oversight of the productive process, management analysis and oversight, and business training. The CETRO-CRECE network supported 29,466 firms between 2001 and 2003 and the budget for the network during that period was 625.9 million pesos. 6 Lastly, ME runs the National Committee for Productivity and Technological Innovation (Comité Nacional de Productividad e Innovación Tecnológica--COMPITE), which provides SME consulting and training services. The objective of the program is to help SME to be more competitive by increasing productivity and improving quality. COMPITE consultants use best-practice business development and management models to improve the management of firms. Between 2001 and 2006, COMPITE supported close to 50,000 firms, which had to pay part of the cost of the services. Table 6 COMPITE: Main Results 2001-2006 2001 2002 2003 2004 2005 2006 Total Services 12,949 13,226 14,192 12,512 16,901 1,338 71,118 SME 6,585 8,550 10,184 9,168 13,746 778 49,011 Hours 82,804 76,003 204,450 238,912 291,097 28,426 921,692 Participants 42,005 35,394 62,876 57,195 65,881 14,390 277,741 SME with ISO 9000 certifications 307 238 277 227 257 195 1,501 * Data from 2001 to June 2006. Source: Secretaría de Economía (2007c) Bancomext Banco Nacional de Comercio Exterior (Bancomext) is the country's export-import development bank. Bancomext supports the participation of Mexican firms in global markets, providing financing, training and technical assistance. Bancomext targets SMEs with export sales of up to US$20 million per year. The bank's major support mechanism is direct medium and long-term credit to exporting firms. In addition, the bank offers direct and Internet-based information services about foreign markets and exporting procedures and requirements for Mexican firms. The bank also promotes the participation of Mexican SMEs in international trade fairs to promote foreign investment in Mexico. Table 7 Bancomext: Main Results 2001-2006 Resources mobilized (million USD) Firms benefited Year Insurance and Loans Total SME Total guarantees 2001 4,893 619 5,512 1,628 1,716 2002 5,442 1,551 6,993 1,807 1,866 2003 5,435 742 6,177 1,202 1,257 2004 4,986 437 5,423 751 826 2005 5,254 138 5,392 533 584 2006 4,865 87 4,952 N/A N/A Total 30,875 3,574 34,449 5,921 6,249 Source: Bancomext (2006a and 2006b) Bancomext has two main programs: Crediexporta and the Technical Assistance Program (Programa de Asistencia Técnica--PAT). Crediexporta is Bancomext's most important financing program mainly targeted to SMEs. The program has five different modalities: (i) 7 working capital, (ii) investment projects; (iii) buyer loans; (iv) guarantees; and (v) credit letters (a guarantee payment instrument). PAT subsidizes 50 percent of the cost of specialized consulting and technical services to help SMEs improve productivity and competitiveness in foreign markets. Some of the specific areas supported by the programs were international certifications, promotion materials, export business plans, and promotional campaigns abroad. From 2001 to 2005, the PAT channeled over US$9 million to more than 7,000 firms, supporting exports of about US$500 million. However, the budget of the program decreased from US$1 million to less than US$100,000 between 2001 and 2005, and coverage dropped from 3,000 to 835 firms over the same period. CONACyT CONACyT is a decentralized agency of the federal government responsible for science and technology policies. The agency is in charge of promoting the accumulation of scientific development and technological modernization through human resources, the support of specific research projects and the diffusion of information about science and technology. Between 2001 and 2006, CONACyT implemented several programs to boost technological capabilities of Mexican firms: 1) the Technological Modernization Program (Programa de Modernización Tecnológica--PMT); 2) the Science and Technology Sectoral Fund for Economic Development; 3) the High Value-Added in Businesses with Knowledge and Entrepreneurs program (Alto Valor Agregado en Negocios con Conocimiento y Empresarios--AVANCE); 4) Mixed Funds Program; 5) the Knowledge and Innovation Program (Programa de Conocimiento e Innovación--PCI); 6) the Support Research and Development in Projects Program (Programa de Apoyo a Proyectos de Investigación y Desarrollo Conjuntos--PAIDEC); and 7) the fiscal incentives program. These programs are detailed below. The PMT was established in 2001 to provide fiscal credits up to 30 percent per year for firms that made investment in research and development. From 2001 to 2006, PMT's coverage increased from 90 to 738 firms. In 2006, the program's budget was US$873 million, of which 40 percent was allocated to SMEs. Table 8 Fiscal Incentives: Main Results 2001-2006 Large firms SME Total Incentives Incentives Incentives Year Firms Projects (million Firms Projects (million Firms Projects (million USD) USD) USD) 2001 60 315 32 90 233 13 150 548 44 2002 76 429 36 125 358 16 201 787 51 2003 102 508 30 143 150 16 245 658 46 2004 132 N/A N/A 225 N/A N/A 357 1,308 89 2005 202 N/A N/A 411 N/A N/A 613 2,083 275 2006 316 N/A N/A 738 N/A N/A 1,054 3,317 367 Total 889 N/A N/A 1,731 N/A N/A 2,620 8,701 873 Source: Conacyt (2003, 2005, 2006 and 2007) 8 In 2002, CONACyT and ME created the Science and Technology Sectoral Fund for Economic Development to support technological innovation in products, materials and manufacturing processes. The fund supported firms in a variety of sectors with subsidies worth US$60 million to 911 firms from 2002 to 2005. Data are not available on the size of the firms supported by the fund. Table 9 Science and Technology Sectoral Fund: Main Results 2002-2006 Firms Investment through Support from the Year benefited contributions (million USD) Fund (million USD) 2002 221 270 13 2003 235 151 19 2004 197 126 11 2005 258 159 16 Total 911 706 59 Source: Conacyt (2003, 2005, 2006 and 2007) In 2003, CONACyT created the AVANCE program as a means to identify business opportunities and promote firms based on scientific or technological developments. Although small in size and resources, AVANCE has nine modalities: New Businesses, Entrepreneurship Fund CONACyT-Nafinsa, Guarantee Fund, Patent Support, Technological Packages, Technology Transfer Offices, AVANCE Business Schools, Strategic Alliances and Innovation Networks for Competitiveness, and Seed Capital. AVANCE's coverage of SMEs increased from 29 firms in 2004 to 92 firms in 2006, with program resources growing from US$7 million to US$26 million over the same period. Table 10 AVANCE: Main Results 2004-2006 Resources Resources Total Total resources Year SME Large (million USD) (million USD) firms (million USD) 2004 29 7 2 0.7 31 8 2005 46 13 2 0.8 48 14 2006 92 26 2 0.8 94 27 Total 167 47 6 2.4 173 49 Source: CONACYT (2003, 2005, 2006 and 2007) Mixed funds are instruments to support scientific and technological development in states and municipalities through trusts that are constituted with resources from the three tiers of government. These funds are promoted by an open national competition scheme and the selection is based on merit and quality. Categories include: (i) applied research; (ii) technological development; (iii) reinforcement of infrastructure; (iv) dissemination; (v) creation and consolidation of research groups and networks; and (vi) comprehensive projects. PCI started its operations in 1998 with a budget of approximately US$500 million for 1999- 2003. Of these resources, 61 percent was for supporting science, 17 percent for technology and 22 percent to establish linkages. The PCI contributed with 40 percent of the resources and the World Bank financed the rest. Its objectives were to promote Mexico's science and 9 technology system, to strengthen links between participants in the national innovation and to contribute to improved productivity, competitiveness and economic growth. PAIDEC was created to improve cooperation between firms and institutions of higher education by encouraging firms to use available knowledge in research and industrial development projects. Eligible research projects included: applied research, design and development of products and processes and development, and adaptation and improvement of existing technologies. CONACyT granted up to 50 percent of financial support to research projects, with a limit of US$250,000. Resources could be used for raw materials, consultancy services, training, project-related travel, equipment for the project (with a maximum of 15 percent of total resources), salaries of researchers and technicians who participated in project patents, and acquisition of literature and databases. The fiscal incentives program of CONACyT was directed towards taxpaying firms that have invested in research projects and technology development in order to develop new projects, materials and processes. The main objective of this program was to increase the annual investment and spending of firms on those three items by allowing firms to recover up to 30 percent of their annual investment. STPS In 1998, Mexico launched the Integral Quality and Modernization Program (CIMO), which was mainly an in-firm training program. The program was implemented and administered by the STPS. In 2002, the program was decentralized to the states under a new name, Training Support Program (PAC). The program provides subsidies to training and technical assistance to SMEs in order to enhance their productivity and welfare of workers. In 2009, PAC changed once again its design, and was rename the Productivity Support Program (PAP). Table 11 CIMO-PAC: Main Results 2001-2006 Workers Firms benefited Subsidies Year trained Micro Small Medium Total (million dollars) 2001 333,474 47,705 7,911 4,770 60,386 19 2002 201,233 27,947 4,634 2,795 35,376 14 2003 210,746 26,130 0 0 26,130 10 2004 299,656 23,622 6,547 3,577 33,746 10 2005 279,725 11,620 12,222 9,928 33,770 11 2006 265,041 17,090 10,585 9,244 36,919 11 Total 1,589,875 154,114 41,899 30,314 226,327 75 Source: Secretaría del Trabajo y Previsión Social (2006) From 2001 to 2006, CIMO-PAC trained about 1.6 million workers, benefiting more than 226,000 firms with an estimated annual budget of about US$11 million. The precise number of workers and firms which benefited from the program is not clear since they could apply several times to receive the courses and the subsidies. Table 11 shows one trend worth noting is that PAC sought a decline in their coverage of workers (21 percent), firms (39 percent) as well as resources (29 percent). 10 Programs by Other Ministries and Agencies Two notable SME-related programs run by other agencies include the National Environmental Audit Program (Programa Nacional de Auditoría Ambiental--PNAA) and the Labor Qualification Certification Council (Consejo de Normalización de Competencias Laborales--CONOCER). PNAA was created in 1992 under the supervision of the Federal Environmental Protection Attorney (Procuraduría Federal de Protección al Ambiente--PROFEPA). This voluntary certification program promotes environmental audits in firms in order to acquire knowledge about how their operations generate pollution, and environmental risks, and how they could comply with environmental regulations and apply best-practices. The program aims to improve the impact of firms on the environment and generate savings through a more efficient use of raw materials and other inputs. At the end of the environmental audit, the firm signs an agreement including an action plan with the main activities and the time needed to make its processes more environmentally friendly. Between 2000 and 2006, PNAA initiated 4,147 audits, signed 2,285 action plans, initiated 7,390 investments, and granted 1,773 "clean industry" certificates. CONOCER is an organization under the Ministry of Education (SEP) that certifies labor skills. It is made up of workers, businessmen, educators, trainers and federal government representatives. CONOCER promotes workers' development through the evaluation and certification of their knowledge, abilities and skills. The main objective of CONOCER is the development and promotion of a skills certification in order to establish general guidelines and define technical rules related to labor skills. CONOCER offers three types of support mechanisms: (i) technical rules for labor skills, (ii) evaluation instruments, and (iii) certification processes. Summary and Overview of Mexico's SME Programs In sum, from 2001 to 2006 there was an important increase in public resources to support SME programs. During this period, agencies and programs increased considerably their coverage. Examples of such increase and expansion of programs include ME, CONACyT, Nafinsa. However, other ministries sought their SME resources decline in the same period, for reasons that are not entirely clear. One possible explanation is that STPS programs were decentralized to the states in 2001, thus reducing the federal fiscal burden. As is immediately apparent from this overview, Mexico has a huge array of programs involved in supporting SMEs in some way. This in itself is an important finding, and suggests that Mexico would do well to design a more coherent framework to orient resources more efficiently and strategically, and avoid program overlap. As well, the lack of easily available information on program budgets, activities and beneficiaries points to a need for improved consolidation of information on SME support, to improve the ability of policymakers to make evaluations and comparisons, and ensure that the country's scarce fiscal resources are achieving the greatest possible impact. Among the various topics these programs address are the following categories (Table 12): (i) technical assistance and training (TAT); (ii) modernization, innovation and technical development (MITD); (iii) knowledge 11 and information exchange (KIE); (iv) fiscal incentives (FI); (v) financial products (FP); (vi) business promotion and opportunities (BPO); and (vii) other types of support (O). Because most programs have a broad set of objectives and types of support mechanisms, most programs fall in more than one category. This is important to bear in mind when discussing the evaluation strategy of this paper, and was an important factor in choosing to evaluate programs first by agency and then, when data permitted, by individual program. Table 12 Programs and support mechanisms Program TAT MITD KIE FI FP BPO O CIMO-PAC x FIDECAP x x X FAMPYME x x x x X Fondo PYME x x x x X X CETRO- x X X CRECE COMPITE x PROSEC x PROMODE x x X PAT x CrediExporta x PNAA X Sector Funds x Mixed Funds x PMT x x PCI x x PAIDEC x x Fiscal x x Incentives CONOCER X Source: Authors' compilations 3.- PAST EVALUATIONS 2 There are a few rigorous program evaluations of SME programs in Mexico. Most evaluations have focused on client satisfaction or program outcomes, such as the number of supports given to beneficiaries. The variations in methodologies have made the comparison of results between and within programs difficult. Several external evaluations of SME programs have been undertaken, with the evaluation methodology as agreed upon by the supervising body and the external evaluators. For example, for FAMPYME and FIDECAP, the Ministry of the Economy hired the National Polytechnic Institute (Instituto Politécnico Nacional--IPN) and 2 World Bank (2007) includes a detailed review of evaluations of SME programs in Mexico. 12 the Economic Administrative and Social Research Center. For the evaluation of CETRO- CRECE, the Ministry of the Economy commissioned the Institute of Engineering at the National Autonomous University of Mexico (Instituto de Ingenieria de la Universidad Nacional Autónoma de México--IIUNAM). Past program evaluations have varied considerably in design and methodology, in part due to vague legal requirements concerning the type of evaluation required. The evaluation methodologies used can be categorized into three main groups in order of the statistical rigorousness of their designs (Table 13): a) non-experimental evaluations (CIMO-PAC, CONOCER, and a recent terms of reference for Fondo PYME); b) operational evaluations (CRECE, FIDECAP, FAMPYME, and PAT); and c) case studies (PMT, PAIDEC and COMPITE). Table 13 Evaluation Studies in Mexico Program Responsible Date Evaluating institution Type of Analysis Evaluated agency completed Comparison with match CIMO STPS STPS 1995, 1997 firms Comparison with match CIMO STPS Alduncín y Asociados 2002 firms Comparison with match CIMO STPS World Bank 2005 firms CONOCER STPS Grupo de Economistas y 2001 Comparison with match Asociados firms Lateral Investment Fund from Recipient views of program COMPITE SE 2003 IDB* impact Recipient views of program CRECE SE IILUNAM 2003 impact Recipient views of program FIDECAP SE IPN 2003 impact Recipient views of program FAMPYME SE IPN 2003 impact Recipient views of program PMT CONACyT CONACyT 2000 impact Recipient views of program PAT BANCOMEXT BANCOMEXT 2000 impact 13 Recipient views of program PAIDEC CONACyT CONACyT 2000 impact *IDB--Inter-American Development Bank. Early Program Evaluations All four evaluations of CIMO-PAC (1995, 1997, 2002, 2005) and the CONOCER evaluation can be characterized as non-experimental by their use of an ex-post matching to form comparison groups in order to isolate the causal impact of program participation. In all cases except CIMO-PAC's 2002 evaluation, random sampling was unnecessary because the universe of firms or workers meeting evaluation criteria were so few. In the 1995 and 1997 evaluations of CIMO, the control group was made up of 316 firms from the economic census that shared size, sector and location characteristics with those of the pilot treatment group. The treatment group consisted of all firms that entered the program in 1991 and 1992 and remained registered in 1993. Though 442 firms met these criteria, only 248 contained the necessary information for evaluation. This may have led to selection bias, because only the best-managed firms who kept organized records were used in the evaluations. This in turn could have led to overestimates of the effectiveness of the program. Similarly, the control group used in the CONOCER evaluation was selected from the same sectors of activity and region as the treatment group, with the intention of matching firm characteristics. The treatment group consisted of firms of those sectors and regions in which the greatest number of labor competency certificates had been issued. The group was selected as such because in 2001 only a few sectors had completed the cycle of supports provided by CONOCER. In the CIMO-PAC 1995 evaluation, a series of regressions were run on productivity, employment and wages using dummy variables to isolate the effect of the program by sector, type of support and firm size. The CIMO-PAC 1997 analysis incorporated regression based on a Cobb-Douglas production function. In the case of CIMO-PAC 2002 evaluation, the analysis consisted of comparing the percentages of responses of the survey questions between the treatment and control groups. Cost benefit studies were also carried out in the CIMO-PAC evaluation (1995 and 1997). In each case the treatment and control groups were compared based on the unit cost per unit impact. Cost benefit indices were calculated based on impact measures divided by private and public costs associated with the program. The evaluations produced mixed results in relation to program outcome and impact objectives. Some programs seemed to have more impact than others, and notable differences were found in the impact of firms of different sizes and in different sectors. For instance, the CIMO-PAC (1995) evaluation concluded that the treatment group's employment increased 8.5 percent while the control decreased 1 percent between 1991 and 1993. The CIMO-PAC (2002) evaluation concluded that while 16.5 percent of the treatment group's sales were exported, the control group firms only exported 9.7 percent of their sales over the same period. The CONOCER evaluation found large differences in outcome indicators between certified and non-certified workers in the forestry and textile industries, but the differences 14 were almost insignificant in the tourism industry. Thus, program performance may be tied to size and sector differences--an issue that should be the subject of further study. Evaluations of both CIMO-PAC and CONOCER administered questionnaires to beneficiaries and members of the control group. CIMO-PAC questionnaires collected quantitative information on personnel, payroll, training, fixed assets, and inventories, as well as qualitative information on productive characteristics of the firms, market, organization, employment, remunerations, and training. The CONOCER questionnaires gathered socio- demographic characteristics, work experience and on-the job training for workers, as well as quantitative data on the production and employment of beneficiaries. For workers, the evaluation focused on gathering indicators related to the objectives of the program such as increases in certification of labor competencies, wages, job-mobility, increased skills, reduced turnover and improved working conditions. They also collected qualitative data with regards to business performance, human resource management and labor competition. The qualitative surveys of CIMO and CONOCER indicated that most beneficiaries are in general satisfied with the programs. However, these results differ between firms of different sizes and firms within different sectors. Medium-sized enterprises tend to be the most satisfied, while small and micro firms perceive that the program supports are insufficient. For instance, in the CIMO-PAC (1997) evaluation it was noted that while the benefits of the supports outweighed the costs for medium-sized enterprises, costs outweighed benefits for small and micro firms. Evaluations of FIDECAP (2003) and FAMPYME (2003) also surveyed beneficiaries to assess the impact of the programs. Only 40 percent of FIDECAP firms said the program resources influenced their performance. The FAMPYME evaluation concluded that beneficiaries were indifferent in their view of whether the program increased employment and production growth. Panel Evaluations Tan and Lopez-Acevedo (2005) used panel data (1991-1996) collected by the Ministry of Labor on two cohorts of CIMO participants and a control group, to evaluate more rigorously the net impacts of CIMO on SME performance. Two previous evaluations of CIMO by the STPS found seemingly contradictory results: evidence of improvements in intermediate outputs (worker training, production processes and adoption of quality control) but no or negative impacts on productivity. While these findings may simply reflect poor design and implementation, they may also be the result of self-selection of low-productivity SMEs into the program. The authors tested this hypothesis using difference in difference methods (DID) to remove pre-intervention productivity differences between groups, and found a sign reversal of the program impact coefficient, with CIMO participants now enjoying a 6-11 percent net performance gain in the 1994 to 1996 period. In a second Mexico study, Tan and Lopez-Acevedo (2007) evaluated CIMO, CRECE and COMPITE using three rounds of enterprise surveys (ENESTYC 1995, 1999 and 2001) with information on program participation linked to annual panel industry surveys fielded by INEGI. These data were used to identify different cohorts of the treatment and control groups matched on propensity scores of pre-program attributes. All three programs showed net gains on intermediate outcomes 15 such as training, adoption of new technology and use of quality control methods. While improvements in these intermediate outcomes were supposed to translate into improved performance, the authors did not find statistically significant net gains in wages, export orientation or productivity. The authors concluded that greater attention to differential treatment "doses"--for example, more versus less training or type of consulting services provided--larger sample sizes, and tracking beneficiaries over a longer horizon may be needed to determine whether these interventions have the hypothesized positive performance impacts. Evaluations in progress With respect to Fondo PYME, in 2009 a team from the Instituto Tecnológico de Monterrey (Mexico City campus) carried out the first evaluation that included aspects of design, processes, perceptions and impacts (ITESM-CCM 2009). The evaluation assessed the fund's capacity to create and maintain formal firms and formal jobs, as well as its potential to increase productivity and sales for established firms, using a methodology based on data envelopment analysis. The study found positive impacts on the productivity of beneficiary firms of about 4 percent on average one year after treatment with respect to non-beneficiary control firms. Business incubators, business accelerators, supplier development, productive projects and seed capital were the support mechanisms with greater impact on productivity, in some case with more than 10 percent. Fondo PYME also showed positive impacts on sales of about US$19,000 in the year following treatment. The evaluation should be taken with reservation because of possible biases due to self-selection and unobserved firm heterogeneity that could have contributed to the positive results. As shown in Section 4, these biases could lead to over-estimating the impact of SME programs. Summing up There are wide variations in evaluation design and methodology, and few programs in Mexico have had impact evaluations, hindering the comparison of programs on the basis of their effectiveness and efficiency. Most of the evaluations have been quantitative, and in those cases with impact evaluations there had been some doubts about the quality of the control group and the methodology. The most rigorous impact evaluation is by Tan and Lopez (2005), but it was limited by small sample sizes and by the lack of more rounds of the panel in order to assess the effects of the interventions over a longer time span. The present analysis represents an improvement over the previous impact evaluation study in terms of the methodology and the information used. First, the present research uses a yearly panel over ten years, which provides more continuous information about entry and exit of firms. As well, a larger sample of firms participating in SME programs is available from the last round of the ENESTYC 2005 survey. Finally, a bounding methodology applied in the panel data allowed for testing the robustness of the impacts of programs to firm exit. 4.- DATA The paper uses the National Employment Salary, Training and Technology (Encuesta Nacional de Empleo, Salarios, Capacitación y Tecnología--ENESTYC) and Annual 16 Industry Survey (Encuesta Industrial Annual--EIA) surveys maintained by INEGI to create the non-experimental panel dataset. The ENESTYC periodically surveys manufacturing firms with sample sizes of between 5,000 and 8,000 establishments, fielded in 1995, 1999, 2001 and 2005. In 2001 and 2005, the ENESTYC included a module of questions on participation in major government SME support programs. In 2005, INEGI also fielded a Micro-ENESTYC survey with greatly expanded coverage of micro and small enterprises. The EIA is the annual manufacturing survey fielded by INEGI, and a linked panel of establishments can be created over the 1992-2006 period from the annual surveys. Our strategy was to link the 2001 and 2005 ENESTYC to the 1994-2005 EIA to exploit the availability of annual panel data in that survey. The ENESTYC uses the same sampling frame as the annual EIA, that is a stratified random sample by 54 sectors and four size categories--micro with 1-15 employees, small with 16- 100, medium with 101-250 and large with over 250 employees. The sample size of each ENESTYC survey is fairly large, varying from 5,000 establishments in 1995 to 7,500 establishments in the 2005 ENESTYC. However, because ENESTYC was never designed to be a panel survey and smaller firms in each survey were randomly sampled, only a small proportion of SMEs can be tracked over time. Few micro enterprises can be linked over time in the ENESTYC, but sample sizes for the panel of small and medium enterprises should be adequate for the proposed analyses (Table 14). Table 14 Number of Panel Firms by Size and ENESTYC Years 1995 and 1999 and 2001 and Size 2001 2001 2005 Micro (1 ­ 15) 2 54 10 Small (16 ­ 100) 161 335 192 Medium (101 ­ 250) 723 1,273 1,192 Large (251 - +) 954 1,404 1,244 Total 1,840 3,066 2,638 Source: Research team estimates from ENESTYC surveys. The 2001 ENESTYC surveys included, for the first time, questions about familiarity with and participation in a list of major government-sponsored SME programs, including date of participation, duration, and type of services used. This module was repeated in the 2005 ENESTYC survey. In both surveys, firms could indicate participation in one or more programs. The 2005 ENESTYC dropped several SME programs that had since ceased operation, and included a number of other SME initiatives introduced since 2001. The authors worked with INEGI on linking establishments from the 2001 and 2005 ENESTYC with earlier ENESTYC surveys, and with the panel EIA covering the 1992-2006 period. INEGI linked the EIA and the 2001 and 2005 ENESTYC that generated a panel EIA dataset for the sample of enterprises that report information on program participation in the 17 ENESTYC 2001 and 2005, and dates of participation if any. The ENESTYC and EIA were linked through an identification code constructed by INEGI. This dataset provides information on the treatment for the years before the intervention and after the intervention. The ENESTYC-EIA panel is conformed of near 2,600 firms, of which around 1,600 firms reported having participated in one or more programs (the treatment group) and 1,000 stated that they had never participated in any programs (the control group). CIMO-PAC, the CONACyT program and the environment program were the three most commonly used by firms in the treatment group (Table 15). Firms are characterized as either currently participating in a program or having participated in the past, with the former category having more respondents. Table 15. SME Program Participation Number of Percent of Not Participating Participating Total Currently Now, Participated SME program Firms Sample Participating in the Past CIMO 282 10.96 142 140 COMPITE 60 2.33 23 37 CRECE 38 1.48 16 22 FIDECAP 8 0.31 6 2 FAMPYME 10 0.39 6 4 MEX-EX 36 1.40 18 18 PATCI 10 0.39 4 6 PMT 20 0.78 10 10 PCI 8 0.31 6 2 PAIDEC 13 0.51 9 4 Fondo Pyme 27 1.05 13 14 PROMODE 9 0.35 3 6 PROSEC 113 4.39 88 25 Mixed or Sectoral Funds 36 1.40 23 13 Productive Chains 47 1.83 33 14 Financing 39 1.52 23 16 Crediexporta 47 1.83 28 19 PAT 44 1.71 31 13 PNAA 247 9.60 189 58 Fiscal Support and Technological Innovation 187 7.27 124 63 State Government Support 67 2.60 40 27 18 Municipal Government Support 37 1.44 29 8 Other 100 3.89 75 25 Source: Linked ENESTYC-EIA panel data. The paper analyzes the SME programs that yielded the larger sample sizes. The programs are: 1) PAC-CIMO, included both in the ENESTYC 2001 and in the ENESTYC 2005 Surveys. 2) PROSEC, only recorded in the ENESTYC 2005. 3) PNAA, only recorded in the ENESTYC 2005. 4) Fiscal Support and Technological Innovation, only recorded in the ENESTYC 2005. 5) State government support, only recorded in the ENESTYC 2005. 6) Any program, referring to any program in the program module of the ENESTYC 2001 and of the ENESTYC 2005, as follows: ENESTYC 2001 and 2005: COMPITE, CRECE, FIDECAP and FAMPYME. ENESTYC 2001: MEX-EX (México Exporta), Technical Assistance and Image Campaign Program (Programa de Asistencia Técnica y Campaña de Imagen-- PATCI), PMT, PCI, and PAIDEC. ENESTYC 2005: Fondo PYME, PROMODE, Mixed or Sectoral Funds, Productive Chains, Financing, Crediexporta, PAT, Municipal Government Support, and other programs. The objective was to explore the impact of several programs in groups by agency, and to analyze programs individually when data allowed, as follows: 3 1) All STPS programs (CIMO-PAC), which include training, BDS, and productivity programs. 2) ME, which includes FIDECAP, FAMPYME, Fondo PYME, COMPITE, CRECE, PROMODE and PROSEC. 3) BANCOMEXT, which includes MEX-EX, PATCI, Crediexporta and PAT. 4) CONACyT includes PMT, PCI, PAIDEC, Fiscal Support and Technological Innovation and Mixed or Sectoral Funds. 3 The other option of grouping by type of program (training, technology upgrade, etc.) was not possible in Mexico, since most programs belong encompass more than one type of intervention (Table 12). 19 5) Other Agency includes Nafinsa, Productive Chains, Financing, PNAA, State Government Support, Municipal Government Support, and Others. Out of 30,199 year-firm observations, 18,435 are for the control group that reported never having participated in any SME programs, and 11,764 are for firms in the treatment group (Table 16). The last two columns refer to the treatment group with year-of-first-participation information, which was used to define a post-program indicator variable with a value of 0 for all years prior to the first-year-of-participation and a value of 1 for the first year of participation and all subsequent years. Table 16. Distribution of Treatment and Control Groups Treatment Group with Program Start Total Group Dates Year Sample Control Treatment Pre-program Post-program 1994 2,440 1,488 952 874 78 1995 2,444 1,490 954 851 103 1996 2,465 1,505 960 837 123 1997 2,495 1,524 971 819 152 1998 2,511 1,532 979 763 216 1999 2,522 1,539 983 642 341 2000 2,528 1,544 984 427 557 2001 2,534 1,548 986 331 655 2002 2,541 1,552 989 266 723 2003 2,573 1,571 1,002 185 817 2004 2,573 1,571 1,002 9 993 2005 2,573 1,571 1,002 0 1,002 Total 30,199 18,435 11,764 6,004 5,760 Source: Linked ENESTYC-EIA panel data. The treatment group and the control group increase with the size of the firm (Table 17). Firm size is defined as "micro" with 15 or fewer workers, "small" with 16 to 100 workers, "medium" with 101 to 250 workers, and "large" with over 250 workers. Table 17. Distribution of Treatment and Control Groups by Firm Size and Sector Sector Micro Small Medium Large Treat Control Treat Control Treat Control Treat Control Food, beverages and tobacco 1 3 22 44 81 101 154 171 Textile industry, clothing and leather industry 0 1 8 31 56 107 62 82 Wood products 0 0 1 7 11 22 12 15 Paper products 0 0 1 9 31 71 25 55 Chemical 0 1 27 56 84 138 93 139 Mineral products 0 2 10 21 24 38 21 40 Basic metalic industry 0 0 1 8 8 21 19 22 Metalic products, machinery and equipment 0 4 18 61 93 135 116 137 20 Other manufacturing industries 0 0 1 0 3 2 6 8 Total 1 11 89 237 391 635 508 669 Source: Linked ENESTYC-EIA panel data. Note: Firm size is defined as follows: micro with 1-15 workers, small with 16-100 workers, medium with 101-250 workers and large with over 250 workers. Considering the distributions presented in Table 17, selecting a control group based on observable attributes such as sector and size is likely to be inadequate. Even with similar sector-size distributions, the treatment and control groups can have very different pre- program values of sales, productivity or wages. The EIA panel contains annual data on measures of firm performance such as sales, gross value of production, employment, total compensation, and income from exports, as well as some intermediate outputs that the programs may affect, such as technology transfers. Comparing the means for the key outcome measures of the treatment and control groups, several points emerge (Table 18). 4 First, there are statistically significant difference in means between the treatment and the control group for wages, employment, and worked hours, suggesting that the treatment group was doing fairly well. Second, these differences are similar before and after the treatment, meaning that the programs were not having additional effect on the performance of firms. This conjecture is analyzed further in the next section, which estimates the impact of programs controlling for selection biases. Table 18. Differences in Means Between the Treatment and the Control Group, Any Program Year When Program Two Years After Program Starts (t-0) Started (t+2) Difference in Difference in Outcome Measures Means t-test Means t-test Employment 95 ** 3.84 100 ** 3.89 Worked hours 217 ** 3.65 240 ** 3.79 Wages 16,389 ** 3.11 18,388 ** 3.37 Gross production 78,708 0.72 99,553 0.81 Inputs 73,491 1.01 78,533 0.99 Value added 5,216 0.12 21,020 0.41 Fixed assets -167,693 ** -5.56 -144,686 ** -3.89 Domestic sales 64,334 * 1.66 52,902 1.16 Foreign sales 34,052 0.44 56,737 0.72 Total sales 98,386 0.99 109,639 1.01 Technology transfers payments -712 -0.35 107 0.07 Technology transfer income -30 -0.18 -46 -0.19 Maquila services expenditures -372 -0.54 -844 -1.17 Maquila services incomes 4,215 * 1.77 4,638 1.46 Source: Linked ENESTYC-EIA panel data. Notes: (1) Monetary variables are in real 2005 pesos. (2) * and ** denote significant differences at 10% and 5% level, respectively. 4 Annex 1 presents the means for the other programs considered in the analysis. 21 5.- MODEL 5.1 Selection biases Consider a general model for firm i in time t which relates outcomes Y to observable firm attributes X and an indicator variable for participation in a program D : Yit = X it + t Dit + it (1) it = vi + u it where is made up of a time-invariant firm-specific component v and a randomly distributed error term u . If firms are randomly assigned to the treatment and control groups, then the treatment and the control groups have similar distributions of the non-observed attributes. In this case, OLS can be used to estimate (1) from post-program cross-sectional data to get an unbiased measure of , the net effect of the program. The challenge is to estimate the net impacts of program participation free of bias from self-selection of firms into programs based on their observable and unobservable productivity attributes. To see this, rewrite (1) separately for the treatment and control groups and difference the two equations to get an expression for as in (2): E[Yit1 X it , D = 1] = X it + + E ( it D = 1) 1 1 (2) E[Yit0 X it , D = 0] = X it + E ( it D = 0) 0 0 E[Yit1 X it , D = 1] - E[Yit0 X it , D = 0] = ( X it - X it ) + + E ( it D = 1) - E ( it D = 0) 1 0 1 0 The differenced equation in (2) identifies two potential sources of bias from non-random assignment, one due to differences between groups in observed attributes X , ( X it - X it ) 1 0 another due to differences in the non-observed attributes u , E (u i1 ) E (u i 0 ) . The first source of bias can be minimized by careful matching of the control group to the treatment group in terms of observables X . However, the second source of bias due to u remains. One partial solution is to include a lot of observable variables that are correlated with the outcome of interest. This reduces the residual variance, and allows one to determine how much selection on unobservables there would have to be in order to overturn the direction of program impact obtained when assuming selection on observables (Altonji, Elder and Taber, 2005). Evaluation studies sought to address these potential selection biases through the combined use of propensity score matching and difference in difference (DID) methods. This paper uses similar methods to accommodate the specific nature of our panel data, which is discussed below. 22 Recent studies have matched the treatment and control groups on the basis of a propensity score. Rosenbaum and Rubin (1983) define the propensity score as the probability of receiving a treatment conditional on a vector of observed covariates. They showed that propensity score eliminates the bias due to observed covariates. Propensity score matching may not be enough if self-selection into programs is also based on productivity attributes not observable to the analyst. The presence of unobserved attributes v can thus bias estimates of , even yielding negative program impacts. The confounding effects of v on can be addressed through DID methods. Let t = 0 and t = 1 represent the pre- and post-participation periods. First differencing equation (1) for the treatment group and the control group eliminates the time invariant v term: E[Yit1 | X it , D = 1] = X it + + u it 1 1 1 (3) E[Yit0 | X it , D = 0] = X it + u it 0 0 0 where Y is a lag operator such that Y = Yit - Yi ,t -1 . The second difference between the differenced values of Y for the treatment and control groups in (3) may be expressed as: E[Yit1 | X it , D = 1] - E[Yit0 | X it , D = 0] = (X it - X it ) + + (u it - u it ) (4) 1 0 1 0 1 0 Equation (4) yields an unbiased estimate of if the evolution over time of observable attributes of the two groups is similar, that is X it = X it , and if the changes in unobserved 1 0 characteristics have means which do not depend upon allocation to treatment, that is, if u it = u it . 1 0 We extend these analytic approaches to accommodate the specific panel structure of our panel data since we have a cohort of firms with observed characteristics and participation in programs between 1994 and 2005. 5.2 Cox proportional Hazard Model Instead of using logit or probit models for program participation, we use a Cox proportional hazard model to estimate the propensity score of the likelihood of program participation for the sample of treatment and control groups followed over the 1994 to 2005 period. 5 The Cox proportional hazard model relates the likelihood of entry into a program, conditional upon survival (non-entry) up to that point in time, to a baseline hazard function and a set of independent variables Z . The underlying hazard function h(t ,..) may be written as follows: 5 An alternative approach is to estimate separate logit models of program participation for different cross-sections (or year intervals) to derive propensity scores for each treatment cohort (or groups of cohorts). This did not prove feasible because of small sample sizes, which led to very imprecise estimates of the logit model. The Cox proportional hazards model was preferred not only because of sample size considerations but also for its unified treatment of the underlying process of selection into programs over time. 23 h{(t ), ( Z1 , Z 2 ,..Z m )} = h0 (t ). exp(1Z i + 2 Z 2 + ...m Z m ) (5) where Z is a vector of m covariates, and h0 (t ) is the baseline hazard when the values of all the covariates are set to 0. This model can be made linear by dividing both sides of equation (5) by h0 (t ) and taking natural logarithms: log{h(t ), ( Z )} / h0 (t ) = 1 Z i + 2 Z 2 + ... m Z m (6) This leaves an equation (6) that is readily estimable, and from which the predicted value of .Z can be calculated. We use the relative hazard of program entry for firms with attributes Z as the propensity score for defining the region of common support for matching successive cohorts of treated firms and their control group. Table 19. Estimates from Cox Proportional Hazards Model. Results from Any Program Participation Model Standard Independent variables Hazard Ratio Error Z-statistic Region Center 1.196 0.1133 1.89 Mexico City 0.824 0.0804 -1.98 Southern 1.251 0.1833 1.53 Sector Textile industry, clothing and leather industry 0.921 0.0998 -0.76 Wood products 1.150 0.1537 1.05 Paper products 0.471 0.1048 -3.39 Chemical 0.851 0.1044 -1.32 Mineral products 0.708 0.1487 -1.64 Basic metalic industry 0.819 0.2027 -0.81 Metallic products, machinery and equipment 1.064 0.1250 0.53 Other manufacturing industries 1.535 0.5984 1.1 Age of firm 1.006 0.0020 3.24 The firm is a branch 0.768 0.0670 -3.03 24 Foreign capital share (%) 1.000 0.0010 0.18 Total sales growth (%) 1.001 0.0010 0.53 Log(total sales) lagged 1 year 1.099 0.0276 3.76 Log likelihood: -6285.7219 Number observations: 21,321 Number firms: 2,375 Number firms participating: 838 The Cox model estimates are used to predict the relative hazard rates for the treatment and control groups. Table 19 shows that a coefficient greater than one indicates a higher probability of enrolling in any SME program. Thus, firms located outside Mexico City, older firms, and firms with higher production have a higher probability of enrolling in SME programs. As the propensity score for each firm, we use the mean of their hazard rates for all years in which they are available. 6 The hazard rate averaged 3.530 for the treatment group and 3.240 for the control group, consistent with the treatment group as a whole having a higher relative probability of program participation. The treatment group has greater density in the upper tail of the distribution of propensity scores than the control group (Figure 1). Nonetheless, within the region of common support, every firm in each group has a positive probability of participating in SME programs, though some may have higher probabilities than others. 7 6 For the treatment group, the means are computed for all years up until the year of program participation, after which relative hazards rates are not defined because the failure event has occurred. 7 The region of common support lies between 1.154 and 7.777 since the minimum and maximum values for the treatment group was 1.108 and 8.0824, and 1.154 and 7.777 for the control group and 7.685. 25 Figure1. Distribution of Propensity Scores 0.10 0.08 0.06 Density of propensity score 0.04 0.02 0.00 -0.02 -0.04 -0.06 -0.08 1.1 2.7 4.3 5.9 7.4 Propensity score Control Tratamiento 5.3 Panel Regressions As discussed, traditional propensity score matching methods are not suitable to accommodate the specific structure of our data. We adopt a more flexible regression approach that allows us to estimate treatment effects taking into account differing entry points into programs, use of multiple types of programs, widely varying time since program participation, and year specific shocks. We rely on fixed-effects models to eliminate the effects of unobserved firm heterogeneity as a source of bias in estimates of program impacts. In the spirit of the propensity score matching approach, we continue to focus on the sample of treatment and control group firms in the region of common support identified by their propensity scores. Consider an expanded equation (1) in levels: Yit = X it + 1 Dit + 2 Dit * YRSit + i + uit (7) which includes the program indicator D , an interaction term between D and a variable YRS measuring years-since-first-participated in the program, and the time-invariant error term vi . Estimating equation (7) in levels is likely to lead to biased estimates of because of the omitted variable vi , with the direction of bias being determined by the correlation between vi and Di . The fixed effects estimator addresses this possibility by taking deviations from variable means so that equation (7) can be rewritten as: 26 Yit - Yi = ( X it - X i ) + 1 ( Dit - Di ) + 2 ( Dit - Di ) * (YRSit - YRSi ) + (u it - u i ) (8) where firm variable means are denoted by a single subscript i . Like first differencing, the fixed effects transformation eliminates the potentially confounding effects of vi . First, we compare treatment effects estimated from a levels model (equation 7) and a fixed- effects model (equation 8) to test for potential biases in estimates of treatment effects from unobserved firm heterogeneity. Second, we test whether program impacts are larger in some programs than in others. In place of D , an indicator for participation in any program, we include indicator variables for participation in different types of SME programs D1i , D2i ,..., Dni , and test for differences in their impacts on outcomes. We note that this specification allows for (but does not explicitly model) multiple program use since each program used by firm i has its own program start date. Finally, we investigate how long it takes for program impacts to be realized. We test for time effects of program impacts from 2 , the estimated coefficient on the YRS interaction with D . An alternative, which we use, is to specify YRS as a set of discrete time intervals to allow for non-linear time-effects of program participation. We use a parsimonious model specification designed to facilitate comparison across different regressions. The following EIA variables were selected for the study: value added, gross production, total sales, worked hours, wages, fixed assets, exports, and some other intermediate variables such as inputs, technology transfers, and technology. These outcome measures are related to program indicator variables that take on a value of 0 for all years preceding the first-year of participation (pre-program period), and 1 for all years that follow including the first-year (post-program period). In addition to the program variable(s), our explanatory variables include indicator variables for firm size (small, medium and large relative to the omitted micro firm), and year dummy variables for 1994 through 2005 to control for the effects of year-specific stochastic shocks. 6.- RESULTS Our objective is to estimate the longer-term impacts of program participation controlling for the effects of observed and unobservable productivity attributes, and to test for differences in the treatment effects of the programs and agencies discussed in Section 5. Also of interest is investigating how quickly or slowly program impacts are realized over time. Finally, we are interested in testing the sensitivity of program impact estimates to the possibility that program participation inhibits firm exit from our panel data. First, looking at the treatment effects on outcomes measured in levels, we notice positive results for participation in any program, suggesting that firms were already performing well compare to the control group (upper part of Table A1). However, these estimates are biases since we need to eliminate the effects of unobserved firm heterogeneity. When we disaggregate the results by agency, we find that the average treatment effects estimated for outcomes measured in levels are not significant for STPS and CONACyT programs, and 27 programs of the ME and Bancomext had negative effects on several outcome variables. The only group to show positive effects was Other Agency. Using the fixed-effects model presented in equation (8), we find that participation in any SME program has positive and significant effects of 6 percent on value added, 5 percent on gross production, 5 percent in total sales, 6 percent on employment, 6 percent on fixed assets. The average treatment effect of participation in programs by ME, CONACyT and Other Agency is now positive and statistically significant at the 1, 5, or 10 percent level for most outcomes (lower part of Table A1). The impact of ME programs ranges between 4 and 6 percent, CONACyT between 7 and 26 percent and Other Agency between 3 and 14 percent. One can thus infer that programs under these agencies had on average a positive impact on the performance of SMEs. The only exception was the STPS program, which had a negative impact on some outcome variables, with the exception of the CIMO program up to 2001 which exhibit a statistical significant positive effect of nearly 27 percent on fixed assets and technology transfer payments (Table A6). The results by program, for those with enough sample size captured in the ENESTYC 2005, indicate that in some programs, such as ME's PROSEC and CONACyT's Fiscal Support and Technology Innovation, the average effects were not significant in levels (equation 7). However, once corrected by time invariant biases, the effects turned to positive in differences (equation 8), suggesting positive impacts from these interventions (Table A2). The estimates for PNAA were positive in levels and positive in differences, suggesting that firms participating in PNAA performed better than comparable firms, even once we control for time invariant bias due to unobservable characteristics. The estimated effect of PNAA is 6 percent increase in value-added, with similar results for the rest of the outcome variables. The estimates for PROSEC and Fiscal Support and Technology Innovation were not significant in levels and were positive and statistically significant in differences. The estimated impact of these programs ranges from 6 to 16 percent in outcome variables. The results for CIMO changed from not significant in levels to negative in differences. These results suggest that CIMO did not have an impact on the performance of firms. Firms that participated in CIMO up to 2001 showed a positive impact on selected outcome variables such as foreign sales, fixed assets and technology transfers (Table A6). These results are in line with the earlier impact evaluation from Tan and Lopez-Acevedo (2005), which found positive impacts on intermediate outputs but no significant impacts on outcomes before 2001. A number of factors could explain why positive impacts were found for PNAA, PROSEC and CONACyT's fiscal incentives program: · On average, each firm benefited by CONACyT programs between 2001 and 2006 received US$264,000, while the average amount that the federal government invested per firm in all of its programs was about US$22,000­12 times less. The amount of resources per firm of CONACyT's programs was second only to Bancomext. · Another possible explanation for CONACyT's programs is their support for innovation. Research indicates that the benefited firms had previously started doing research and 28 development around technological projects, lowering costs compared to starting to innovate from scratch. · The case of PROSEC is similar to the fiscal incentives of CONACyT. This program provides tax breaks for imported inputs needed for manufacturing, and hence individual firm internalizes its benefits. Therefore, it is in the firm's interest to maximize utility in terms of quality and costs. · The Environmental Audit Program (PNAA) is among the best known federal programs among manufacturing firms in 2002, according to the CIPI (2002), and its use kept growing. Between 2002 and 2006, the program tripled the number of audits initiated-- from 293 in 2002 to 933 in 2006--and the number of clean industry certificates issued doubled in the same period, going from 169 in 2002 to 338 in 2006. The firm's investment in the PNAA is relatively low and the benefits--such as accessing markets with tough environmental regulations--could far exceed costs. Thus far, we have estimated the average treatment effect of program participation without consideration for whether these effects vary over time. We test for time effects by including interactions terms between the program participation measure and time since entering a program, as in equation (8). Rather than forcing a functional form on these time effects (for example, with a quadratic specification of time and time squared), we define a set of indicator variables for different intervals (1, 2, 3, 4, 5-6, 7-9 and over 10 years) following the date of entering the program. This allows the effects of the interaction terms between the program indicator and time since participation to vary non-linearly with time in and after the program. The resulting estimates can be interpreted as the time effects of treatment if several assumptions hold. First, these effects are estimated holding constant all other time-varying factors, including inflation and macroeconomic shocks. The model accounts for these factors by including year dummy variables to capture year-specific stochastic shocks. A second assumption is that self-selection into treatment is not dependent upon time. The presence of cohort effects in treatment--firms that choose to participate early are different from those that join in later years--can introduce bias into these estimates. None of the indicator variables for time since participation are statistically significant before four years (Table A3). Beginning with four year after program entry, the estimated coefficient in fixed assets become positive and increase in value and statistical significance. Using the example of fixed assets, the treatment effect is 7 percent at four years, increasing to 14 percent at 5-6 years, after to 22 percent at 7-9 years and then to 42 percent from 10 years on since program entry. One issue that arises is that our data set does not include firms (including treatment firms) that exit. This raises the possibility of survivor bias if program participation increases the possibility of firms surviving that would otherwise exit (stop operations). In the absence of 29 good data on firm exit probabilities, we bound our estimates of program impacts by sensitivity analysis in which we re-estimate outcome models after dropping the bottom 5 percent of the treatment group in terms of outcome variables (e.g productivity, assuming that the lowest productivity firms would otherwise exit). Several points emerge from comparing the treatment effects with trimming and the original fixed effect models (Tables A4 and A5). First, the significance and magnitude of the program impact is quite similar under the trimming and the original estimates. Second, there are isolated differences between both groups: for some programs or variables the estimates based on trimming data were slightly higher than the original estimates. For example, for ME programs, the outcome variables of employment, gross production, and total sales effects in the trimmed data barely increased compared to the original estimates. Second, for STPS programs, the treatment effects of some variables such as sales, gross production and value added in the original were negative and now are not significant. The key point to infer from this sensitivity analysis is that, in general, the direction and size of treatment effects are robust to controls for potential biases from firm exit. 7.- CONCLUSIONS The significant public resources invested by Mexico in recent years into programs supporting SMEs are not well evaluated. While promoting a more vibrant SME sector is a laudable goal, the relative success of current and past programs is difficult to ascertain with any rigor. This paper attempts to contribute to broadening and deepening efforts to evaluate SME program impacts, with the goal of improving the efficiency and effectiveness with which these resources are spent. It does so, first, by providing a detailed description of many of the main recent programs administered by various government agencies, and second, by designing and implementing a novel evaluation methodology to better assess outcome impacts over time. Microenterprises and SMEs make up 99 percent of enterprises in Mexico, employ about 64 percent of the workforce, and account for over 40 percent of GDP. Given the importance of SME in the economy, governments in Mexico over the last 20 years have put in place a wide variety of SME support programs. An inventory identified 151 SME programs administered by different government agencies including CONACyT, Bancomext, and the ministries of economy, labor, finance, and environment, among others. While their stated objectives may differ, collectively these support programs seek to promote the productivity, quality, and competitiveness of small enterprises, encourage technology upgrading, training, and conservation, and improve earnings and safe working conditions for the workforce in SMEs. As is apparent from the overview in Section 2, Mexico has a huge array of programs involved in supporting SMEs in some way. This in itself is an important finding, and suggests that Mexico would do well to design a more coherent framework to orient resources more efficiently and strategically, and avoid program overlap. As well, the lack of easily available information on program budgets, activities and beneficiaries points to a need for improved consolidation of information on SME support, to improve the ability of policymakers to make evaluations and comparisons, and ensure that the country's scarce fiscal resources are achieving the greatest possible impact. 30 Just how effective these SME programs have been in achieving their objectives is unclear. In Mexico, impact evaluations of SME programs are rare--most are qualitative in nature and small in scope, either measuring beneficiary satisfaction with support services or else easily quantified program-coverage indicators. Few measure the net benefits of program participation by comparing the performance of beneficiaries with that of a comparison group made up of similar enterprises that did not participate in the program, and on which cost- benefit assessments are made. None take into account the complicated biases from unobserved firm heterogeneity and self-selection that plague efforts to measure the true impacts of program participation. Mexico is not unique in this regard. Most countries, both advanced and developing, are still in the process of increasing rigorously evaluation of their SME programs, as noted in the forthcoming World Bank Regional Study on Impact Evaluation of SME programs. This paper uses a unique firm panel data to rigorously evaluate the impacts of SME programs in Mexico. The paper makes use of the program module in the ENESTYC 2001 and 2005, which include retrospective questions about firm participation, date-of-participation, and type of support received in, and familiarity with any one or more of SME programs administered by several agencies. The ENESTYC was linked to the EIA to form a panel of firms over 10 years. In this way, pre- and post-program outcomes variables are tracked over time for both the treatment and control groups. Several approaches are used to assess the impacts of program participation. A generalization of propensity-score matching and DID estimation adapted to the structure of our panel data is used to test for program impacts on intermediate and final outcome measures. This approach allowed for controlling selection biases that have plagued previous impact evaluation studies such as self-selection, unobserved heterogeneity and attrition. Our results indicate that program participation in certain types of SME programs is associated with higher value added, sales, export, and employment. This is the case for the programs of the Ministry of Economy (PROSEC), PNAA, and CONACyT's Fiscal Incentive Programs. Why these programs yielded positive effects and programs like CIMO did not? Maybe because these three programs (PROSEC), PNAA, and CONACyT's Fiscal Incentive tend to reach the crème of SMEs: mid-sized, exporting firms with high technology, clean processes, and more human and physical capital-intensive production techniques. The implication is that this type of firm may have very different characteristics in comparison to those served by other programs--i.e., smaller, local-market oriented, low-tech, less human and physical capital intensive. Our panel data also identifies the timing of the effects of program participation on outcomes. We found that some of the positive effects take some time after the third or fourth year of exposure into the program. These results remain robust after trimming the bottom 5 percent of our treatment group. Several research lines emerge from our analysis. One key finding is that the very high number of programs and their constant evolution over time (changing names and structures, closing old programs and opening new ones) makes rigorous impact evaluations a major challenge. Thus one area is to investigate is cohort effects in the panel for those programs 31 that underwent major design changes. An initial research was carried out in this paper with the CIMO-PAC that could serve as example for the analysis of other programs. We plan to carry out further analysis for the CIMO-PAC program, which has been in place for the last 11 years but underwent a dramatic change in its design and implementation in 2002. 32 ANNEX 1 Table A1. Program Impacts of Any Program and by Program Agency. Levels and Fixed Effects Model with Propensity Score Matching Tech. Value Gross Worked Foreign Maquila Total sales Employment Wages Fixed assets transfers added production hours sales services payments Levels Model Any program 0.241 ** 0.231 ** 0.220 ** 0.157 *** 0.163 *** 0.058 0.181 * 0.179 0.286 -0.163 (0.1) (0.09) (0.1) (0.05) (0.05) (0.05) (0.11) (0.19) (0.23) (0.2) STPS -0.143 -0.155 -0.152 -0.015 -0.004 -0.086 -0.060 -0.418 -0.270 -0.139 (0.14) (0.14) (0.14) (0.08) (0.08) (0.07) (0.16) (0.28) (0.34) (0.3) ME -0.269 -0.357 ** -0.349 * 0.038 0.027 -0.186 ** -0.438 ** 0.283 0.591 -0.449 (0.19) (0.18) (0.18) (0.1) (0.1) (0.09) (0.2) (0.37) (0.44) (0.39) BANCOMEXT -1.000 *** -1.116 *** -1.192 *** -0.230 -0.233 -0.386 *** -0.891 *** 0.366 -1.131 * 0.487 (0.27) (0.26) (0.27) (0.15) (0.15) (0.13) (0.3) (0.5) (0.67) (0.51) CONACyT -0.063 -0.055 -0.052 -0.077 -0.112 0.096 -0.040 -0.527 0.398 -0.283 (0.23) (0.22) (0.23) (0.13) (0.13) (0.11) (0.25) (0.41) (0.52) (0.42) Other Agency 0.853 *** 0.894 *** 0.897 *** 0.353 *** 0.374 *** 0.299 *** 0.749 *** 0.520 ** 0.600 * -0.107 (0.13) (0.12) (0.13) (0.07) (0.07) (0.06) (0.14) (0.25) (0.31) (0.26) Fixed Effects Model Any program 0.052 *** 0.049 *** 0.046 *** 0.060 *** 0.060 *** 0.006 0.059 *** -0.035 -0.091 -0.025 (0.02) (0.01) (0.01) (0.01) (0.01) (0.01) (0.02) (0.04) (0.06) (0.06) STPS -0.046 ** -0.030 * -0.034 ** -0.003 -0.007 -0.001 0.061 ** -0.252 *** 0.267 *** -0.110 (0.02) (0.02) (0.02) (0.01) (0.01) (0.01) (0.03) (0.06) (0.1) (0.09) ME 0.069 *** 0.052 *** 0.051 *** 0.034 ** 0.035 ** -0.005 -0.041 0.013 -0.139 0.028 (0.03) (0.02) (0.02) (0.01) (0.01) (0.01) (0.03) (0.06) (0.1) (0.09) BANCOMEXT -0.063 0.018 0.041 -0.025 -0.026 -0.032 * -0.094 * -0.075 0.235 -0.354 ** 33 (0.04) (0.03) (0.03) (0.02) (0.02) (0.02) (0.05) (0.09) (0.2) (0.14) CONACyT 0.100 *** 0.094 *** 0.082 *** 0.092 *** 0.088 *** 0.001 0.066 * 0.253 *** 0.082 0.247 ** (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.07) (0.11) (0.11) Other Agency 0.077 *** 0.062 *** 0.061 *** 0.045 *** 0.051 *** 0.026 *** 0.110 *** 0.036 -0.103 0.082 (0.02) (0.01) (0.01) (0.01) (0.01) (0.01) (0.02) (0.05) (0.08) (0.08) Sample size 27506 27742 27376 27658 27655 26957 27260 13445 5924 9049 Table A2. Program Impacts by Program in ENESTYC 2005. Levels and Fixed Effects Model with Propensity Score Matching Tech. Value Gross Worked Foreign Maquila Total sales Employment Wages Fixed assets transfers added production hours sales services payments Levels Model CIMO -0.069 0.0022 0.0746 -0.016 0.0035 0.0699 0.1753 -0.265 0.3865 -0.436 (0.21) (0.21) (0.22) (0.12) (0.12) (0.1) (0.24) (0.42) (0.53) (0.42) PROSEC -0.091 -0.054 -0.019 -0.049 -0.138 0.161 -0.125 1.575 *** 0.548 -0.844 (0.32) (0.31) (0.32) (0.18) (0.18) (0.16) (0.35) (0.59) (0.8) (0.65) PNAA 0.981 *** 0.921 *** 0.912 *** 0.317 *** 0.345 *** 0.297 *** 0.992 *** 0.727 * 0.437 0.671 (0.21) (0.2) (0.21) (0.12) (0.12) (0.1) (0.23) (0.4) (0.49) (0.48) Fiscal Support & -0.010 0.116 0.179 -0.039 -0.074 0.166 0.240 -0.055 0.553 -0.339 Tech. Innovation (0.29) (0.29) (0.3) (0.16) (0.16) (0.14) (0.33) (0.56) (0.8) (0.49) State Goverment 1.489 *** 1.161 ** 1.078 * 0.273 0.282 0.165 0.687 0.472 0.834 1.595 Support (0.57) (0.57) (0.58) (0.32) (0.32) (0.27) (0.63) (1.1) (1.72) (1.1) Fixed Effects Model CIMO -0.06 ** -0.04 * -0.046 ** -0.03 * -0.03 * -0.003 0.0533 -0.084 -0.01 -0.322 ** (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.08) (0.15) (0.13) PROSEC 0.167 *** 0.164 *** 0.133 *** 0.058 *** 0.069 *** 0.003 0.136 *** 0.153 * 0.217 0.137 (0.04) (0.03) (0.03) (0.02) (0.02) (0.02) (0.05) (0.08) (0.15) (0.14) 34 PNAA 0.065 ** 0.043 ** 0.058 *** -0.023 -0.014 0.052 *** 0.074 ** 0.065 -0.032 -0.199 (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.07) (0.12) (0.13) Fiscal Support & 0.146 *** 0.094 *** 0.094 *** 0.108 *** 0.100 *** -0.015 -0.001 0.160 * -0.106 0.370 *** Tech. Innovation (0.04) (0.02) (0.03) (0.02) (0.02) (0.02) (0.04) (0.09) (0.14) (0.14) State Goverment 0.155 ** 0.025 -0.017 0.057 0.075 * -0.040 0.127 -0.101 0.255 0.165 Support (0.07) (0.05) (0.05) (0.04) (0.04) (0.03) (0.09) (0.15) (0.32) (0.46) Sample size 21501 21663 21379 21631 21629 21106 21319 10921 4603 7207 Source: Linked ENESTYC-EIA panel data. Notes: 1) ***, ** and * denote statistical significance at the 1, 5 and 10 percent level, respectively. 2) Numbers in () correspond to standard errors. Table A3. Time Effects of Any Program Participation (time since started the program). Fixed Effects Model with Propensity Score Matching 5 - 6 years 10 + year Sample Outcome variable 1 year later 2 years later 3 years later 4 years later later 7 - 9 year later later size Value added 0.000 0.003 -0.026 0.013 0.029 -0.073 0.054 27506 (0.029) (0.031) (0.033) (0.034) (0.033) (0.049) (0.133) Gross production 0.005 0.019 0.004 0.006 0.029 0.028 0.150 27742 (0.02) (0.022) (0.022) (0.023) (0.023) (0.034) (0.092) Total sales 0.014 0.024 0.005 0.006 0.016 0.060 * 0.196 ** 27376 (0.02) (0.021) (0.022) (0.023) (0.022) (0.034) (0.091) Employment 0.010 0.006 -0.020 -0.034 * -0.028 -0.054 ** 0.043 27658 (0.015) (0.016) (0.017) (0.018) (0.017) (0.026) (0.069) Worked hours 0.008 0.008 -0.024 -0.039 ** -0.021 -0.050 * 0.044 27655 (0.016) (0.017) (0.018) (0.019) (0.018) (0.027) (0.074) Wages -0.012 -0.004 0.003 0.000 -0.009 0.035 * 0.096 * 26957 (0.012) (0.013) (0.014) (0.014) (0.014) (0.021) (0.057) 35 Fixed assets -0.011 -0.015 0.013 0.071 * 0.136 *** 0.217 *** 0.416 *** 27260 (0.036) (0.038) (0.04) (0.041) (0.04) (0.06) (0.162) Inputs 0.008 0.027 0.005 0.006 0.038 0.091 ** 0.262 *** 27736 (0.021) (0.023) (0.024) (0.025) (0.024) (0.036) (0.098) Foreign sales -0.022 -0.026 0.015 -0.059 -0.052 -0.079 -0.304 13445 (0.065) (0.071) (0.075) (0.079) (0.075) (0.116) (0.34) Tech. transfers payments 0.043 -0.005 -0.115 0.043 0.042 -0.215 0.210 5924 (0.105) (0.113) (0.122) (0.127) (0.127) (0.183) (0.54) Maquila services -0.102 -0.049 0.174 0.125 0.018 -0.190 0.238 9049 (0.106) (0.115) (0.122) (0.13) (0.127) (0.179) (0.432) Source: Linked ENESTYC-EIA panel data. Notes: 1) ***, ** and * denote statistical significance at the 1, 5 and 10 percent level, respectively. 2) Numbers in () correspond to standard errors. Table A4. Bounding Impacts of Program Participation. Trimming Bottom 5% of Treatment Group Outcomes. Fixed effects model with PSM Tech. Value Gross Worked Foreign Maquila Total sales Employment Wages Fixed assets transfers added production hours sales services payments Any program 0.064 *** 0.053 *** 0.052 *** 0.057 *** 0.056 *** 0.019 *** 0.081 *** 0.015 -0.097 * 0.005 (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.02) (0.03) (0.06) (0.06) STPS -0.036 -0.023 -0.025 -0.004 -0.004 0.014 0.073 *** -0.263 *** 0.204 ** -0.025 (0.02) (0.02) (0.02) (0.01) (0.01) (0.01) (0.03) (0.06) (0.1) (0.09) ME 0.053 ** 0.056 *** 0.053 *** 0.043 *** 0.039 *** 0.002 -0.024 0.077 -0.088 0.081 (0.03) (0.02) (0.02) (0.01) (0.01) (0.01) (0.03) (0.06) (0.1) (0.09) BANCOMEXT -0.062 0.029 0.044 -0.001 -0.002 0.006 -0.095 * -0.010 0.186 -0.371 *** (0.04) (0.03) (0.03) (0.02) (0.02) (0.02) (0.05) (0.09) (0.19) (0.14) CONACyT 0.138 *** 0.100 *** 0.085 *** 0.077 *** 0.071 *** -0.015 0.055 0.262 *** -0.028 0.236 ** 36 (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.07) (0.11) (0.11) Other Agency 0.071 *** 0.049 *** 0.052 *** 0.030 *** 0.033 *** 0.035 *** 0.139 *** 0.039 -0.067 0.077 (0.02) (0.01) (0.01) (0.01) (0.01) (0.01) (0.02) (0.05) (0.07) (0.08) Sample size 27006 27299 26878 27181 27159 26496 26786 13161 5802 8882 Source: Linked ENESTYC-EIA panel data. Notes: 1) ***, ** and * denote statistical significance at the 1, 5 and 10 percent level, respectively. 2) Numbers in () correspond to standard errors. Table A5. Bounding Impacts of Program Participation. Trimming Bottom 5% of Treatment Group Outcomes. Fixed effects model with PSM Tech. Value Gross Worked Foreign Maquila Total sales Employment Wages Fixed assets transfers added production hours sales services payments CIMO -0.036 -0.032 -0.037 * -0.031 ** -0.028 * 0.0033 0.0509 -0.128 * 0.098 -0.26 ** (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.08) (0.15) (0.13) PROSEC 0.127 *** 0.169 *** 0.140 *** 0.048 ** 0.048 ** 0.029 * 0.125 ** 0.195 ** 0.182 0.224 (0.04) (0.03) (0.03) (0.02) (0.02) (0.02) (0.05) (0.08) (0.15) (0.14) PNAA 0.068 ** 0.053 ** 0.069 *** -0.041 *** -0.036 ** 0.063 *** 0.107 *** 0.055 -0.033 -0.090 (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.07) (0.12) (0.13) Fiscal Support & 0.160 *** 0.114 *** 0.110 *** 0.073 *** 0.071 *** -0.020 0.001 0.225 *** -0.208 0.373 *** Tech. Innovation (0.04) (0.02) (0.03) (0.02) (0.02) (0.02) (0.05) (0.08) (0.14) (0.14) State Goverment 0.128 * 0.029 -0.036 0.057 0.065 * -0.052 * 0.106 -0.035 0.272 0.514 Support (0.07) (0.05) (0.05) (0.04) (0.04) (0.03) (0.09) (0.15) (0.31) (0.47) Sample size 21140 21329 21017 21269 21260 20792 20982 10701 4516 7084 Source: Linked ENESTYC-EIA panel data. Notes: 1) ***, ** and * denote statistical significance at the 1, 5 and 10 percent level, respectively. 2) Numbers in () correspond to standard errors. 37 Table A6. Program Impacts of CIMO in ENESTYC 2001. Models with Propensity Score Matching Value Gross Total sales Employment Worked Wages Fixed Foreign Tech. Maquila added production hours assets sales transfers services payments Levels Model CIMO - -0.301 * - ** -0.084 - - * - - - - 0.249 0.346 0.080 0.157 0.243 0.551 0.560 0.177 (0.17) (0.17) (0.17) (0.09) (0.09) (0.08) (0.2) (0.35) (0.4) (0.39) Fixed Effects Model CIMO - -0.010 - 0.016 0.006 0.004 0.085 ** - *** 0.459 *** 0.150 0.028 0.014 0.286 (0.03) (0.02) (0.02) (0.02) (0.02) (0.01) (0.04) (0.08) (0.14) (0.13) Sample size 22269 22452 22147 22386 22383 21835 22116 11411 5079 7309 Fixed Effects Model. Trimming Bottom 5% of Treatment Group Outcomes CIMO - -0.014 - 0.006 - 0.026 ** 0.103 *** - *** 0.154 0.149 0.048 0.010 0.002 0.253 (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) (0.03) Sample size 22075 22290 21955 22196 22188 21644 21939 11315 5039 7254 Source: Linked ENESTYC-EIA panel data. Notes: 1) ***, ** and * denote statistical significance at the 1, 5 and 10 percent level, respectively. 2) Numbers in () correspond to standard errors. Table A7. Differences in Means Between the Treatment and the Control Group, by Program or Agency Year when program starts Two years after program (t-0) started (t+2) 38 Outcome mesures Difference in means t-test Difference in means t-test CIMO Employment -43 -1.01 -29 -0.54 Worked hours -117 -1.15 -76 -0.59 Wages -9,255 -1.03 -8,562 -0.79 Gross production -303,104 -1.25 -345,393 -1.19 Value added -113,099 -0.98 -172,786 -1.40 Fixed assets -193,055 ** -2.63 -193,925 ** -1.96 Domestic sales -108,115 * -1.69 -99,548 -1.14 Foreign sales -164,875 -1.07 -180,879 -0.96 Total sales -272,990 -1.47 -280,428 -1.18 Technology transfers payments 1,964 0.49 1,081 0.36 Technology transfers incomes -133 -0.44 -162 -0.30 Maquila services expenditures -700 -0.52 -1,117 -0.69 PROSEC Employment 187 ** 3.15 177 ** 2.84 Worked hours 394 ** 2.71 389 ** 2.55 Wages 45,108 ** 3.44 42,438 ** 3.20 Gross production 704,012 ** 2.39 954,131 ** 2.93 Value added 252,437 ** 2.26 508,979 ** 3.77 Fixed assets -207,734 ** -2.44 -33,804 -0.35 Domestic sales 115,450 1.20 125,036 1.11 Foreign sales 609,016 ** 2.83 584,511 ** 2.70 Total sales 721,094 ** 2.66 714,333 ** 2.48 Technology transfers payments -756 -0.14 -206 -0.05 Technology transfers incomes -117 -0.28 -143 -0.24 Maquila services expenditures -1,094 -0.58 -412 -0.21 PNAA Employment 122 ** 2.93 136 ** 2.64 Worked hours 315 ** 3.14 326 ** 2.61 Wages 20,203 ** 2.28 24,873 ** 2.34 Gross production 117,217 0.49 94,694 0.34 Value added 48,386 0.43 30,441 0.25 39 Fixed assets -106,174 -1.49 -205,590 ** -2.15 Domestic sales 173,235 ** 2.82 176,821 ** 2.11 Foreign sales -65,276 -0.43 -53,454 -0.29 Total sales 107,958 0.60 123,367 0.53 Technology transfers payments 356 0.09 1,277 0.40 Technology transfers incomes -120 -0.40 244 0.46 Maquila services expenditures -29 -0.02 643 0.47 Fiscal Support and Technological Innovation Employment 36 0.76 49 0.81 Worked hours 67 0.59 123 0.83 Wages 5,828 0.58 11,602 0.93 Gross production -137,372 -0.51 -205,452 -0.61 Value added -76,014 -0.59 -82,757 -0.57 Fixed assets -96,081 -1.18 -120,148 -1.06 Domestic sales -31,145 -0.44 -63,476 -0.63 Foreign sales -52,448 -0.30 -88,517 -0.41 Total sales -83,593 -0.40 -151,994 -0.56 Technology transfers payments 350 0.08 3,055 0.80 Technology transfers incomes -79 -0.23 -186 -0.30 Maquila services expenditures 418 0.27 -284 -0.15 State Goverment Support Employment 151 ** 1.98 124 1.15 Worked hours 375 ** 2.04 295 1.13 Wages 18,260 1.14 26,363 1.20 Gross production 47,695 0.11 118,247 0.20 Value added -23,143 -0.11 11,102 0.04 Fixed assets 25,709 0.19 -108,968 -0.55 Domestic sales 83,385 0.72 189,488 1.07 Foreign sales 21,853 0.08 -23,097 -0.06 Total sales 105,238 0.32 166,391 0.35 Technology transfers payments 1,369 0.19 915 0.14 Technology transfers incomes -122 -0.23 269 0.25 Maquila services expenditures -51 -0.02 -439 -0.13 STPS Programs Employment -9 -0.27 11 0.30 40 Worked hours -33 -0.42 23 0.25 Wages -5,296 -0.80 -1,388 -0.19 Gross production -298,296 * -1.76 -352,139 * -1.92 Value added -114,297 -1.42 -154,592 ** -1.97 Fixed assets -217,885 ** -4.14 -223,928 ** -3.52 Domestic sales -106,498 ** -2.20 -106,589 * -1.76 Foreign sales -149,780 -1.40 -169,639 -1.43 Total sales -256,278 ** -1.97 -276,228 * -1.83 Technology transfers payments 164 0.06 -621 -0.32 Technology transfers incomes -118 -0.56 44 0.13 Maquila services expenditures -737 -0.76 -1,269 -1.12 ME Programs Employment 73 * 1.94 88 ** 2.17 Worked hours 144 1.56 198 ** 2.00 Wages 13,110 1.62 16,425 * 1.93 Gross production 195,511 1.16 318,116 * 1.70 Value added 37,485 0.57 146,829 * 1.83 Fixed assets -222,112 ** -4.34 -145,117 ** -2.48 Domestic sales -46,509 -0.78 -51,251 -0.72 Foreign sales 284,039 ** 2.39 301,320 ** 2.56 Total sales 237,530 1.54 250,068 1.52 Technology transfers payments -1,102 -0.35 -749 -0.33 Technology transfers incomes -110 -0.46 -122 -0.35 Maquila services expenditures -1,131 -1.00 -1,208 -0.95 BANCOMEXT Programs Employment -38 -0.68 -49 -0.74 Worked hours -101 -0.74 -101 -0.63 Wages -14,186 -1.22 -16,255 -1.22 Gross production -316,839 -1.08 -473,986 -1.46 Value added -144,087 -1.03 -186,754 -1.31 Fixed assets -131,265 -1.41 -255,569 ** -2.26 Domestic sales -144,394 * -1.74 -264,778 ** -2.47 Foreign sales -97,827 -0.53 -129,629 -0.62 Total sales -242,222 -1.07 -394,407 -1.48 Technology transfers payments -3,372 -0.70 -3,153 -0.86 Technology transfers incomes -106 -0.29 -117 -0.20 41 Maquila services expenditures -841 -0.50 -699 -0.34 CONACyT Programs Employment 54 1.33 66 1.32 Worked hours 110 1.12 147 1.21 Wages 6,802 0.81 11,325 1.12 Gross production -100,911 -0.47 -137,665 -0.55 Value added -75,559 -0.74 -83,202 -0.76 Fixed assets -95,152 -1.43 -106,643 -1.22 Domestic sales -9,132 -0.15 -25,182 -0.31 Foreign sales -45,119 -0.34 -63,588 -0.39 Total sales -54,251 -0.33 -88,770 -0.43 Technology transfers payments 1,036 0.30 5,387 * 1.90 Technology transfers incomes -72 -0.27 -149 -0.33 Maquila services expenditures 744 0.60 633 0.40 Other Agency Programs Employment 151 ** 5.49 153 ** 4.97 Worked hours 359 ** 5.51 383 ** 5.08 Wages 27,687 ** 4.88 30,350 ** 4.80 Gross production 94,599 0.66 116,169 0.73 Value added 8,318 0.12 20,283 0.29 Fixed assets -131,760 ** -3.10 -111,801 ** -2.05 Domestic sales 205,522 ** 5.01 214,445 ** 4.18 Foreign sales -76,837 -0.85 -55,726 -0.54 Total sales 128,685 1.17 158,719 1.21 Technology transfers payments 1,450 0.61 453 0.27 Technology transfers incomes 77 0.40 21 0.07 Maquila services expenditures 766 0.94 317 0.35 Source: Linked ENESTYC-EIA panel data. Notes: (1) Monetary variables are in real 2005 pesos. (2) * and ** denotes significant differences at 10% and 5% level, respectively. 42 43 REFERENCES Bancomext (2006a) Informe de Rendición de Cuentas 2000-2006 Primera Etapa 1o. de diciembre de 2000 - 31 de diciembre de 2005, México. ---------- (2006b) Informe de Rendición de Cuentas 2000-2006 Segunda Etapa 1º de enero de 2006 - 30 de noviembre de 2006, México. CONACyT (2003) Informe General del Estado de la Ciencia y la Tecnología, México. ---------- (2005) Informe General del Estado de la Ciencia y la Tecnología, México. ---------- (2006) Informe General del Estado de la Ciencia y la Tecnología, México. ---------- (2007) Informe General del Estado de la Ciencia y la Tecnología, México. eNetwork (2007) La Realidad de la Incubación en América Latina y el Caribe: Modelos y Resultados, presentación para la Segunda Conferencia de Incubadoras Latinoamericanas y del Caribe, Santiago de Chile, Chile, July 24 and 25, 2007. González, V. (2005) Avances del Sistema Nacional de Garantías en México, Presentación para el X Foro I beroamericano sobre Sistema de Garantías y Financiamiento para las Micro y PyMEs, Secretaría de Economía del Gobierno Mexicano: Valladolid, Spain, September 2005. Instituto Tecnológico y de Estudios Superiores de Monterrey, Mexico City Campus (2008) Evaluación Integral 2008-2009 del Fondo Nacional de Apoyo a la Micro, Pequeña y Mediana Empresa (Fondo PyME). Mexico: October 2009. McKenzie, David and Christopher Woodruff (2008), "Experimental Evidence on Returns to Capital and Access to Finance in Mexico, Policy Research Working Papers, The World Bank, March 2008. Nacional Financiera (2002) Informe de Actividades 2002, Mexico. ---------- (2003) Informe de Actividades 2003, Mexico. ---------- (2004) Informe de Actividades 2004, Mexico. ---------- (2005) Informe de Actividades 2005, Mexico. ---------- (2006a) Informe de Actividades 2006, Mexico. ---------- (2006b) Informe de Rendición de Cuentas de la Administración 2000 ­ 2006 Etapa 1: Informe que comprende del 01 - diciembre - 2000 al 31 - diciembre ­ 2005, Mexico. Rosenbaum, P. and D. Rubin (1983), "The Central Role of the Propensity Score in Observational Studies for Causal Effects", Biometrica, 70 (1), pp. 41-55. Tan, H. and López-Acevedo, G. 2004. Evaluating SME Training Programs: Some Lessons from Mexico's CIMO Program. Mimeo, April 15, 2004. The World Bank, WBI Tan, Hong and Gladys Lopez-Acevedo (2007), "How Well Do SME Programs Work? Evaluating Mexico's SME Programs Using Panel Firm Data", World Bank Institute and LAC, Washington DC; also Policy Research Working Paper 0-3339. The World Bank (2008), " Evaluating Small and Medium Enterprise Support Programs in Latin America", Research Grant Proposal, mimeo. Secretaría de Economía, Reporte e-SINafinsa. 44 ---------- (2001) Programa de Desarrollo Empresarial 2001-2006, Mexico. ---------- (2006a) Libro Blanco. Fondo de Apoyo Para la Micro, Pequeña y Mediana Empresa (Fondo PYME), Mexico. ---------- (2006b) Informe de Rendición de Cuentas 2000-2006. Etapa 1: Informe que comprende del 01-diciembre- 2000 al 31- diciembre-2005, Mexico. ---------- (2006c) Informe de Rendición de Cuentas 2000-2006. Etapa 2: informe que comprende del 1 de enero al 30 de junio de 2006, Mexico. ---------- (2007a) Programas y Apoyos para las Micro, Pequeñas y Medianas Empresas y los Emprendedores. Presentation made for CONCANACO SERVYTUR, September 21, 2007. Secretaría del Trabajo y Previsión Social (2006), Libro Blanco PAC, Mexico. Soto, Jorge, Hugo Fuentes, and Victor Aramburu (2009), Review of SME programs in Mexico. 45