World Bank Group Support for Small and Medium Enterprises A Synthesis of Evaluative Findings © 2019 International Bank for Reconstruction and This work is a product of the staff of The World RIGHTS AND PERMISSIONS Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in this Because The World Bank encourages Washington, DC 20433 work do not necessarily reflect the views of The dissemination of its knowledge, this work may be Telephone: 202-473-1000 World Bank, its Board of Executive Directors, or reproduced, in whole or in part, for noncommercial the governments they represent. purposes as long as full attribution to this work is Internet: www.worldbank.org given. The World Bank does not guarantee the Photo: Stanislas Fradelizi / World Bank Any queries on rights and licenses, including accuracy of the data included in this work. The boundaries, colors, denominations, and subsidiary rights, should be addressed to other information shown on any map in this work World Bank Publications, The World Bank Group, 1818 do not imply any judgment on the part H Street NW, Washington, DC 20433, USA; of The World Bank concerning the legal fax: 202-522-2625; e-mail: pubrights@worldbank.org. status of any territory or the endorsement or acceptance of such boundaries. World Bank Group Support for Small and Medium Enterprises A Synthesis of Evaluative Findings September 16, 2019 Contents Abbreviations .....................................................................................................................................................iv Acknowledgments ........................................................................................................................................... 5 Management Response .................................................................................................................................vi 1. Background, Objective, and Definition................................................................................................. 1 Defining SMEs................................................................................................................................................5 2. Is the World Bank Group Doing Things Right? ................................................................................. 8 Main Forms of SME Support and Their Effectiveness .......................................................................8 3. Findings on Specific SME Approaches ............................................................................................... 18 4. Should the World Bank Group Distinguish SMEs As a Firm Size Category to Support? ... 28 5. Lessons on SME Support from Other Multilateral Development Banks ................................. 31 6. Conclusions .................................................................................................................................................. 34 Bibliography ..................................................................................................................................................... 36 Boxes Box 1.1. Main Findings and Recommendations of the FY 2014 IEG Evaluation on SMEs .......... 3 Box 1.2. World Bank Group Research Definition of Small and Medium Enterprise .....................7 Box 2.1. Advisory Services and Investment Services ............................................................................. 13 Box 3.1. Matching Grants for Innovation and Entrepreneurship ..................................................... 23 Figures Figure 2.1. Distribution of Evaluated SME Projects by Region, Country Income Level, and Institution ........................................................................................................................................................... 10 Figure 2.2. Sectoral Focus of Small and Medium Enterprise Support Projects ........................... 11 Figure 2.3. Types of Small and Medium Enterprise Support Interventions, by Institution ...... 12 Figure 2.4. Problems Identified (percent of operations) ..................................................................... 15 Tables Table 1.1. IFC Micro, Small, and Medium Enterprise Definitions and Proxies ................................ 6 Table 2.1. Recent World Bank Group Research Findings ..................................................................... 9 Table 2.2. Small and Medium Enterprise Projects that Tracked Gender Indicators ................... 13 Table 2.3. SME Projects that Tracked Gender Indicators ....................................................................14 Table 3.1. Project Outcome Ratings of Successful by Intervention Type and Institution ......... 18 Table 5.1. Challenges Raised in MDB Evaluations of their SME Support Programs .................. 32 iii Abbreviations BDS business development services FSAP Financial Sector Advisory Program IEG Independent Evaluation Group IFC International Finance Corporation M&E monitoring and evaluation MDB multilateral development bank MIGA Multilateral Investment Guarantee Agency MSME micro, small, and medium enterprise PIU project implementation unit SIP Small Investment Program (MIGA) SMEs small and medium enterprises All dollar amounts are U.S. dollars unless otherwise indicated. iv Acknowledgments This note was authored by Andrew Stone, Adviser, under the supervision of Stoyan Tenev, senior manager, and José Candido Carbajo Martinez, director. It reflects substantial inputs from Eric Oldsman (Nexus Associates, Inc.), and Nadia Paola Ramirez Abarca (Independent Evaluation Group). Additional input was received from Rebecca Xuetong Wang. It was peer reviewed by Paul Holden. Emelda Cudilla provided administrative support. The author acknowledges the helpful inputs of colleagues in the World Bank and the International Finance Corporation in conceptualizing and refining this work. v Management Response The Independent Evaluation Group (IEG) thanks management for the detailed and thoughtful comments, a number of which will assist IEG in improving and strengthening the report. Where a comment does not take account of the synthesis report format and methodology discussed in the concept note and at meetings with World Bank and International Finance Corporation (IFC) at the concept state, IEG simply notes this in its response. In this regard, any comments suggesting IEG treat unevaluated work of the Bank Group or engage in an accountability exercise to report on Management Action Rapid (MAR) fulfillment are noted as outside the scope of a synthesis report. This matrix references comments received by IEG both formally and informally and was constrained by the extreme delay in the formal delivery of comments by Operations Policy and Country Services. Management Comment IEG Response World Bank Management Para. 3: … SME [small and medium enterprise] The text now recognizes underlying heterogeneity. The topic of defining size composition is by nature highly heterogeneous criteria is inherent to size-based enterprise support approaches. The ability to and dependent on, among other things, the eco- distinguish SMEs from non-SMEs is necessary for any assistance based on nomic structure, income level, and endowments size-based targeting. The learning presented in the report is entirely consistent of a given economy. …We suggest that the report with the World Bank’s current thinking “moving beyond firm size as th e main recognize this heterogeneity more explicitly. The dimension to be targeted,” indeed acknowledging a wealth of recent World current thinking at the World Bank is moving be- Bank work supporting this. yond firm size as the main dimension to be targeted. Rather, the focus is to design and articulate a specific approach to support SMEs to enhance firm growth and the creation of quality jobs by the private sector in the economy. vi Management Comment IEG Response 4. [T]he translation of some of the trends IEG agrees that, within the limits of the synthesis format, as much granular identified into useful operational guidance at the learning as possible should be extracted. This is constrained to what is in the individual-project level could be strengthened. source evaluative material and caution must be exercised in generalizing from The report attempts to do this in parts, especially small samples, as other comments (for example, on selection mechanisms in where it offers specific project examples of how matching grant schemes) indicate. The team is able, to refine and add details success was achieved. It would be helpful if the in only a few places, as per other responses below. report could provide more detail or in-depth analysis of the presented findings of previous research and evaluations. [subsequent examples] 5. Although the report makes general references The concept note defined the scope and method-ology of the evaluation and to deficiencies in design and monitoring, it stops this was discussed with World Bank and IFC management. The synthesis short of providing examples of tools and format cannot incorporate ongoing or anticipated work that has not been platforms to support effective project evaluated or published, as synthesis does not involve any new evaluation and implementation. We wish to draw the team’s there is no existing evaluation available on ComPEL. Further, the National attention to the Finance, Competitiveness, and Quality Infrastructure Development in Ethiopia ComPEL proposal projects a Innovation Global Practice ComPEL impact work program running through fiscal year (FY)21. The Ethiopia Women evaluation initiative that is working jointly with Entrepreneurship Project (P122764) closes at the end of the current calendar Development Impact Evaluation with a view to year. helping project teams mainstream impact evaluations in Finance, Competitiveness, and Innovation operations. Another example is the Ethiopia Women Entrepreneurship Project (P122764). 6. We would suggest the report include and Thank you. IEG finds this article relevant in considering the design of discuss the findings of the World Bank Working evaluation of matching grant schemes. Paper 6296 “Learning from the experiences vii Management Comment IEG Response [experiments] that never happened: Lessons from trying to conduct randomized evaluations of matching grant programs in Africa.” 6. [T]he report’s conclusion that a “first-come- IEG will revise this finding contained in the microevaluations but, given the first-served approach” is a successful feature of small “n,” this has limited generalizability. The World Bank’s own major matching grants requires further examination: publication on the topic is inconclusive: This seems inconsistent with existing operational “Selection mechanism. Out of the 106 projects in the sample, 85 have evidence as well as with some findings from explicitly specified the type of selection mechanism based on which grants behavioral economics (that entrepreneurs were awarded to beneficiaries, and the vast majority of them, 80 projects, use a procrastinate applying unless they have clear first-come, first-served selection mechanism that awards grants in order of deadlines). application to any firm that passes the preidentified eligibility criteria. Only five projects have a true competitive selection mechanism that assigns scores to each proposal and gives grants to the top scoring firms down the list until the money allocated for that round runs out. While selecting the “best” projects is intuitively appealing, the capacity of the implementing agency to do so can be questioned, especially if projects are evaluated on the basis of their additionality and spillover potential, as we argue should be the case, rather than simply growth potential. On the other end, a competitive selection mechanism could generate more interest if the grants are perceived to be limited and their awarding time-bound. Case study 9 looks at an example of a project with competitive selection. Due to the small number of projects choosing to select grantees with a competitive process, it is hard to assess whether the choice of selection mechanism actually brings any systematic benefits.” Hristova, Diana, and Antoine Coste. 2016. How to Make Grants a Better Match for Private Sector Development. Washington, DC: Bank Group. viii Management Comment IEG Response 7. Para 3.29 “….for example, in the Russian The reference was inadvertently omitted and is now added. This is a direct Federation, where IFC’s and Bank’s work in quotation from the IEG published major evaluation “Knowledge -Based support of micro, small, and medium enterprise Country Programs: An Evaluation of World Bank Group Experience” (2016). (MSME) financing had “less coherence and coordination.”…. The source of this quotation is not included in the report, and we do not find this to be an accurate characterization of the Russian Federation program… 8. With reference to Georgia’s Country IEG’s text is faithful to the source, the 2014 IEG CPSCR Re view. To nuance, it Partnership Strategy Completion Report (World can elaborate that this was called a “minor weakness.” In detail, it notes the Bank 2014b) para 3.29, we find the findings in the failure to achieve an outcome indicator relating to “the high number of firms original report to be more nuanced than presented indicating a problem with access to finance in the BEEPs [Business in the draft synthesis report. ….[T]he strategic Environment and Enterprise Performance Survey]” and states: “To some objectives were complementary rather than extent this reflects an inconsistency in the Bank Group’s approach where the contradictory. emphasis placed on improving the resilience of the banking sector in the second outcome of the first pillar, was at odds with the desire to increase lending to businesses. In the event neither of these objectives were met. The appropriate sequencing would have been first to focus on the health of the banking system and only later to try to promote improved access for SMEs.” 9. With reference to table 1.1 on page 5 (where The table refers to IFC-published standards, based on the IFC website. IEG Turkey is mentioned in the note below the table) finds the same table and footnote now at we note that the credit line limit reaches https://www.ifc.org/wps/wcm/connect/industry_ex $3.5 million for medium enterprises, although the t_content/ifc_external_corporate_site/financial+inst text in the note claims below $2 million. You may itutions/priorities/ifcs+definitions+of+targeted+sect ors wish to revise this figure. ix Management Comment IEG Response We suggest a minor edit to the synthesis note Do the comments reflect recent changes not yet published or internal policies under table 1.1., on page 5, as follows: … at variance with IFC’s published norms? Or does it reflect differences between replacing the wording “all EU accession World Bank and IFC? countries” with “newer EU accession countries.” The IEG team could consider including the Thank you. The team has now reviewed these works. As noted, the team following research in the synthesis: excludes literature focused on support for microenterprises as opposed to Bruhn, Miriam, Rekha Reddy, and Claudia Ruiz. SMEs. In the case of Bruhn, Reddy and Ruiz (2018) the characteristics of 2018. “Sowing the Seeds for Rural Finance: The beneficiary firms is not discussed, and the term SME does not appear in the Impact of Support Services for Credit Unions in article. In the case of McKenzie et al. the title is self-explanatory and the pilot Mexico.” World Bank Policy Research Working firms all had fewer than 10 employees. Bruhn and Loeprick (2014, WPS 7010) Paper 8483, World Bank, Washington, DC. again focuses on firms that are by Bank Group norms microenterprises, McKenzie, David, Suresh de Mel, and Chris finding an effect of specialized tax policy on the registration of micro Woodruff. 2019. “Micro-Equity for businesses. Microenterprises.” Policy Research Working Bruhn, Karlan, and Schoar (2018) is already referenced in box 1.2 noting Paper 8799, World Bank, Washington, DC. articles that appear to be about SMEs but are actually focused mostly on Bruhn, Miriam, and Jan Loeprick. 2016. “Small microenterprises. Business Tax Policy and Informality: Evidence from Georgia.” International Tax and Public Finance 23 (5): 834–853. Bruhn, Miriam, Dean Karlan, and Antoinette Schoar. 2018. “The Impact of Consulting Services on Small and Medium Enterprises: Evidence from a Randomized Trial in Mexico.” Journal of Political Economy 126 (2): 635–687. x Management Comment IEG Response International Finance Corporation Management Comments 13. For this reason, the conclusions of the The team agrees that a synthesis note by convention is confined to examining synthesis note in some instances repeat the same evaluated projects. findings and recommendations of the 2014 IEG However, the evidence underpinning the synthesis note is all new in the sense review on SMEs. It would be helpful for IFC to that it was not available when the 2014 IEG evaluation was written (that is, in understand the effectiveness of the actions we 2013). The evidence base extends to years subsequent to that considered by the have taken in response to the 2014 report, starting 2014 evaluation. The synthesis format does not permit evaluation using more with projects that were implemented and recent and as yet unevaluated projects. IEG would welcome a more active completed at least two years after the major partnership through a learning engagement to consider recent changes. evaluation. 14. At the time the concept note was developed, The discussion at the time of the concept note related to the inclusion of older IEG shared with the Bank Group the list of IFC projects already evaluated at the time of the original evaluation. They were Investment projects that would be the sample. struck from the list. At the time of the concept note meeting, IEG requested That initial list had 33 Investment Services SME- IFC’s own identification of SME projects but did not receive a project list. In related projects…21 investment projects were response to comments, IEG has reviewed all projects (World Bank, IFC, flagged as Micro in IFC’s systems. To provide Multilateral Investment Guarantee Agency [MIGA]) and eliminated those more clarity on the methodology, it would be found not to involve SME support (see below). The table of projects can be helpful if the SME project selection criteria shared bilaterally. applied for the analysis is mentioned upfront in the report, along with a final Advisory Services/Investment Services project list included in the annex. Related to this point, it would be useful for the full list of 114 Bank Group SME support. projects be included in the annex as well. 15. The report is very much focused on access to The team appreciates that IFC works through multiple channels to support finance, while IFC is also working on other key SMEs. This report was focused on what could be learned from the evidence xi Management Comment IEG Response success factors for SMEs, providing them with base defined in the concept note. Value chain work and business development access to markets and skills. IFC is linking SMEs services (including skills development) are discussed within the limits of this into value chains, for example helping small evidence base. farmers to integrate into the value chains of major agribusiness players, and SME technology providers to connect with private hospitals (TechEmerge in India and Brazil). IFC also has specific training programs to increase SME competitiveness, such as Business Edge and the next generation of skills development platforms currently under development. 16. As part of Anticipated Impact Measurement IEG takes note of this as yet unevaluated work but again notes the limited and Monitoring (AIMM), IFC has developed a scope of the synthesis format. Reference to the deep dive is now added. framework to assess SME support for improved access, affordability and quality of financial services. This is also closely aligned with IFC’s financial inclusion deep dive on SME finance, which outlined a conceptual framework for IFC’s SME finance operations (Investment Services and Advisory Services). 17. Advisory Services-Investment Services The IEG has reexamined the evidence and finds that most of IFC revised Advisory draft report makes little reference to projects Projects are linked to an existing IFC investment operation (77 percent). Of 21 where advisory was linked to investment and Advisory Services projects they were linked to Investment Services, 6 had vice-versa. On average, IFC investment projects successfully achieved their development outcome rating (that is, obtained a delivered in tandem with advisory services development outcome rating of moderately satisfactory or above). Looking achieved better development outcomes … so it from the point of view of evaluated IFC investments, only four of the revised xii Management Comment IEG Response would have been useful if the Synthesis Report portfolio of evaluated IFC investment operations were linked to an existing also covered some aspects of such Investment IFC Advisory Project (4 out of 25). As requested, IEG has added a box on this Services-Advisory Services links in SME projects. finding. 18. IFC management believes that it is important The suggestion that IEG comment on IFC’s unevaluated work exceeds the for the resulting report to at least make some scope of a synthesis report. However, we will try to include more of them in a mention of ongoing and forward-looking list of initiatives for which there is not validated evaluative material to draw elements that IFC is currently engaged in. With on. this in mind, we are providing IEG with more information on some of our efforts and engagements through a summary table with attachments at the end of this document. These include more recent entrepreneurship initiatives including the Startup Catalyst program and Techemerge, as well as the ISC update and IFC + Digital Entrepreneurship (both presentations come from IFC’s SME Task Force). We hope that you will find them helpful… 19. It is … very difficult for IFC and the World This is the situation described in the synthesis note. Bank to apply one simplified definition across its operations, given different institutional needs as well as the World Bank’s engagement with different agencies within a country. Within IFC we use a standard definition across all our operations with some degree of flexibility (in loan proxies) to distinguish between more advanced economies and lesser developed ones. xiii Management Comment IEG Response 19. It is … very difficult for IFC and the World This is the situation described in the synthesis note. Bank to apply one simplified definition across its operations, given different institutional needs as well as the World Bank’s engagement with different agencies within a country. Within IFC we use a standard definition across all our operations with some degree of flexibility (in loan proxies) to distinguish between more advanced economies and lesser developed ones. 19. It is … very difficult for IFC and the World This is the situation described in the synthesis note. Bank to apply one simplified definition across its operations, given different institutional needs as well as the World Bank’s engagement with different agencies within a country. Within IFC we use a standard definition across all our operations with some degree of flexibility (in loan proxies) to distinguish between more advanced economies and lesser developed ones. 21. In addition to the SME Launchpad, other Para. 1.5 will be modified to correct the reference to We-Fi and additional recent SME programs worth mentioning include programs will be mentioned. the Small Loans Guarantee Program, the Women Entrepreneurs Opportunity Facility, and the Global SME Finance Facility. Please also correct “Women Entrepreneurs Finance Initiative.” 22. Para 1.6 (Blended Finance Facility) we suggest Agree. IEG will revise the text. the following edit: “ … created the IDA xiv Management Comment IEG Response [International Development Association] PSW BFF … ” 23. Paragraph 1.11 (SME Definition): The section The synthesis note is describing a situation factually and discussing its recaptures the findings and recommendation for a implication. There is no recommendation here. consistent SME definition across the Bank Group that was in the 2014 IEG review of the World Bank Group support to SMEs. IFC continues to maintain the same stance as in the Bank Group Management Response. 24. Chapter 2 (“Is Bank Group Doing Things The team is uncertain to which text this comment pertains. IEG did not, in the Right?”) A general observation: Another aspect to context of the sources for this synthesis note, find the basis for this conclusion. consider would be adopting a flexible However, a number of recent IEG macroevaluations have found benefits in programmatic approach… such complementary and sequential approaches, as noted in the conclusion. 25. Paragraph 3.5 The second statement is Agree, the text is erroneous and has been corrected to reflect the true confusing as it is unclear if the 29 percent refers to percentages. Advisory Services projects with positive development outcomes, or if what was actually meant is that Advisory Services had a 71 percent positive development outcome rating. 25. …It would be beneficial to distinguish IEG appreciates this suggestion. Although the number of evaluated IFC between the performance of SME programs in projects in fragile and conflict-affected situation countries was too small to different contexts such as fragile and conflict- make robust comparisons, the team has added a discussion of differences in affected situations and low-income countries. problems encountered by projects in IDA and non-IDA countries. 25. Regarding the 50 percent success rate for IFC Although the original IEG SME evaluation (2014) found the performance of SME credit lines, we suspect that the success rates Investment Services projects was higher with accompanying Advisory xv Management Comment IEG Response are influenced by whether there was Services, the team did not find new evidence of this in the projects reviewed in accompanying advisory projects or not… this synthesis. 26. IFC does estimate jobs created and supported The team has clarified the text to indicate that this reflects IEG’s review of by the SMEs it finances. … Many of our financial evaluation documents associated with evaluated World Bank lending and IFC intermediaries in emerging countries do not have investment projects, summing reported numbers of jobs and loans. The the management information system capabilities projects reviewed would not have been subject to the AIMM process, but the to gather this information and share such a large team can note that AIM now includes and estimation of job creation. volume of data with us. The majority of them do not track the number of jobs at the SME level on an ongoing basis. … IFC prefers to conservatively estimate the number of net jobs that may have been created by SMEs receiving financing from our client FIs. These estimates are now also included in the AIMM write-up for SME projects where job creation is a key component of the intervention … All multilateral development banks face similar challenges when gathering data/reporting on metrics like jobs. IFC … IFC would appreciate knowing the source of the data on which the 68,000 number is based. 27, 28. Management thanks IEG for recognizing its Because this is not an accountability exercise, we are not reporting on the final efforts to include the “gender lens” in its status of MAR actions from the 2014 evaluation but do now note that many operations. The note informs that gender was actions have ensued under the MAR since the 2014 report and have added to treated in 57 percent of the evaluated projects. IFC the list of newer programs mentioned that are not yet evaluated by IEG. has provided guidance to both Investment and Advisory Services staff (in 2018 and 2016, xvi Management Comment IEG Response respectively) on the criteria to flag projects that help close gender gaps. Please note that only for projects that are gender-flagged, that is, projects that include the following: (i) a gender gap analysis; (ii) an intervention (that is, a specific set of activities targeting women, such as World Banking on Women, insurance projects and so on); and (iii) a sex-disaggregated indicator, project guidelines currently require IFC staff to include a gender-specific indicator, such as “Number and Value of Women-SME loans outstanding,” in Board Papers and the Development Outcome Tracking System /AIMM systems…. [I]t is expected that in future years IEG will see more projects that address gender gaps as well as gender-specific indicators and results. 29. Table 2.2, Page 11: The table excludes Agree. IEG has revised the table to include advisory services. 50 percent of IF Advisory Services from the gender tracking, and Advisory Services projects in the evaluated portfolio mentioned gender and it is unclear why. 44 percent had an indicator on gender. 30. Chapter 3 IFC would appreciate learning what IEG counted as SME support projects that sought to increase SME access to criteria was used by IEG to select the 27 finance or other benefits (even if only a part of the benefits went to SMEs). IEG Investment projects. As mentioned above a has revisited the project coding and reclassified 10 projects, including 2 IFC majority of the original 33 sampled were flagged projects, as false positives–Compartamos Guatemala (32770) and CAMESA II Micro (MI) in IFC’s systems. (33639). Some other projects included in the IFC portfolio supported both microenterprises and SMEs or clearly mentioned SME or “small enterprise” xvii Management Comment IEG Response beneficiaries. IEG attaches a table with language that identified projects as SME support. 31. Paragraph 3.4 (Compartamos Guatemala): This Agree, this was an error and the team has corrected this. is strictly a microfinance project with a microfinance institution. … there is no reference to SME lending. 32. Establishing and enforcing borrower This initiative is noted in the text. However, the team is unaware of an IEG eligibility criteria: [T]he Global SME Finance evaluation of GSMEF. Facility (GSMEF) was designed to incentivize Partner Financial Institutions (PFIs) to reach the targeted segment of SMEs that are credit constrained. This Facility has demonstrated how IFC can work with banks to develop and ensure that eligibility criteria to target an agreed segment of SMEs can be put in place, as shown on page 21 of the GSMEF Mid-Term Report One of the key metrics for gap assessment in Noted. AIMM is to look at the percentage of firms that are financially constrained and/or underserved. … As part of AIM IFC has included questions related to access to credit … 33. IFC agrees that the effectiveness of its Noted investment operations is accentuated when there is a good understanding of the PFI’s customer segmentation. That said, as the Report points out, xviii Management Comment IEG Response IFC’s PFIs in general have a heterogenous diverse portfolio…. 34. Risk-Sharing Facilities It would be helpful if IEG has edited this to indicate it is based on only the 2 RSFs evaluated. IEG could share the report that forms the basis of The team can add Small Loan Guarantee Program to the list of as yet the statement “IFC’s evaluated risk-sharing unevaluated initiatives and looks forward to its future evaluation. facilities targeting SMEs performed poorly.” In response to IFC’s own review of its risk-sharing facilities, which were…not impactful primarily because of lack of use, IFC, in FY18, introduced the Small Loan Guarantee Program, which incorporates operational support and advisory services for clients…. 35. Paragraph 3.10 (Bank of St. Lucia): The IEG will correct the reference in this paragraph, which should cite IEG’s 2016 statement on IFC’s efforts to support the World evaluation Cluster country program evaluation on small states (Vol. 1): Regional Bank of St. Lucia is inaccurate. We would like to program evaluation of the Organisation of Eastern Caribbean States —Antigua and inform IEG that in 2014 IFC started an Advisory Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the project “Secured Transactions and Collateral Grenadines. The IEG evaluation’s finding is accurately represented in the text. Registries program in the Caribbean” to support IEG reminds IFC that unevaluated Bank Group work is out of scope. countries establish modern secured transactions systems and collateral registries for security interests in movable property, with St. Lucia being one of the first countries in the scope of this program. In FY20 IFC will also deploy resources to stablish a modern Collateral Registry in St. Lucia, after the regulatory framework has been xix Management Comment IEG Response modernized as a result of IFC’s efforts in the country. 37. Paragraph 3.13 (SMEs have needs in multiple IEG used an example from a small state experience to illustrate a broader areas): Care should be taken when basing global point. Many sources clearly show multiple needs of SMEs, including much of conclusions on experiences in small island the literature and the many enterprise surveys. economies. 38. Paragraph 3.16 (IFC Fund): Care should be Agree, team will modify the language. taken when basing global conclusions on a single fund experience. 39. Paragraph 3.23 (Credit Information) The last A careful reading would show the finding on public credit registries in no sentence suggests that public registries are less way diminishes Ayyagari et al’s identification of a need to better understand effective although it then states that more research which aspects of credit information systems contribute most to improving the is needed to see which aspects contribute most. allocation of credit 39. Paragraph 3.23 There is a typo in the third Thanks, the team will correct it. sentence where ‘private’ is repeated twice. 41. Paragraph 3.26 – 3.31 It should be known that Noted. The suggestion that IEG comment on IFC’s unevaluated work is not an SME Entrepreneurship Task Force led by the within the scope of the synthesis report. office of the IFC Chief Executive Officer aims to (i) systematically embed SME diagnostics in frameworks such as Financial Sector Assessment Program and CPSD, (ii) start identifying interventions that address the findings of these frameworks, and (iii) strengthen coordination within IFC and with the World Bank on recommending and implementing these interventions. xx Management Comment IEG Response 42. It would have been beneficial to also consider This section is accurately reflecting the findings of the 2016 IEG evaluation of IFC’s SME push program…There are also other, Knowledge-based country programs. A reference has been added. more specific examples of IFC’s work in value chain financing. 43. Paragraph 3.28 (IFC’s experience in the A reference to the source will be added. This finding is based on the 2016 Russian Federation): IFC would appreciate IEG evaluation Knowledge-Based Country Programs: An Evaluation of World receiving the source of this assessment. IFC would Bank Group Experience. say that its support for Russian SMEs through the P. 61 “There was less coherence and coordination in the World Bank and IFC financial sector has been generally successful financial sector work in the Russian Federation in support of the CPS [Country Partnership Strategy] expected outcome of increased access to finance for entrepreneurs. The IFC work on securitization and insurance was not aligned with CPS emphasis on MSME financing, which was a major focus of IFC investments and supported by a World Bank study. The work on securitization did not produce the outcomes envisioned due to the financial crisis. In the case of insurance, the consolidation of the regulatory agencies left the ongoing IFC work uncertain on the way forward.” 44. Paragraph 3.29 (Georgia CPS): … IFC played a Noted. critical role in mobilizing debt and equity financing to support Georgian banks’ liquidity and capital needs. IFC 45. Para 4.4 Turnover is the most pragmatic IEG is not advising banks (or other financial intermediaries) how to run their approach given that it gives them an idea of the businesses. IFC is the intended audience in this case. firm’s wallet size and therefore the potential business that they can derive from the firm. It is not clear how practical the mentioned alternative measures would be in segmentation. xxi Management Comment IEG Response 45. We strongly recommend that IEG take into IEG added a reference to this database. It does not appear to comprise part of account the updated MSME Economic Indicators IFC’s operational policy. database of 2019. 45. It should be acknowledged in the report that Where in the text is it suggested there is a “unanimous” view in the literature? the literature regarding SMEs and Jobs is far from The text is not intended to assert this. unanimous. 45. Fast Growth. [A] a good environment to allow Agree. The synthesis faithfully reports the authors’ conclusions for reform. small start-ups to grow is important. In many countries a few large firms dominate markets and access to scarce government support, constraining the ability of gazelles to emerge. 45. Economic Distortions of Firm Growth: [O]ne The concept note laid out the approach to the literature and the synthesis note should also consider that governments listen to faithfully reported the relevant findings. the big firms that have time and resources to lobby. 46. Para 5.1: The text that follows in the Thank you, the team will correct this. paragraphs below refers to European Bank for Reconstruction and Development, seems that the European Investment Bank was inadvertently mentioned here. 52. Point 1 (Inconsistent definition of SMEs limits The team points out that IFC has often cited in support of its SME work learning): In line with our prior comments, we do articles whose definitions of SME do not align with its operational definition not think IEG has made the case for this. of SME. Nor do definitions used in the enterprise surveys comply with IFC’s definition. The applicability of learning from research is constrained by these inconsistencies. xxii Management Comment IEG Response 52. Point 2. IFC credit lines are structured very How does this pertain to the lack of rigorous research findings on SME credit differently from the World Bank’s. IFC also has lines? different monitoring and evaluation frameworks from the World Bank in these projects… 52. Point 5. The point about the benefits of Well noted, but the referenced evaluations did not necessarily focus primarily combining both Advisory Services and on the IFC portfolio. Investment Services are well received but could have come up much earlier in the report. 52. Point 6 IFC possesses robust monitoring IEG’s point is that “Learning would require robust and longer-term approaches to capabilities through the Development Outcome monitoring and evaluation.” To the extent that IFC is moving in that direction , Tracking System, and the ongoing deployment of presumably it accepts the point. AIMM will establish even more refined capabilities to monitor development impact results for projects. 52. Point 8 IFC has increased its efforts to improve Agree. This point is about generic problems across multilateral development frameworks, better define beneficiaries, include banks, not specifically about IFC. eligibility criteria in legal agreements, guide staff on flagging SME/Gender projects and comply with procedures of its project governance. Multilateral Investment Guarantee Agency Management Comments 53. The synthesis report makes no references to Management comments accepted by IEG are incorporated into the evaluations Bank Group management comments on the used as an evidence base for the synthesis report. The scope of the synthesis relevant Evaluation Reports as well as progress in note does not incorporate additional materials, nor is this an accountability management actions for implementing exercise that tracks the MAR. Recommendations agreed between IEG and Bank xxiii Management Comment IEG Response Group management, tracked and assessed through the MAR system. 54. [T]he synthesis report notes the The Small Investment Program (SIP) has been subject to important reform that recommendation for reforming the MIGA Small was acknowledged through the MAR process. However, this not an Investment Program (SIP, box 1.1, 3). MIGA notes accountability exercise that tracks the MAR. A sentence was added to the box in response that the Agency has successfully noting that progress was achieved on the MAR action plan. reformed SIP after an internal review that informed the FY14–16 MIGA Strategic Directions formulation and drew on the SME evaluation findings. 55. MIGA notes that Gasynet is a trade facilitation Agree, this was included in error due to a reference in the project to benefiting project in Madagascar without any SME focus and both small and large enterprises. The team will exclude this project as a false hence should be removed from the Report. positive. 57. (para 3.8, 17), the report states that the MIGA’s comment is fully consistent with IEG’s text. MIGA guarantee provided to Raiffeisen Leasing (Serbia) against the risk of transfer restrictions and expropriation of funds did not increase leasing, particularly to SMEs; instead, the loan was used to support the institution in surviving the crisis. …MIGA notes from the IEG Validation Note that although the project fell short of achieving some of the expected development impacts, its survival contributed toward the financial sector’s stability xxiv Management Comment IEG Response and could enhance the prospect of longer-term postcrisis financial sector development. 59. MIGA also suggests that the Report could IEG has now added features of successful MIGA projects. IEG followed the draw on the positive evidence available in evidence in its report, noting MIGA’s 62.5 percent positive success rate, and other IEG-validated MIGA project evaluations the positive experience of coordination of MIGA with Bank Group and IFC in regarding SME support that were reviewed for Georgia, for example. the Report. 60. MIGA also notes the findings and This is outside the scope of the current work. conclusions from project evaluations to be consistent with those of IEG’s (2011) Macro evaluation of MIGA Financial Sector Guarantees (MIGA’s Financial Sector Guarantees in a Strategic Context, World Bank 2011). Management Response Annex Note: IEG counted as SME support projects that sought to increase SME access to finance SMEs (even if a only a part of the loan went to SMEs), projects that supported the capacity of Financial Institutions to serve SMEs or those that support scale up of intermediary lending to SMEs as well as those that sought to deliver other services to SMEs. Table MR.1. Revised Table with Rationale for Inclusion of IFC Projects as Small and Medium Enterprises Project ID Project Name Aspect Indicating SME Support 33643 TRM Microfinance A percentage went to SMEs, which were loans of up to $10,000. PES, p. 8 31661 FINCA Kyrgyz 2 As of 2016 about 90% of the loans were focused on microfinance and 10% were SME loans, maintaining the focus on small loans. 33362 Finca Tanzania 2 IFC Portfolio Officer visited 4 SME clients and 4 individuals who were part of group lending (EVnOTE, P. 14). 31289 NBD SME Frontier The project was expected to increase NBD’s SME portfolio by 60% by the end of 2016. xxv Project ID Project Name Aspect Indicating SME Support 31871 Patria Credit The project was expected to have the following development impact: (1) enhance Patria’s capacity to support productive micro and small enterprises (EvNote, 2) 30905 Progresemos III Progresemos provides microloans to small businesses run by women in rural areas. 31449 Vitas The PAD said that the project will contribute to economic growth in Lebanon by increasing access to finance for MSMEs 30345 Xinjiang MCC The project aimed to support the growth of the company and to increase access to finance for MSMEs in the region. 29366 CHUEE SME IB Has an outcome indicator targeting SMEs 30178 FATEN Has an outcome indicator targeting SMEs 29679 Regional CL Has an outcome indicator targeting SMEs 29386 Shenzhen VTB Has an outcome indicator targeting SMEs 29413 Abank Sub Loan Has an outcome indicator targeting SMEs 27694 AMSME BOA - Africa MSME Finance Description: The Program is an institution-building initiative designed to enhance capacity and Tanz speed up the transfer of know-how to banks serving micro,small, and medium enterprises ("MSME Banks" or the "Program Banks") in sub- Saharan Africa (PAD, P. I) 28471 ANZRoyal RSF old Has an outcome indicator targeting SMEs 28787 BBVA Paraguay CL Help deepen access to finance for smaller and medium-size enterprises, particularly in the agribusiness sector, which traditionally have limited access to long-term funding (achieved). 29916 BIB B loan Has an outcome indicator targeting SMEs 29985 Eurobank Serbia Has an outcome indicator targeting SMEs 28635 FDC Remittance Has an outcome indicator targeting SMEs 29683 FINCA Holding Portfolio growth: … the Company failed to scale up SME lending, which at the time of appraisal was a fundamental assumption in the financial projections and strategic goal of FINCA management (XPSR, p. 13). 26667 GroFin AF The project consisted of the establishment and operation of a specialized private equity fund called Grofin Africa Fund … for a targeted sum of $150 million. The aim was to provide risk capital … plus management-related inputs to very small enterprises in Africa. There were at the lower end or somewhat below the enterprises ordinarily within the investment scope of most “SME” financing institutions. 29951 MTB - EE loan Has an outcome indicator targeting SMEs 28867 ProCredit MSME Has an outcome indicator targeting SMEs xxvi Project ID Project Name Aspect Indicating SME Support 26262 BBSB Has an outcome indicator targeting SMEs 27051 SBDFT -MSME The Trust currently offers four main lending products: micro loans, mortgage loans, consumer loans, and small Loan business developmental loans. Small Business Developmental Loans— The loans under this category are small business oriented and on-lent for the production of rice and other cash crops. The borrowers under this program include farmers and small manufacturing enterprises located outside of Georgetown. These loans are comparatively bigger in size and the financing terms tend to be longer and adjusted to the crop growing cycle. (Board paper, 8). Note: MSME = micro, small, and medium enterprise. xxvii 1. Background, Objective, and Definition Highlights • Continued Significance. Small and medium enterprise (SME) support remains a major focus of the World Bank Group portfolio. • What is an SME? The categorization of SMEs matters in delivering the right types of services and benefits to the right types of firms. Yet the Bank Group remains inconsistent in defining SMEs and in applying definitions to target its support. Its research often doesn’t align with its operational definitions of SMEs making it more difficult to learn from experience. • Range of approaches. The Bank Group uses a broad range of approaches in its work with SMEs. It assists in general investment climate reforms and in providing targeted support to SMEs mainly in the form of credit lines, advice to financial intermediaries, risk-sharing and credit guarantee schemes, business development services, matching grants, and value chain links. • Business environment and business development services interventions. Evidence from evaluation and research indicates that SMEs benefit from general institutional and business environment reforms, and from some types of targeted approaches such as well-designed business development services support. However, there remains a limited understanding of the mechanisms that strengthen SME management and entrepreneurial capabilities to encourage growth in productivity and employment. • Evidence on approaches. The evidence is mixed on the effectiveness of matching grants and specialized tax regimes, and research evidence is absent on the Bank Groups’ leading approach: lines of credit. If SMEs are to benefit, evaluative evidence on value chain links suggest the need to pay attention to (i) design features oriented toward their inclusion; (ii) distributional effects along the supply chain; and (iii) risks for development outcomes associated with market structure. • Eligibility criteria. A key challenge in Bank Group approaches that use financial intermediaries to reach SMEs has been establishing and enforcing eligibility criteria that direct resources to truly finance-constrained SMEs. Defining effective SME eligibility criteria for credit lines is predicated on a good understanding and segmentation of the lending portfolio of participating financial institutions. • Performance factors. Key performance factors for the Bank Group’s SME portfolio include technical design of projects, quality of supervision and project implementation units, risk assessment and mitigation, the existence of relevant prior 1 analytic work, and client engagement and ownership. Several recent evaluations highlight superior performance where the Bank Group’s SME support is continuous, coordinated among Bank Group institution, and uses multiple instruments. • Longer-term effects. Sustainability is a key challenge to several SME support approaches. Effective learning requires longer-term approaches to track and evaluate the effects of SME interventions. • Size as a basis for support. World Bank research increasingly questions the use of size as a main criterion for providing support to enterprises – pointing to firm age, fast growth and productivity as alternative, more effective criteria. • Other MDBs and SMEs. The experience of other MDBs emphasizes and confirms the importance of strengthening the strategic and conceptual framework for SME support, better defining SMEs, strengthening eligibility requirements and targeting of SME support programs, tracking and monitoring of outcomes, and strengthening understanding of the additionality of interventions. 1.1 Small and medium enterprises (SMEs) play a substantial role in developing economies, generating employment and growth. As income levels increase, formal SMEs tend to comprise a larger share of the economy, while the micro and informal sectors recede. Economic growth creates opportunities often filled by SMEs entering or sometimes “graduating” from microenterprise status. 1.2 SMEs can face specific challenges that larger (or smaller) firms do not experience. Whereas microenterprises may entirely escape formal taxation and regulation, SMEs tend to face stronger obligations regarding taxes and regulations in spite of their substantial informal activity. Yet, due to their limited size, SMEs do not usually benefit from the economies of scale and bargaining power of large companies.1 Where market, policy and institutional failures thwart SME entry or growth, addressing these challenges can “level the playing field” to allow better functioning of markets, encouraging competition and productivity growth. This may allow SMEs to realize their potential in generating jobs and growth in developing economies. 1.3 However, the nature of SME support, the challenges SMEs face, and the true impact of SMEs on the economy are not without controversy: Is subsidized financial support to SMEs justified? Do SMEs suffer special challenges compared with other sizes of firms? Do SMEs make a unique contribution to growth and jobs or are they merely part of the natural ecology of firms? 1.4 The 2014 IEG evaluation “The Big Business of Small Enterprises” assessed the Bank Group’s SME activity while addressing the above questions in detail. Since then, 2 the number of World Bank SME projects increased substantially peaking at 505 in 2016, before declining somewhat in FY17 and more in FY18 to 336. Box 1.1. Main Findings and Recommendations of the FY 2014 IEG Evaluation on SMEs The 2014 Independent Evaluation Group work comprehensively evaluated World Bank targeted support to small and medium enterprises (SMEs). It identified a portfolio of approximately $3 billion per year dominated by financing through loans, leasing, investments, or guarantees but also involving advisory services to governments and financial institutions; business development, training and advisory services targeting SMEs; and activities to support SMEs through value chains, networks and clusters. It found that “any credible justification of targeted support to SMEs must be focused on establishing well-functioning markets and institutions, not simply providing a temporary supply of benefits to a small group of firms during a project's lifespan.” It found inconsistent definitions and approaches to SME support and a lack of appropriate justification, targeting and knowledge generation or learning. It therefore recommended that to make its interventions more effective, the Bank Group should do several things: i. Rationale. The International Finance Corporation, the Multilateral Investment Guarantee Agency (MIGA), and the World Bank should harmonize their SME approaches and make clear the objectives and analytic justification for such targeted small and medium enterprise support, how it relates to systemic reform, where it is appropriate, what main forms it will take, and how it will be monitored and evaluated. ii. Approach. World Bank Group management should refine its SME approaches to enhance relevance and additionality by shifting benefits from better-served firms and markets to frontier states (those with underdeveloped financial systems, especially low- income and fragile/conflict countries), frontier regions, and underserved segments. iii. Research. Bank Group management should institute a tailored research agenda to support and assist the clarification and refinement of its SME support approach. […T]his should produce more policy- and contextually relevant distinctions of the definition of SME; a better understanding of the dynamic contributions of SMEs to economic growth, employment, and poverty alleviation; deeper knowledge about how the design of interventions should vary contingent on country conditions; a project-relevant definition of the frontier; a clearer view of the correct sequencing and combinations of systemic and targeted [support]; and more rigorous analysis of the actual performance and impact of key types, combinations, and sequences of Bank Group and other donor interventions. iv. Work quality. Relevant Bank Group management should provide guidance and quality control so that every project document for Bank Group projects targeting SMEs will: a. Define the group of firms to benefit by measurable criteria such as the number of employees and annual revenues 3 b. Justify the definition of the beneficiary group targeted (which could be a subset of SMEs) based on country-specific evidence that this group suffers from size-specific market failures and constraints c. Specify and wherever appropriate embed in legal provisions the mechanism to reach the targeted group d. Include in its results framework and monitoring and evaluation framework the indicators of the impact of the project on the targeted group and on the constraints or market failures justifying the project. v. MIGA Small Investment Program. MIGA should reform its Small Investment Program by radically rethinking its approach to providing guarantees for investments in SMEs through the SIP program, considering either a merger with its regular program or a fundamental redesign to improve performance. As with other major evaluations, the SME evaluation was followed by a management action plan that was monitored over four years under the Management Action Review process, with completion or substantial progress found on most commitments. 1.5 The World Bank Group strategy continues to position SMEs as key vehicles to promote employment, value chain development, economic and social inclusion, and resilience in the face of fragility and conflict. In addition, recent programs have been initiated like the SME Launchpad, aimed at encouraging “innovative design concepts to support the SME agenda” and We-Fi (Women Entrepreneurs Finance Initiative), a multidonor partnership hosted by the Bank Group to address constraints to women- owned SMEs. IFC’s 2017 Deep Dive on SME finance envisioned a multiinstitutional approach conditioned on the maturity of country financial sectors.2 1.6 Recently, the International Development Association (IDA) 18 Replenishment created the IDA Private Sector Window (PSW) Blended Finance Facility, which has as one of its objectives to blend PSW funds with IFC investments “to support SMEs— which are critical to job creation and women’s empowerment.” Further, the Bank Group committed under IDA 18 in the theme “Jobs and Economic Transformation” to “systematically carry out impact analyses of SME and entrepreneurship programs across IDA countries to assess their overall impacts and differentiated outcomes for women and youth and … develop operational guidelines to inform future operations.” 1.7 This note synthesizes findings regarding SMEs and SME support from recent IEG evaluations, independent evaluations by other MDBs, and relevant Bank Group research. The objective of the exercise is to shed light on the following policy and strategic questions regarding Bank Group support to SMEs: • Does the Bank Group know which approaches serve effectively the needs of SMEs and in which contexts? 4 • What are the factors of success or failure of the main types of Bank Group interventions in support of SMEs?3 What lessons of experience can be drawn? • Should the Bank Group distinguish SMEs as a firm size category to support? 1.8 The findings in this synthesis also seek to inform the Bank Group evolving approach to supporting SMEs. Defining SMEs 1.9 Support to SMEs relies heavily on targeted development assistance that depends on there being a clear way to differentiate them from large enterprises and microenterprises. Its effectiveness thus depends on how well and consistently SMEs are defined. There are underlying challenges of heterogeneity across countries and sectors. Yet assisting SMEs depends on there being a clear differentiation of their needs and/or the means of supporting them from those offered to other size categories of enterprises. The rationale for SME support has tended to be couched in terms of their economic contribution and their employment impact (World Bank 2017; IFC 2016), yet neither can be ascertained without differentiating them from other sizes of enterprises. 1.10 However, the World Bank continues to have no definition of what an SME is. It generally relies on country standards, which vary widely4 and do not necessarily follow criteria rooted in empirical differences in the behaviors, characteristics or constraints of firms. IFC and the Multilateral Investment Guarantee Agency (MIGA) use a common standard definition, according to which “an enterprise qualifies as a micro, small or medium enterprise if it meets two out of three criteria of the IFC MSME Definition (employees, assets and sales), or if the loan to it falls within the relevant MSME loan size proxy”5 (see table 1.1). 5 Table 1.1. IFC Micro, Small, and Medium Enterprise Definitions and Proxies IFC MSME Definition MSME Loan Size Proxy Employees Total assets Annual sales Loan size at origination Indicator (no.) ($) ($) ($) Micro enterprise < 10 <100,000 $100,000 <10,000 Small enterprise 10–49 100,000 – <3,000,000 $100,000 – <100,000 <3,000,000 Medium enterprise 50–300 3,000,000–15,000,000 3,000,000– <1,000,000 or 2,000,000a 15,000,000 Source: https://www.ifc.org/wps/wcm/connect/industry_ext_content/ifc_external_corporate_site/ financial+institutions/priorities/ifcs+definitions+of+targeted+sectors Note: In comments received on this report, IFC notes its “credit line limit reaches $3.5 million for medium enterprises” in Turkey. IFC = International Finance Corporation; MSME = micro, small, and medium enterprise. a. The $2 million was for more advanced countries including Argentina, Brazil, Chile, China, Colombia, India, Korea, Mexico, Morocco, Peru, the Russian Federation, South Africa, Thailand, Tunisia, Turkey, and all EU accession countries – Poland, Hungary, Czech Republic, the Baltics, and Slovenia. 1.11 World Bank Group research also has no single standard SME definition. Most Bank Group research either conflates SMEs with microenterprises or uses criteria that are inconsistent with either the IFC and MIGA thresholds for defining SME or the World Bank’s country-specific definitions. This makes it difficult to assess the relevance of the research findings for SMEs and their generalizability across different contexts (box 1.2.). The size threshold used in enterprise surveys are consistent across countries but inconsistent with IFC, MIGA and many World Bank operational definitions.6 1.12 Acknowledging these problems in defining the target of SME support, the chapters that follow examine evidence generated about existing size-based approaches. Such approaches remain an important part of the Bank Group stated approach to enterprise support and of its portfolio. SME support also remains a central part of most client countries’ private sector development strategies. 6 Box 1.2. World Bank Group Research Definition of Small and Medium Enterprise Bank research sometimes relies on the thresholds defined in its enterprise surveys – namely “Firm size levels are 5–19 (small), 20–99 (medium), and 100+ employees (large-size firms).”7 This means that a substantial portion of “small” enterprises in World Bank research are microenterprises in IFC/MIGA operational terms, and a substantial portion of “medium enterprises” are small by IFC/MIGA definition. It further implies that most of the “medium” enterprises that IFC and MIGA support would be classified as “large” by the World Bank. Th is matters because the Bank Group is generating knowledge that may not be applicable to a large part of its SME practice. Some researchers clearly fit their research to the operational definitions of SME. Cravo and Piza (2016) conduct a meta-analysis of “the Impact of Business Support Services for Small and Medium Enterprises on Firm Performance in Low- and Middle-Income Countries.” These authors exclude studies of microenterprises because they are “thought to have a different nature compared with SMEs,” although they do not specify a lower limit in terms of employees or sales, only an upper limit (250 employees). Iacovone, Maloney and McKenzie (2019) test individual and group consulting services to auto parts manufacturers in Colombia with the aim of improving management practices and boosting performance. This study excluded businesses with less than 10 employees and included only firms with 10 to 310 employees. Yet a large amount of SME research uses evidence from microenterprises according to IFC/MIGA’s operational definitions. For example, • Anderson, Chand, and Zia (2016) examine mark eting training programs for “small businesses” in Cape Town, South Africa where the beneficiary firms have an average of 2.4 employees. • Bardasi, Gassier, Goldstein and Holla (2017) use a randomized experiment to assess the impacts of training of women with established “small businesses” in urban Tanzania which have on average 1.3 workers. • McKenzie and Woodruff (2015) explore the adoption of good business practices among “micro and small” firms where, “[i]n the combined sample, the median firm has zero employees.” • McKenzie, Assaf and Cusolito (2015) present the results of a randomized experiment with a matching grant program for “small enterprises” in the Republic of Yemen in where the median firm has five employees. • Bruhn, Karlan and Schoar (2016) examine the impact of management consulting services on total factor productivity and return on assets for “Mexican small and medium enterprises.” Its randomized trial is carried out on a group dominated by microenterprises (67 percent) with only 5 percent of firms being medium size. 7 2. Is the World Bank Group Doing Things Right? Main Forms of SME Support and Their Effectiveness 2.1 As discussed above, firms’ size can limit access to input and product markets and affect legal and regulatory compliance burden. Access to finance is often identified as the main constraint SMEs face. Credit lines, risks sharing facilities and advisory services to banks aim to address constraints in the credit markets for SMEs. Access to other inputs, such as business services, and access to product markets, is often facilitated through business development support, matching grants, and value chain/links support. Finally, certain types of investment climate reforms aim specifically to reduce compliance costs for SMEs. IEG found a limited amount of new research on some of the major instruments employed by the Bank Group to support SMEs (table 2.1). Not surprisingly, a major finding is a need for more research to deepen the understanding of how these mechanisms work under varying country conditions and of their impacts. For example, IEG could find no rigorous Bank Group research on SME lines of credit. 8 Table 2.1. Recent World Bank Group Research Findings Source Topic Finding Beck and Cull Financing Small formal enterprises, usually too big for microfinance institutions but (2014) not formal or established enough for banks, are especially affected by shallow financial markets. Divanbeigi and Legal and “[T]the efficiency and strength of laws, regulations and institutions is Ramalho (2015) regulatory relevant to domestic small and medium-size companies throughout their reforms life cycle.” Hristova and Matching grant Although most World Bank Group projects are successful, “the definition Costa (2016) schemes of success rarely reflects a measurement of broad and sustainable economic benefits that would justify the subsidization of private enterprises with public funds.”8 Programs need a specific identification of a market failure, consideration of alternative instruments, cost-benefit analysis, an assessment of the potential for additionality and spillovers, a realistic exit strategy, and a “robust monitoring and evaluation (M&E) system.” Ayyagari et al. Credit Generally positive additionality of credit guarantee schemes that provide (2017) guarantee “provide partial guarantees by covering a share of the default risk of the schemes loan and are used in many developed and developing economies to alleviate financing constraints of SMEs.” Yet they find weak sustainability “especially in countries with weak institutional frameworks.” Cravo and Business “Understanding the mechanisms underlying each intervention is crucial if Piza’s development one is interested in designing SME interventions for different contexts.” services Matching grant schemes associated with modest, significant employment effects. “More rigorous impact evaluations are needed to fill the large knowledge gap in the field.” Ayyagari et al. Business They find insufficient evidence on “what specific types of managerial (2017) development capital are important for SMEs” and on “how to overcome market failures services that are preventing more SMEs from availing these services.” In spite of “indicative evidence that overall business support interventions help improve firm performance and create jobs,” “little is still known about which interventions work best …and why.” Cusolito, Business In a five-country randomized control trial in the Balkans, the authors Dautovic, and development found an increased investment readiness score for “innovative start-ups McKenzie services and small and medium-size enterprises.” (2017) Engelschalk Specialized The authors find “potential benefits in terms of compliance cost and Loeprick MSME tax reductions” that “encourage higher participation rates,” yet poor design (2015) regimes “acts as a disincentive for small businesses to grow to a turnover level above the presumptive regime threshold … and attract larger businesses that look for areas in which to reduce their tax liability.” Note: MSME = micro, small, and medium enterprise. 2.2 To capture lessons since its 2014 SME Evaluation, IEG reviewed all its evaluation products and relevant new Bank Group research in search of new evidence and lessons about Bank Group support to SMEs. From its micro evaluations, IEG tapped the 9 findings and lessons from evaluations of 104 SME support projects that had been completed by Bank Group and evaluated by IEG between FY14 and FY18 with associated commitments of $5.2 billion.9 Almost all of these projects were completed by FY15, so their findings are not expected to reflect changes implemented as a result of IEG 2014, but rather add a base of more recent evaluative evidence that was not included in the earlier work. Lessons from meso and macroevaluations were identified by an index search through all qualifying evaluations FY14 and beyond with substantial coverage of SME (or MSME) issues (Francisco et al. 2016). Figure 2.1. Distribution of Evaluated SME Projects by Region, Country Income Level, and Institution a. World Bank Group Projects Evaluated by IEG 2014– b. World Bank Group Projects Evaluated by IEG 2014– 18, by Region 18, by income level AFR (n=33) Low (n=35) ECA (n=20) LCR (n=16) Lower Middle (n=36) MNA (n=13) EAP (n=10) Upper Middle (n=27) SAR (n=9 OTH (n=3) High (n=2) 0 10 20 30 0 10 20 30 40 WB Lending IFC-IS IFC-AS MIGA WB Lending IFC-IS Note: Panel b excludes four regional projects. AFR = Africa; AS = Advisory Services; EAP = East Asia and Pacific; ECA = Europe and Central Asia; IFC = International Finance Corporation; IS = Investment Services LCR = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia; SME = small and medium enterprise. 2.3 The projects in the evaluated portfolio reflected the global nature of the World Bank, spread across all regions and distributed with a preponderance in low and lower middle-income countries, with fewer projects in upper-middle-income countries (figure 2.1, panel b). However, while World Bank lending and IFC Advisory Services were skewed toward low and lower middle-income countries, a majority of IFC lending and MIGA projects were in upper-middle-income countries. Although most of the evaluated SME activity focused on the financial sector, this concentration was much stronger for IFC and MIGA, whereas World Bank SME lending was spread across sectors (figure 2.2.). 10 Figure 2.2. Sectoral Focus of Small and Medium Enterprise Support Projects World Bank Group Projects Evaluated by IEG 2014–18 Financial Sector (n=67) Manufacturing, Agribusiness, Services (n=22) Multiple (n=8) Telecoms, Media, Technology Venture Inv (n=5) Infrastructure (n=2) 0 20 40 60 WB Lending IFC-IS IFC-AS MIGA Source: Independent Evaluation Group based on portfolio review. Note: The figure is based on 104 projects identified with small and medium enterprise intervention. 2.4 Broken down by number of interventions (where a multicomponent project may involve more than one type of intervention), Bank Group institutions engage differently and use different instruments (figure 2.3). MIGA’s evaluated portfolio, consistent with its mandate, is entirely in the form of guarantees, worth $778.8 million. IFC investments rely heavily on lines of credit, which comprise 88 percent of its evaluated portfolio activities and 81 percent of its commitments ($396.5 million). Its other instruments are guarantees (13 percent of commitments) and funds (6 percent of commitments). IFC’s support to SMEs is augmented by an active Advisory Services program, mostly aimed at financial institutions (to encourage or reshape SME lending) but also oriented toward business development services (BDS) and, to a lesser extent, value chain development. The World Bank has more lending activities in business development, matching grants and investment climate (although this can include work on credit information and collateral laws) than in lines of credit, with the largest share of commitments in investment climate (44 percent or $595 million), lines of credit (33 percent), and business development (5 percent). Although they make up 20 percent of activities, matching grants comprise only 6 percent of commitments, and value chain activities comprise 10 percent of activities and 3 percent of commitments. This division may understate the use of credit lines, as they are sometimes used for value chain and BDS activities. 11 Figure 2.3. Types of Small and Medium Enterprise Support Interventions, by Institution World Bank Group Projects Evaluated by IEG 2014–18 100% 2% 3% 8% 6% 90% 10% 4% AS to Government 80% 8% AS to FI 70% 13% Risk sharing / GCR / 57% Political Guarantee 60% Value Chain 18% 50% 100% Funds 88% 40% Lines of Credit 20% 9% Investment Climate 30% 6% 20% 3% Matching Grants 10% 25% 20% Business Development 0% WB Lending IFC-IS (N=25) IFC-AS (N=35) MIGA (N=7) (N=88) Source: Independent Evaluation Group, based on portfolio review. Note: The figure is based on 155 intervention types identified for the 104 projects. Advisory Services and Analytics is not evaluated by the Independent Evaluation Group and is therefore not included in the analysis. AS = Advisory Services; GCR = Guarantee— Commercial Risk; IFC = International Finance Corporation; IS = Investment Services; WB = World Bank. 2.5 By institution, 75 percent of World Bank lending had positive development results (“above the line”). Regarding IFC investment services, 44 percent had positive development outcomes while only 29 percent of Advisory Services did not. Meanwhile, IFC’s SME credit lines achieved a 50 percent positive rate. Two of its three value chain projects were successful. Regarding MIGA, 57 percent of IEG evaluated guarantees achieved positive results. Regarding leading World Bank lending products, the highest success rate was achieved in lines of credit with 82 percent achieving positive development results. 2.6 In terms of tracking outcomes (and even outputs) of Bank Group–supported SME activities, the record was quite spotty. Collectively, World Bank managed to attribute over 62,000 jobs to its SME-related activities, but IFC rarely measured jobs. IFC was far better at tallying the number of SME loans supported (almost 68,000 by IFC’s count), while the World Bank record here was weak. 12 2.7 Gender was incorporated in half of the projects evaluated to a varying degree (table 2.2). Gender was treated in almost half of World Bank’s SME support projects (47 percent). Yet only about a third of them included activities that support women and have related indicators. Another 14.4 percent of projects included activities without related indicators, recording gender disaggregated data and/or specific gender activity. IFC’s investment operations mainstreamed gender in 44 percent of the evaluated portfolio, but the related indicator targets were achieved in 20 percent of the evaluated operations. IFC’s SME Advisory projects treated gender in 50 percent of the cases. Only one of the evaluated MIGA guarantees operations included women as a target population, but achievement of the gender target wasn’t tracked. Table 2.2. Small and Medium Enterprise Projects that Tracked Gender Indicators World Bank Lending (n = 42) IFC-IS (n = 25) Project Characteristics (no.) (percent) (no.) (percent) Jobs created 8 19 1 4 Jobs created for women 1 2 — — SMEs financed 3 7 8 30 SME microfinance loans — — 7 28 Women borrowers — — 1 4 Microfinance loans for women — — 6 22 Source: Independent Evaluation Group based on portfolio review. Note: — = not available; IFC-IS = International Finance Corporation Investment Services; SME = small and medium enterprise. Box 2.1. Advisory Services and Investment Services Most of the evaluated IFC Advisory Services (AS) projects are linked to an IFC investment services operations (70 percent). However, the success rate of such linked advisory projects was low—29 percent—the same rate as all evaluated AS projects. To better understand the complementarity of AS and IS, it is important to compare the success rate of linked IS projects to those not linked with AS. It is possible that even if AS projects did not achieve their objectives, they increased the likelihood that IS projects achieved theirs. Unfortunately, the number of linked IS operations in the portfolio of evaluated projects (4) was too low to validly make such a comparison. Finally, many advisory projects (77 percent) were linked to other advisory projects, suggesting the potential for continuity and complementarity. 13 Table 2.3. SME Projects that Tracked Gender Indicators World Bank Lending IFC-IS IFC-AS MIGA Total (N = 42) (N = 25) (N = 30) (N = 7) (N = 104) (no.) (%)a (no.) (%)a (no.) (%)a (no.) (%)a (no.) (%)a Gender 22 52 11 44 15 50 1 14 49 Activities 22 52% 11 44% 15 50% 1 14% 49 47.1% Indicator 10 24 11 44 12 40 1 14 34 33.7 Achieved 9 5 3 Not Achieved 1 5 5 No information 1 4 1 Without Indicator 2 5 2 13 4 3.8 Data disaggregated 8 19 1 8 9 8.7 by gender Unintended impact 2 5 2 1.9 on women Source: Independent Evaluation Group, based on portfolio review. Note: The table is based on 104 projects identified with small and medium enterprise interventions. Advisory Services and Analytics is not evaluated by the Independent Evaluation Group and is therefore not included in the analysis. AS = Advisory Services; IFC = International Finance Corporation; IS = Investment Services; MIGA = Multilateral Investment Guarantee Agency. a. Percent of total projects. Performance Factors 2.8 Overall, positive features in SME projects that performed well were also identified as deficiencies in some underperforming SME projects. For example, project technical design, monitoring and evaluation (M&E) framework, and prior analytic work were each prominent among both success factors and problem issues. 2.9 The most common problems identified across the portfolio of 104 projects were the following: M&E framework (31 percent of identified problems); technical design (19 percent of identified problems); project implementation unit (PIU) (16 percent of identified problems); inadequate risk assessment/ mitigants (12); external shocks (10 percent); issues with baseline data or targets (9 percent); insufficient political or institutional analysis (8 percent); and insufficient prior analytic work (8 percent). Yet the distribution of these challenges varied by institution and SME product (figure 2.4.): • Regarding World Bank lending, the most common challenges lay in the adequacy of the M&E framework, the PIU, political and institutional analysis, and technical design; 14 • For IFC investments, the main challenges were adequacy of risk assessment and mitigation, technical design, baseline data or targets, and external shocks; • For IFC advisory, the top challenges were adequacy of the M&E framework, technical design and external shocks; and • For MIGA guarantees, the top challenges included adequacy of risk assessment, reference to prior analytic work, and adequacy of baseline data or targets. 2.10 In IDA countries, projects were more likely to encounter problems with inadequate M&E frameworks and data, the PIU, overly complex design and inadequate political or institutional analysis. Non-IDA countries were more likely to encounter problems with inadequate risk assessment or mitigants, inadequate baseline data or unrealistic targets and external shocks/crises. 2.11 Most lessons of experience were consonant with these findings related to internal Bank Group factors. They include keeping the design simple and easily verifiable (33 percent of the lessons), having a robust results/M&E framework (11 percent), and benefiting from prior analytical work (7 percent). Figure 2.4. Problems Identified (percent of operations) 100% M&E Framework (n=59) 11% 10% 90% 25% Technical Design (n=36) 32% 10% 80% 20% PIU (n=31) 70% 11% 4% Risk Assessment/mitigants 30% 60% (n=23) 19% Baseline data or targets 19% 22% 50% (n=18) 6% External Shock (n=19) 40% 6% 3% 20% 5% 13% 6% Political or institutional 30% 4% analysis (n=16) 11% 10% Prior analytic work (n=15) 11% 13% 20% 2% 5% 5% 2% 10% Complex design (n=14) 5% 10% 8% 11% 3% 3% 3% 10% 2% 4% 5% 0% 1% WB-Lend IFC-IS IFC-AS MIGA Source: Independent Evaluation Group, based on portfolio review. Note: The figure is based on 249 problems identified for the 104 projects. Advisory Services and Analytics is not evaluated by the Independent Evaluation Group and is therefore not included in the analysis. AS = Advisory Services; GCR = Guarantee—Commercial Risk; IFC = International Finance Corporation; IS = Investment Services; WB = World Bank. 15 2.12 Evaluation findings include the following: • Technical design. Positive features of technical design include a simple project design; complementary technical assistance or advisory work; clear and specific development objectives and well elaborated results frameworks (causal chains); and indicators that facilitate attribution of project benefits. For example, the Morocco MSME Development Project featured a simple design and investment was complemented by technical assistance providing training and advice to supported intermediary institutions. Weaker technical designs often suffered from excessive complexity, had vague objectives or underlying logic, were detached from any prior analytic work, had weak targeting to SME beneficiaries, or had inadequate counterpart capacity. Some credit lines and guarantees lacked well-defined covenants specifying the obligations of financial institutions to serve SMEs. • Supervision, PIU. Effective supervision involves a project team diligently and consistently monitoring progress, addressing delays, and actively supervising the project. For example, in Kyrgyz Republic Village Investment Projects I and II (active from FY04–15), IEG found that the project team “provided excellent implementation support, which allowed for minimal interruption of project activities during the two internal conflicts … that occurred during implementation. In addition, the World Bank team took the necessary steps to rectify the structurally deficient infrastructure.” For World Bank projects, a positive feature of the PIU includes capability in key areas such as procurement, financial management, and social inclusion issues. Less successful PIUs can have poorly-defined or excessively broad responsibilities or lack key competencies. • Risk assessment and mitigation. Risk assessments and specific risk mitigation measures were a leading technical challenge affecting all three Bank Group institutions. Strong risk assessment clearly identifies risks and their mitigants and realistically anticipated the time and institutional capacity needed to achieve institutional change. Weaknesses include inadequate analysis of sector and country-specific factors and political and institutional factors in assessing feasibility and the lack of mitigation for sector risks. • Analytic work. Prior analytic work can effectively inform the design of investments and guarantees. The Mexico Rural Finance Development Structural Adjustment Loan (P074655) benefited from prior analytical work done by the Bank of Mexico’s financial sector and the financial sector assessment done by the World Bank and the IMF (PPAR). 16 • Long-term client engagement can strengthen institutions that focus on SMEs. IEG’s evaluation on IFC’s Client Engagement Model (IEG 2017) finds cases where IFC played a critical role in enabling established social enterprises to grow and expand their services into markets –such as microfinance, SMEs lending, integrating farmers into supply chains and power supply.” Through repeat client engagements, IFC was also able to have strong demonstration effects. ACLEDA Bank in Cambodia established new lending services for women and SMEs before expanding its business model and service offerings regionally, with important demonstration effects in Cambodia and beyond. 17 3. Findings on Specific SME Approaches 3.1 Breaking down evaluative evidence by the type of SME intervention sheds light on lessons about specific SME instruments. Outcome rates varied widely by instrument and institution, but each must be interpreted in context to derive relevant lessons. Table 3.1. Project Outcome Ratings of Successful by Intervention Type and Institution World Bank Lending IFC-IS IFC-AS MIGA Total (no.) (%) (no.) (%) (no.) (%) (no.) (%) (no.) (%) Financial Lines of Credit 11 82 22 50 2 50 35 60 Funds 7 71 1 0 8 63 Risk sharing / GCR 3 67 2 0 7 57 12 54 / Guarantee— Political Risk Subtotal—Financial 21 76 25 44 2 50 7 57 55 58 Advisory / TA Business 22 73 7 0 29 55 Development AS to Financial 2 100 20 35 22 41 Institution Matching Grants 18 72 18 72 Investment Climate 16 75 1 0 17 71 Value Chain 9 78 3 67 12 75 AS to government 2 0 2 0 Subtotal – Advisory 67 75 33 27 100 59 / TA Total 88 75 25 44 35 29 7 57 155 60 Source: Independent Evaluation Group based on portfolio review. Note: The figure is based on 155 intervention types identified for the 104 completed project operations. Projects are coded successful if outcome / development outcom e/ development effectiveness is highly satisfactory / highly successful, moderately satisfactory / mostly successful, and satisfactory/successful. GCR = Guarantee-Commercial Risk; TA = technical assistance. 3.2 Lines of credit. Project evaluations show that World Bank SME lines of credit with satisfactory ratings have a clear project objective with an explicitly established SME target group and client ownership. Overall, they suffer from some weaknesses in the M&E framework but have generally exceeded their targets for lending. For example, the Bosnia and Herzegovina SME Access to Finance project (P111780), a credit line of $157 million, had a clear design and result chain, exceeded the number expected for firms financed under the project (338 versus the target of 140), and supported the creation of 1,937 jobs. It benefited from clear project ownership at all levels of 18 government. Yet the results framework lacked details to better understand the sustainability of the outcomes and their attribution to World Bank actions. 3.3 Reaching small enterprises and sustaining such reach can be a substantial challenge for lines of credit to SMEs, especially in rural areas. IEG’s 2016 Industrial Competitiveness and Jobs evaluation found that, while legal policy and institutional support often succeeded in improving the environment for financing rural SMEs and farmers, “efforts to channel funds to farmers and rural enterprises through rural cred it showed a lower success rate, undermined by macroeconomic factors and low take-up of credit lines.” In Kazakhstan, for example, a World Bank credit line for farmers was twice restructured in the early 2000s because the intermediary banks preferred to focus on large borrowers rather than small-scale agricultural and rural lending. In another example in Kyrgyz Republic, the 2008 financial crisis diverted intermediary banks’ focus away from farmers and rural small and medium enterprises, the original target of SME support. 3.4 IFC lines of credit most often suffered from deficiencies in risk assessment and mitigation, project technical design, and the setting of adequate baseline data or targets. Some projects (for example, Patria Credit) had insufficient assessment of the feasibility of the project sponsors’ strategy, assessment of multilateral finance corporations’ capacity of reach loans sector/social group (loans to women entrepreneurs and agribusiness sector) development outcomes and shortcomings regarding the measurement of portfolio at risk. 3.5 A key challenge for IFC’s credit lines has been establishing and enforcing borrower eligibility criteria that direct credit lines to SMEs that are truly constrained in their access to finance. Strict criteria relying primarily on credit-worthiness may lead to facilities being underused or may channel funds to SME firms that already have access to finance on reasonable terms. IEG’s Brazil Country Program Evaluation (IEG 2017), for example, looked at IFC’s effort to reach MSMEs with credit lines through “second-tier banks” as intermediaries. It found that the impact of IFC’s credit lines “is difficult to identify, given how the eligibility criteria for sub-borrowers are defined.” It recommended that IFC (i) enhance its monitoring systems to examine whether SMEs that have relatively less access to long-term credit are reached, and (ii) sharpen the subborrower eligibility criteria and associated legal documents of IFC financial market investments aimed primarily to reach SMEs. 3.6 Defining effective SME eligibility criteria for credit lines is predicated on a good understanding and segmentation of the lending portfolio of participating financial institutions. Many of the commercial institutions supported by IFC serve a heterogenous 19 clientele that include micro and SMEs to a varying degree. IEG’s 2015 Financial Inclusion evaluation finds that 3.7 “In the countries where IFC operates, microloans represent about five to ten percent of the loan portfolios of those banks that IFC supports wi th investments…. The majority of IFC-supported banks (90 percent) have mixed portfolios; the rest of the portfolios are up to 10 times large and go to clients taking out significantly larger loans, including small and medium enterprises (SMEs). ” 3.8 Although the same evaluation finds that mixed portfolios are “not necessarily a bad thing” since SMEs are being served, it argues for better segmentation of targeting of the micro and small and medium-size enterprise market, and for accurate reporting on the reach to the very small retail segment of the microcredit market. 3.9 Guarantees and risk-sharing facilities. Like credit lines, guarantees and risk-sharing facilities are often used to induce financial intermediaries to finance SMEs. Instead of providing a dedicated line of credit, they support financing for SMEs by assuming responsibility for a portion of associated risks. Although very few risk-sharing facilities were evaluated in the period, weak performance of these instruments was related to lack of tailoring of risk identification and mitigation to specific risks after inadequate sector analysis. For example, the guarantee provided by MIGA to Raiffeisen Leasing against the risk of transfer restrictions and expropriation of funds in Serbia did not increase leasing, particularly to SMEs; instead, the loan was used to support the institution in surviving the crisis. The evaluation attributed this mainly to a lack of specific risk mitigation after inadequate analysis of the sector and the client’s business performanc e and weak assessment of expected impacts. 3.10 IFC’s two evaluated risk-sharing facilities targeting SMEs both had an unsuccessful development outcome rating. Challenges included targeting benefits to SMEs and scaling interventions to match both supervision and absorptive capacities of financial intermediaries. For example, the RSF CHUEE IB SME facility (29366) that support SME’s access to energy efficiency and renewable energy financing was underused due to strict eligibility criteria, which made it difficult to identify qualified SMEs. Also, the ANZ Royal RSF (28471) was underused due to a misunderstanding of subborrower eligibility criteria. Generally, the literature indicates that risk-sharing interventions provide financial additionality, but the financial sustainability of their outcomes can be difficult in countries with weak institutional frameworks (Ayyagari et al. 2016). 3.11 The number of evaluated MIGA guarantees is small. Well-performing MIGA guarantees displayed some common characteristics. These included consistency between 20 the project and government's strategies, correct identification of environment and social risks and, in times of crisis, commitment of a parent financial institutions to support and maintain the overall exposure levels of their subsidiaries. 3.12 Advisory services to financial institutions. A majority of evaluated IFC Advisory Services projects focused on providing technical assistance to financial intermediaries (57 percent) for expanding their SME lending. The majority (65 percent) of these were unsuccessful. Identified deficiencies included inadequate M&E frameworks and technical designs, but external shocks (such as changing market conditions, financial crises, and change in the client’s priorities) also affected implementation. In Mauritius, for example, IFC’s support for capacity building in selected banks achieved sustained expansion of lending to SMEs. On the other hand, IFC’s efforts to support the Bank of St. Lucia (IEG 2014) to establish an SME lending practices was poorly sequenced and substantively unsuccessful, because fundamental constraints—such as weak enforceability of collateral, lack of an effective credit information system and poor payment culture of local entrepreneurs —remained unaddressed.10 3.13 BDS. BDS comprise a diverse range of programs: open-ended matching grants for services; dedicated support for training; advice in financial or operational management; worker technical skills; market (and value chain) access; innovation; and others. Understanding the mechanisms underlying each program is crucial to inform the design of SME interventions for different contexts (Cravo and Piza 2016). Although several studies have found certain services helpful, little has yet been established about which interventions work best for SMEs across contexts. Cravo and Piza do find that regional location and firm size matter – for example, BDS interventions are associated with lower performance benefits in Africa and lower employment and labor productivity benefits in Latin America and the Caribbean. They also find that, among SME beneficiaries, “larger firms are associated with larger impacts .” 3.14 Bank Group–supported BDS to SMEs with satisfactory ratings benefited from good design and effective collaboration with external actors, other donors and clients. For example, IEG found that the Mozambique Competitiveness and Private Sector Development project, active from FY09 to FY16, enhanced the competitiveness of SMEs by strengthening business services and providing training to entrepreneurs and workers. The project had a design well aligned with the objectives, followed a clear logical chain and had a coherent M&E framework. The project team engaged with other development partners and the project benefited from the government’s strong commitment. However, the project lacked detailed guidance on how to measure results, which would have helped ensure the collection of better-quality information. 21 3.15 SMEs have needs in multiple areas, hence they may benefit from a coordinated delivery of complimentary services. Yet this is rare. IEG’s Mauritius country case study (IEG 2016) found the World Bank helped bring coherence to diverse SME BDS. Initially, most BDS “were of mixed quality and not very usefu l for meeting contemporary requirements of SMEs. Moreover, effectiveness of these programs was hampered by coordination challenges among the multiple agencies that were implementing the programs.” The World Bank supported the creation of an “an inter- agency strategic committee” to improve coordination and information ex change and to identify gaps to address. This was supported by an integrated M&E framework. 3.16 Matching Grant Schemes and Funds. Matching grants can subsidize firms to use commercial support services (for example, BDS), strengthen enterprise (and entrepreneurial) capabilities, or help firms to innovate or tap new markets (including export markets) in commercializing their products and services. Key features of successful World Bank’s SME matching grants schemes were (i) carefuly consideration of modalities for the management; (ii) accompanying technical assistance, capacity building and institution strengthening; (iii) a good mix of international expertise and local knowledge; (iv) sufficient resourcing of fieldwork and due diligence functions; (v) autonomy of administration; (vi) adaptability to market demand; and (g) assessment of the situation and needs. There is an unresolved debate on whether the selection mechanism for such schemes should be competitive or “first-come, first-served.”11 Earlier IEG work reinforces the importance of flexible design and expeditious processing but also points to the importance of engaging the local private sector. Failures arose not only from the absence of these features but also from political interference and unfavorable macroeconomic conditions. 22 Box 3.1. Matching Grants for Innovation and Entrepreneurship IEG’s Evaluation on Innovation and Entrepreneurship (2013) took a detailed look at matching grant schemes. It found that the matching grant mechanism was effective where: • the selection process was rigorous and followed clear procedures; • project design was flexible to deal with changing circumstances; • processing of claims for reimbursement was expeditious; and • the private sector or private sector associations were involved in administration. Conversely, ineffectiveness was associated with: • A failure to correctly identify target beneficiaries; • Slow and costly implementation and low uptake; • Eligibility criteria that were either too strict or too lax; • Complex processing and reimbursement procedures, yielding firm-level cash flow problems; • Rigid donor budgetary and procurement processes, yielding delays; • Political interference and associated management rotations; • Unfavorable macroeconomic conditions that increased investment uncertainties. 3.17 Most World Bank funds projects operated similarly as grants to enterprises for diverse purposes, including providing training, financing investments, increasing employment and supporting research and development. Funds encountered some difficulties with adequacy of implementation capacity, M&E or technical design. For example, the YF Innovation Serbia Project, active from FY11–16, exceeded its targets for the development of technologies, products and processes, but experienced operational challenges including slower than expected disbursements and high staff turnover. 3.18 IFC funds that support SMEs are different in that they invest equity in small businesses—instead of providing grants or credit—to foster the creation of new industries and promote employment growth. Although generalization is not possible, the one evaluated IFC fund in scope fell short in outcomes including the number of enterprises financed and portfolio performance. The project failed to collect and monitor important operational information and lacked a cost-effective operational model and an appropriate strategy for expansion. A separate IEG review of IFC funds (IFC Investment Funds Sector Highlights FY12–17) finds that successful funds benefited from well- qualified and experienced fund managers and adhered to their original strategies. Unsuccessful SME funds suffered from a fund manager without a strong background in SME investment and/or lack of hands-on IFC supervision. 3.19 Value chain / link operations. The Bank Group’s growing attention to global value chains has heightened awareness of the opportunities that both global and domestic value chains can offer to SMEs for links to larger operators and markets. Agribusiness and tourism are two sectors where these links have been prominent. IEG’s 23 Rural Nonfarm evaluation found that IFC’s investments in food and agribusiness can have important benefits when strengthening links is an explicit objective. These investments in local companies can create strong up- and/or downstream links to the rural economy. For example, an investment in an Indonesian food company, besides directly employing thousands of people, was associated with important upstream links, new demand for local farm inputs (shallots) from 7,000 local farmers, and downstream links to 400,000 retailers selling the product, many of them SMEs. Often, however, the attention to SME links is lacking. In tourism sector activities, IEG found few IFC projects that sought to enhance supply links with SMEs in the rural economy. A positive example, however, was the case of Sri Lanka, where IFC’s Advisory Services linked 100 SMEs to large hotels, creating jobs along the value chain that reached into the rural economy, including restaurants, guest houses and diver operators. 3.20 Evaluated Bank value chain operations in agribusiness and tourism generally exceeded their operational targets. Seven of nine evaluated projects had positive overall development outcome ratings. However, the projects were often faulted for overly ambitious design relative to government capacity. For example, the Guinea MSME Development Project, active from FY13–18, exceeded targeted outcomes for the development of MSME support in selected value chains. The project had to be restructured to adapt to limited counterpart capacity. This was pursued both by reducing the complexity and number of activities and by an additional focus on strengthening the counterpart agency’s capacity (for example, enhancing procurement processes, staffing and training). 3.21 The 2016 rural nonfarm evaluation found that World Bank value chain projects in agriculture and tourism did not effectively define or measure their intended competitiveness effects on SMEs across the value chains they were supporting (World Bank 2016). This impeded the ability to track the backward and forward links among the farm sector, the nonfarm sector, and impacts on poverty. 3.22 Over time, IEG found that IFC has been taking a more integrated approach to value chain work, engaging along several points of the supply chain. IEG found that the food and beverage sector interventions generally offered especially strong links to the rural economy and rural employment. This has included support for both small farmers and rural SMEs. 3.23 An important finding was that Bank Group projects rarely pay attention to distributional effects along the supply chain, including their impact on SMEs. For example, evaluations of “productive partnerships” in Latin America found a complete lack of available data [to] elucidate distribution of revenues among producers, middlemen, aggregators, and processors. At the same time, experience indicated that 24 supporting corporations with substantial market power could disadvantage small producers in the value chain. In project preparation, systematic attention was not paid to identifying risks for development outcomes associated with market structure. 3.24 Investment climate reform. In the evaluated SME portfolio, investment climate reforms were the largest single area of World Bank engagement in terms of commitments (44 percent). Although investment climate reforms benefit the entire economies, SMEs may disproportionately benefit from such interventions in business areas where firms face more constraints. 3.25 For example, research indicates a significant positive relationship between the introduction of credit bureaus and MSME access to finance, with associated gains in employment (Ayyagari et al. 2016). This same relationship does not hold for large firms. The Bank Group supports both public and private credit bureaus. A broad review of the literature finds that both public and private credit information lead to an improvement in the efficiency of credit allocation decisions and loan performance (Ayyagari et al. 2017). However, the authors identify evidence that public registries are less effective, and a need for more research about which aspects of their design contribute most to improving the allocation of credit. 3.26 Bank Group investment climate interventions with satisfactory ratings in general benefit from strong client government commitment, prior analytical work and a design that is appropriate for the project context. Successful projects also engage external actors and donors. 3.27 Specialized tax regimes aim at facilitating compliance by micro and small enterprises by reducing the cost of compliance (Engelschalk and Loeprick 2015). Research on their effectiveness suggest potential benefits in terms of compliance cost reductions that encourage higher participation rates. But it also finds evidence of poor design that acts as a disincentive for small businesses to grow to a turnover level above the presumptive regime threshold (barrier for migration upwards out of the system) and attracts larger businesses that look for areas to reduce their tax liability. The findings suggest that a proper design of such tax regimes is vital to reduce distortions, as is proper enforcement. 3.28 Continuous, coordinated, multiinstrument engagement. Given SME’s multifaceted needs and constraints, the biggest impact is achieved over time with continuous, coordinated, multiinstrument engagements drawing from the full Bank Group tool kit. IEG found this exemplified in the South Africa country program, where the Bank Group had a “strong engagement in supporting MSME reforms.” Points of strength included the following: 25 • Consideration of country context, including the role of SMEs and industrial policies; • Consistent benchmarking through enterprise surveys for both SMEs and microenterprises; • Effective issue coverage including the incentives framework (where analytic focus on tax and investment incentives contributed to a tailored approach).12 3.29 IEG found that the Country Partnership Strategy for South Africa and associated results framework provided strategic direction to Advisory Services work and enabled a more programmatic and longer-term approach to achieve higher development impact than a more ad hoc set of activities would. It found that the programmatic approach was able to both respond to client requests where there was alignment —for example, in lowering tax and regulatory compliance costs to MSMEs —and to “generate ownership” for reforms that had not been requested, such as a modern, secured transactions regime. The approach was facilitated by local partnerships that supported the success of the project. The World Bank and IFC coordinated well in providing sequenced diagnostic and advisory support to private sector development reforms in support of MSMEs. After the World Bank carried out “core analytic work,” a series of World Bank and IFC knowledge activities on a variety of issues were undertaken, where “the distribution of follow-up work was based on the respective institutions’ areas of expertise and experience.” IFC technical assistance also followed, but in the area of taxes, IFC used “experienced Bank staff to manage the projects.” 3.30 China also presents a positive example of a programmatic approach and coordination on MSME finance, with the World Bank producing a Financial Sector Advisory Program (FSAP) assessment and other knowledge products, paving the way for a World Bank loan in support of MSMEs, while IFC followed a programmatic series of advisory activities diagnosing and addressing the legal and institutional framework for MSME finance. This positive example is in contrast with other less successful experiences, for example, in the Russian Federation a few years ago, where IFC’s and Bank’s work in support of MSME financing had “less coherence and coordination.”13 3.31 IEG found in its review of the Georgia’s Country Partnership Strategy Completion Report (IEG 2014) good coordination among World Bank, IFC, and MIGA, particularly, a coherent multicomponent engagement of IFC through investment climate related advisory services, facilitation of access to finance for SMEs through the banks, and investments in the Agriculture sector. However, it did find a “minor weakness” in two contradictory strategic objectives for the financial sector: the macro-pressures for a more conservative approach to lending by the banking system were at odds with 26 sectoral outcomes pressing for more lending to SMEs. IEG suggested a sequenced rather than simultaneous approach to the two objectives would have been more effective. 3.32 The benefits of a continuous, multifaceted engagement were also seen in the case of North Macedonia in IEG’s Industrial Competitiveness and Jobs Evaluation (2016). The World Bank supported the country’s manufacturing competitiveness with exceptionally well-sequenced and well-executed tools and operations, starting with analytical work, followed by policy dialogue, integration of various instruments into a development policy loan platform, and accompanying technical assistance. Key elements of the nation’s industrial policy for the manufacturing sector were emphasized by the Bank Group’s Country Partnership Strategy, “such as accessing new markets, developing high value-added products, removing trade barriers, supporting small and medium enterprises and entrepreneurs, and introducing new technologies, tech zones, clusters, [and] foreign direct investment.” A World Bank–IFC Collaboration Group, a feature of the coordinated approach, identified priority areas for collaboration between the two institutions, including trade logistics and job creation and employment. 3.33 Conversely, in Pacific Island Countries (IEG 2016), IEG found too little coordination between World Bank and IFC in support of MSMEs. In those countries, a joint World Bank–IFC approach was needed to support private sector development in general, and specifically micro, small, and medium enterprises (particularly in Tonga and Samoa). This could have started with analytic work to identify priority areas requiring support. 27 4. Should the World Bank Group Distinguish SMEs As a Firm Size Category to Support? 4.1 The following reasons are typically given to justify Bank Group support to SMEs: • Differential constraints faced by SMEs relative to larger firms; • SME’s presence and importance in the economy as engines for jobs, economic growth and opportunity; and • SMEs as an important source of innovation and dynamism in the economy. 4.2 We will briefly examine each of these reasons. 4.3 Different constraints. The fixed costs of regulatory compliance and financing are disadvantageous to smaller firms, a finding well established in the literature for decades. For example, Levy (1993) found regulatory compliance costs for small firms “disproportionately large for the smallest enterprises—in part because its fixed-cost elements are invariant with firm size, and in part because large, but not small, enterprises can hire managers to handle dealings with government officials. Even as a fixed cost, the bureaucratic burden can inhibit the expansion of operating enterprises that are too small to hire a manager.” These considerations are supported empirically, for example, by firm survey evidence that associates financial access constraints with firm size (World Bank Group et al. 2017). 4.4 Economic impact. As a 2017 Bank Group publication makes clear, the Bank Group justifies SME support also by their large presence in economies, their contributions to innovation, employment growth and inclusion: “Small and medium enterprises (SMEs) are the economic backbone of virtually every economy in the world. SMEs represent more than 95 percent of registered firms worldwide, account for more than 50 percent of jobs, and contribute more than 35 percent of Gross Domestic Product (GDP) in many emerging markets. Moreover, the contribution of SMEs to GDP actually increases as economies develop – with SMEs in the developed world contributing well over 50 percent of GDP. SMEs generate most of the new jobs that are created; they help diversify a country’s economic base; they promote innovation; they help deliver goods and services to the bottom of the social pyramid; and they can be a powerful force for integrating women and youth into the economic mainstream” (Alibhai et al. 2017). 4.5 The above arguments are not always rooted in firm evidence. Recent research raises questions about the use of size as a primary basis for enterprise support. Given that the stated objective of Bank Group firm support is to promote economic growth and 28 jobs, recent Bank Group research suggests there may be stronger enterprise criteria to use, including firm age, productivity, and fast growth experience: 4.6 Enterprise Age. A recent World Bank literature review on SMEs, Age and Jobs (Aga et al. 2015) observes that “firm age is perhaps more important than size to explain net job growth.” Although the study finds that SMEs create more jobs when the analysis does not control for age or exit, further analysis sheds doubt on whether SMEs really do grow faster than large enterprises.14 “Studies on developed economies increasingly show that firm size is less important as an explanatory variable; instead, employment creation is driven by a small set of young and fast-growing firms, which indicate that firm age and rate of growth are more important than size.” 4.7 Fast Growth. Consistent with the finding above, an ever-expanding literature on fast-growing firms (sometimes called “gazelles”) focuses more on firm dynamics than on static indicators such as size. The 2019 World Bank flagship report on High-Growth Firms (Goswami et al. 2018) examines high-growth firms in 11 countries—Brazil, Côte d’Ivoire, Ethiopia, Hungary, India, Indonesia, Mexico, South Africa, Thailand, Tunisia, and Turkey—using longitudinal data sets maintained by national statistical agencies or revenue authorities. As with the term “SME,” definitions matter: the authors point to multiple definitions of a high-growth firm based on a minimum rate and duration of growth and/or a relative definition based on the top percentile of firms in the distribution. Each definition would yield a set of “high -growth” firms. Depending on the definition used, across the countries in the study, high-growth firms account for between 1.4 percent and 34.1 percent of the relevant firm population. The analysis shows that high-growth firms contribute disproportionately to employment growth, accounting for between 6.0 and 15.2 percent of net job creation in the six countries for which sufficient data are available. No matter how they are defined, high growth is a temporary characteristic. Firms go through a “high -growth episode,” but these periods tend to be short-lived and typically are not repeated. High-growth episodes are more likely to occur during the early years of operation, but such episodes are not limited to young firms. 4.8 The 2019 high-growth firms flagship report concludes that targeting firms with the potential for high-growth episodes “may be neither feasible nor advisable.” Instead, citing multiple studies, the authors argue for policies to increase the likelihood of high- growth episodes across firms. Such policies woul d address “factors such as innovation, agglomeration and network economies, managerial capabilities and worker skills, global links, and financial development, which contribute significantly to increasing the probability of a high-growth episode.” 29 4.9 Productivity Growth. Another World Bank Equitable Growth, Finance and Institutions flagship report examined enterprise level productivity (Cusolita et al. 2018). Although it cites a few studies on SMEs, the study’s lack of focus on firm size is noteworthy. Starting from the statement that “growth of productivity…is the main driver of the development process,” it seeks to explain why productivity growth has apparently waned. It employs manufacturing production firm-level data for a variety of developing economies to offer “new guidance for productivity analysis.” It presents a framework with three major drivers of productivity: upgrading firms (within-firm productivity gains), improving allocative efficiency of production across firms (between- firm productivity gains) and improving firm entry and exit (selection). Although some of its findings are methodological, it also emphasizes the need to improve policy shaping the enterprise operating environment and human capital and firm capabilities. It cites findings that economic distortions limit firm growth, potentially increasing the predominance of SMEs in some economies by constraining more productive firms from growing larger. It endorses competition policy and the reduction of distortions to work on productivity through the reallocation channel, including by opening markets to international trade, exposing state-owned industries to competition, and reducing their ability to prevent the emergence of competitors. Further implications concern human capital policy, policies to encourage value creation beyond efficiency, encouraging risk taking and experimentation, and redesign government productivity. Not one of its recommendations to increase productivity involves size-based policies or criteria, but all seek to create conditions where all firms are encouraged to either improve productivity or exit. 30 5. Lessons on SME Support from Other Multilateral Development Banks 5.1 Since IEG’s 2014 Evaluation, several other MDBs have carried out evaluations of their SME support, producing lessons that have some resonance for the Bank Group. While each evaluation finds at least some benefits either for participating SMEs or participating financial intermediary institutions, they point to substantial shortcomings of the SME support activities of kindred institutions. Each of the MDBs—African Development Bank (AfDB), Asian Development Bank (ADB), Inter-American Development Bank (IADB), and European Bank for Reconstruction and Development (EBRD)—provide substantial SME financing, much of it in the form of credit lines through financial intermediaries aimed at supporting SMEs: • AfDB reviewed 46 investment operations, 16 technical assistance grants, mostly complementing lines of credit; and eight institutional support projects from 2006 through 2013 reflecting a commitment value of close to $1 billion; • ADB reviewed 182 SME support operations from 2005–17 with allocations of $5.3 billion, including 128 focused on access to finance, 70 supporting business environment and business services measures, 16 value chain operations, as well as support for women in business; • IADB focused on credit for on-lending to SMEs totaling in value $7.85 billion through financial intermediaries (FIs) from 2005 to 2014 and, separately, $219 million in direct lending to SMEs; • EBRD reviewed a sample of its credit lines, described as “a mainstay of the [World] Bank's operations, reaching up to 20 per cent of total annual business volume” as “a means to develop specialized MSME lending capacity, a conduit of financial resources and a core business line.” 5.2 Although the modes of operating and programmatic details varied, the evaluations raised some important and common challenges summarized in table 5.1. 31 Table 5.1. Challenges Raised in MDB Evaluations of their SME Support Programs Strategic/ Conceptual Eligibility and Tracking/ Framework Defining Targeting to Monitoring of Lacking SMEs SMEs Outcomes Additionality AfDB (2015) ✓ ✓ ✓ ✓ ✓ ADB (2018) ✓ ✓ ✓ ✓ ✓ IADB (2014, ✓ ✓ ✓ ✓ ✓ 2016, 2017) EBRD (2018) ✓ ✓ ✓ Source: Independent Evaluation Group review of referenced evaluations. Note: MDB = multilateral development bank; SMEs = small and medium enterprises. 5.3 Strategic or conceptual frameworks. The ADB evaluation found that “ADB lacks an explicit policy, strategy, or operational plan for its SME operations” and that the rationale behind the prioritization of areas of ADB support to SMEs was not clear. It recommends stronger M&E and monitoring of results. The AfDB evaluation highlighted the challenge of very limited information on outcomes, such as employment, and serious problems with data collection by clients. It further pointed to lack of information on sustainability. It recommended improving the collection of information on project achievements and establishing a results reporting system for tracking, monitoring, and reporting development results. IADB finds that FIs are “unable to track the relevant SME portfolio” according to their varying definitions of SME. It found monitoring of FI’s “unreliable.” In its direct SME lending, it found that “most operations aimed to increase sales, production, and employment, but operations rarely included specific indicators to measure progress toward these objectives.” EBRD finds that “ despite the importance and ubiquity of credit lines in [World] Bank operations and strategic commitments the [World] Bank lacks a clear definition of what they are; there is no formal strategic framework setting out their role in contrast to other types of engagement with financial intermediaries.” 5.4 Definition of SMEs. AfDB’s evaluation found projects had no harmonized definition of SME and recommended to adopt one. IADB concluded that client FIs “have their own definitions [of SMEs], and their criteria to define SME size are not homogenous.” Furthermore, the means of classification varied: unlike the Bank Group, only 10 percent of FIs defined SME by number of employees, while 35 percent classified them by revenue, 25 percent by loan amount, and 16 percent by assets. 5.5 Eligibility and targeting of SMEs. AfDB’s evaluation highlighted their projects’ loose eligibility requirements that were weakly targeted to SMEs, although there had been improvements in recent years. A finding was that only a minority of SME financing went to SMEs. It recommends strengthening eligibility conditions to ensure that SMEs 32 are effectively reached. ADB identified a need to “specify the parameters for ADB’s support for SMEs.” It found ADB invested disproportionately more resources in countries at higher levels of development than in countries where SMEs face even more constraints. IADB found the targeting of its intermediary lending was constrained by varying definitions, weak tracking and lack of monitoring. Separately, it discovered that its direct support to SMEs was challenged: “70% of clients in regular SME operations were owned by a larger firm or groups and 18% were newly-established Special Purpose Vehicles” making it “questionable whether such clients should be classified as SMEs.” 5.6 Tracking or monitoring outcomes. ADB evaluation concluded that SME operations often had poor design and monitoring frameworks and lacked credible performance data for timely progress monitoring and midcourse adjustment to improve development results. IADB found that beneficiary lists used for monitoring are an “unreliable mechanism” to monitor the FI’s relevant portfolio. It identified a lack of tracking of “SMEs’ performance on revenues, jobs, and exports” although it was able to construct a single case study of some of these outcomes in Brazil. EBRD concluded that there was a need to “monitor and report their results in an effective manner.” It recommended that it should “review a sample of sub-borrowers to report on outcomes at that level.” AfDB found a lack of information on outcomes and sustainability. 5.7 Additionality. AfDB concluded that few FIs expanded their SME portfolios, and that technical assistance did not appreciably influence project results. IADB found that “the fungibility of money makes it difficult, if not impossible, to attribute the funding of specific beneficiaries” to its interventions. It fails to find evidence that FIs are not financing companies that they “may have financed in any event.” IADB discovered that “almost all” (84 percent) of the SME beneficiaries of its direct financing “had access to credit before the IIC, mostly from commercial banks” and mostly on similar terms. It recommended, “Do not continue providing direct loans to SMEs.” ADB found “no covenants that required [financial intermediaries] growing the relevant portfolio by a minimum amount.” 33 6. Conclusions 6.1 As the Bank Group looks to the future and considers how to best support private sector growth, employment and inclusion, it must confront important challenges: (i) how to build from the effective features and lessons of experience of its past support to SME; and (ii) whether it should continue to focus on small and medium size as a criterion for its enterprise support activities. This report has marshalled research and evaluative evidence to assist in that endeavor. IEG looks forward to additional learning and engagement with Bank Group management that can contribute to the SME support agenda. 6.2 Key findings of this Synthesis Note include the following: 6.3 Defining SMEs. The problem of inconsistent definition of SMEs limits what we know about supporting them from evaluation and research and merits a cautious approach to interpreting and generating evidence on what works. 6.4 SME credit lines. There remains a lack of evidence that lines of credit, the Bank Group’s leading instrument to support SMEs, is effective and beneficial. Similarly, evidence is limited on the benefits to SMEs of most value chain interventions. Addressing the lack of data and M&E collection (especially on beneficiaries) would generate missing evidence on key approaches to SME support. This could help bring practice in line with evidence of what works, why and for whom. 6.5 Business development. There is important evidence of benefits from well- designed BDS programs. At the same time, there remains a limited understanding of what mechanisms strengthen SME management and entrepreneurial capabilities to encourage growth in productivity and employment, suggesting an opportunity for good research. 6.6 Lessons of experience. Several IFC approaches to support SMEs, such as credit lines and risk-sharing facilities, have faced challenges in their effectiveness. These challenges may be assisted by learning from the attributes and context of successful projects and products. 6.7 Sustained engagement. IEG evaluations provide evidence on the important benefits of continuous, coordinated, multiinstrument engagement informed by analytic work and on the effectiveness of combining investment and advisory interventions. 6.8 Sustainability and monitoring. Sustainability is a key challenge to several SME support instruments. Learning would require robust and longer-term approaches to M&E. 34 6.9 Firm size criteria. There is growing evidence of the limitations of size as a criterion for allocating support to enterprises, while competing criteria such as firm age, fast growth and productivity are drawing important attention. At the same time, there is strong evidence of the heterogeneity of SMEs, which vary across many dimensions including size, sector, location and formality but also across different characteristics of owners, managers, and workers. 6.10 MDB challenges. While the modes of operating and programmatic details varied, evaluations by kindred MDBs raise some important and common challenges relating to their SME support activities. These include a lack of a strategic and conceptual framework, inconsistent or missing definition, eligibility and targeting to SMEs, tracking and monitoring of outcomes, and additionality of their SME support. 35 Bibliography African Development Bank. 2015. Evaluation of Bank Assistance to Small and Medium Enterprises (2006–2013) Summary Report. Abidjan, Côte d'Ivoire: African Development Bank. http://www.economistiassociati.com/files/IDEV%20SME%20Report_WEB.pdf. Aga, Gemechu, David C. Francis, and Jorge Rodriguez Meza. 2015. “SMEs, Age, and Jobs: A Review of the Literature, Metrics, and Evidence. ” Policy Research Working Paper 7493, World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/23455. Alibhai, Aly Salman, Simon Bell, and M. Gillette Conner. 2017. What's Happening in the Missing Middle? Lessons from Financing SMES. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/707491490878394680/Whats-happening-in- the-missing-middle-lessons-from-financing-SMEs. Anderson, Stephen, Rajesh Chandy, and Bilal Zia. 2016. “Pathways to Profits: Identifying Separate Channels of Small Firm Growth through Business Training Policy.” Policy Research Working Paper 7774, World Bank, Washington, DC. Asian Development Bank. 2018. Support for Small and Medium-Sized Enterprises, 2005–2017: Business Environment, Access to Finance, Value Chains, and Women in Business. Manila, the Philippines: Asian Development Bank. https://www.adb.org/sites/default/files/evaluation-document/346336/files/support-smes- 2005-2017.pdf. Ayyagari, Meghana, Asli Demirguc-Kunt, and Vojislav Maksimovic. 2017. “SME Finance.” World Bank Policy Research Working Paper WPS 8241, World Bank, Washington, DC. http://documents.worldbank.org/curated/en/860711510585220714/SME-finance Ayyagari, Meghana, Pedro Francisco Juarros, Martinez Peria, Maria Soledad, and Sandeep Singh. 2016. “Access to Finance and Job Growth: Firm-Level Evidence across Developing Countries.” Policy Research Working Paper WPS 7604, World Bank, Washington, DC. Bardasi, Elena, Marine Gassier, Markus Goldstein, and Alaka Holla. 2017. “The Profits of Wisdom: The Impacts of a Business Support Program in Tanzania.” Policy Research Working Paper 8279, World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/29070. Benjamin, Nancy with Kathleen Beegle, Francesca Recanatini, and Massimiliano Santiniet. 2014. “Informal Economy and the World Bank.” Policy Research Working Paper 6888, World Bank, Washington, DC. https://openknowledge.worldbank.org/bitstream/handle/10986/18799/WPS6888.pdf?sequ ence=1&isAllowed=y. Bruhn, Miriam, Dean Karlan, and Antoinette Schoar. 2013. “The Impact of Consulting Services on Small and Medium Enterprises: Evidence from a Randomized Trial in Mexico.” Policy Research Working Paper WPS 6508, Impact Evaluation Series IE 95, World Bank, Washington, DC. 36 Bibliography http://documents.worldbank.org/curated/en/516471468278741470/The-impact-of- consulting-services-on-small-and-medium-enterprises-evidence-from-a-randomized- trial-in-Mexico. Charlier, Florence M., Stephen Hutton, Basil G. Kavalsky, and Swizen Rubbani. 2016. Cluster Country Program Evaluation on Small States: Pacific Island Countries Program Evaluation (FY05–15), Vol. 1. Washington, DC: World Bank. Cirera, Xavier, and William F. Maloney. 2017. The Innovation Paradox: Developing-Country Capabilities and the Unrealized Promise of Technological Catch-Up. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/28341. Cravo, Tulio A., and Caio Piza. 2016. The Impact of Business Support Services for Small and Medium Enterprises on Firm Performance in Low and Middle-Income Countries A Meta-Analysis (World Bank Policy Research Working Paper 7664. https://openknowledge.worldbank.org/bitstream/handle/10986/24501/The0impact0.of00e s000a0meta0analysis.pdf?sequence=1&isAllowed=y Cusolito, Ana Paula, and William F. Maloney. 2018. Productivity Revisited: Shifting Paradigms in Analysis and Policy. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/3058. Divanbeigi, Raian, and Rita Ramalho. 2015. “Business Regulations and Growth.” Policy Research Working Paper WPS 7299, World Bank, Washington, DC. European Bank for Reconstruction and Development. 2018. Special Study: Credit Lines - Lending through Financial Intermediaries. London: European Bank for Reconstruction and Development. https://www.ebrd.com/cs/Satellite?c=Content&cid=1395278819058&pagename=EBRD%2F Content%2FDownloadDocument Engelschalk, Michael, and Jan Loeprick. 2015. “MSME Taxation in Transition Economies: Country Experience on the Costs and Benefits of Introducing Special Tax Regimes.” Policy Research Working Paper WPS 7449, World Bank, Washington, DC. Goswami, Arti Grover, Denis Medvedev, and Ellen Olafsen. 2018. High-Growth Firms: Facts, Fiction, and Policy Options for Emerging Economies. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/391161542221639630/High-Growth-Firms- Facts-Fiction-and-Policy-Options-for-Emerging-Economies. Hristova, Diana, and Antoine Coste. 2016. How to Make Grants a Better Match for Private Sector Development. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/693731491973004765/How-to-make-grants- a-better-match-for-private-sector-development. Inter-American Development Bank. 2014. A Comparative Analysis of IDB Approaches Supporting SMEs: Assessing Results in the Brazilian Manufacturing Sector. Washington, DC: Inter- American Development Bank. https://webimages.iadb.org/publications/english/document/A-Comparative-Analysis-of- IDB-Approaches-Supporting-SMEs-Assessing-Results-in-the-Brazilian-Manufacturing- Sector.pdf 37 Bibliography ———. 2016. Evaluation of IDB Group Work through Financial Intermediaries: SME Finance. Washington, DC: Inter-American Development Bank. https://webimages.iadb.org/publications/english/document/Evaluation-of-IDB-Group- Work-through-Financial-Intermediaries-SME-Finance-Background-Report.pdf ———. 2017. Evaluation of Direct Support to SMEs by the IIC. Washington, DC: Inter-American Development Bank. https://webimages.iadb.org/publications/english/document/Evaluation-of-Direct- Support-to-SMEs-by-the-IIC.pdf. International Finance Corporation. 2017. MSME FINANCE GAP: Assessment of the Shortfalls and Opportunities in Financing Micro, Small and Medium Enterprises in Emerging Markets. Washington, DC: International Finance Corporation. https://www.smefinanceforum.org/sites/default/files/Data%20Sites%20downloads/MSM E%20Report.pdf. McKenzie, David John, Nabila Assaf, and Ana Cusolito. 2015. The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen. Policy Research Working Paper 7462, World Bank, Washington, DC. https://www.ifc.org/wps/wcm/connect/industry_ext_content/ifc_external_corporate_site/ financial+institutions/priorities/ifcs+definitions+of+targeted+sectors World Bank. 2014a. Brazil—Country Program Evaluation, FY2004–11: Evaluation of the World Bank Group Program. Independent Evaluation Group. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/257491468239377080/pdf/Brazil-Country- program-evaluation-FY2004–11-evaluation-of-the-World-Bank-Group-program.pdf. ———. 2014b. “Georgia—Country Partnership Strategy Completion Report Review for FY2010– 13.” Country Partnership Strategy Completion Report Review 87337, Independent Evaluation Group, World Bank, Washington, DC. ———. 2015a. Financial Inclusion—A Foothold on the Ladder toward Prosperity? Independent Evaluation Group. Washington, DC: World Bank. ———. 2015b. World Bank Group Support to Industrial Competitiveness and Its Implications for Jobs. Independent Evaluation Group. Washington, DC: World Bank Group. https://hubs.worldbank.org/docs/imagebank/pages/docprofile.aspx?nodeid=24649966. ———. 2016a. “Cluster Country Program Evaluation on Small States: Mauritius Country Case Study (FY07–15): Enhancing Competitiveness and Private Sector Development. ” Working Paper, Independent Evaluation Group, World Bank, Washington, DC. ———. 2016b. Cluster Country Program Evaluation on Small States (Vol. 1): Regional Program Evaluation of the Organisation of Eastern Caribbean States - Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines. Independent Evaluation Group. Washington, DC: World Bank. ———. 2016c. Growing the Rural Nonfarm Economy to Alleviate Poverty: An Evaluation of the Contribution of the World Bank Group. Independent Evaluation Group. Washington, DC. World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/RuralNonFarm.pdf. 38 Bibliography ———. 2016d. Knowledge-Based Country Programs: An Evaluation of World Bank Group Experience. Independent Evaluation Group. Washington, DC. World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/kbcp_eval.pdf. ———. 2016e. SME Action Plan. Washington, DC: World Bank Group. ———. 2017. The International Finance Corporation’s Approach to Engaging Clients for Increased Development Impact. Independent Evaluation Group. Washington, DC: World Bank Group. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/IFCClientEngage ment.pdf. World Bank Group. “Enterprise Surveys.” https://www.enterprisesurveys.org/methodology 39 1 “In addition to easier business entry procedures, informal business could in principle benefit from a more secured and predictable taxation system, in particular for SMEs.” (Benjamin 2014). 2 The framework for IFC’s engagement was presented to the Board in an IFC Deep Dive (see figure). It is not clear what are the criteria for levels of sector maturity or to what extent this constitutes World Bank Group strategy. 3 Additional questions of current interest to operational teams include the following: • How best can the growth of high productivity SMEs be encouraged? • What is the effectiveness of a reported ongoing shift in the portfolio from credit lines to entrepreneurship support and capital market-related interventions? • How effective are links programs in encouraging SME upgrading, growth, and job creation? • What is the impact of the IFC- or World Bank–hosted programs of support to women- owned SMEs including the Women Entrepreneurs Finance Initiative (We-Fi) and the Women Entrepreneurs Opportunity Facility? and 40 • How effective is the Bank Group approach to the growing role of digital technologies (including digital and mobile banking and the use of artificial intelligence and big data) in its support for SMEs? • How effective are a host of other Bank Group initiatives and new or ongoing (as yet unevaluated) approaches including the Competitiveness Policy Evaluation Lab (ComPEL), IFC’s TechEmerge, IFC’s small loan guarantee program and the Global SME Finance Facility (GSMEF)? These topics could not be treated in this report given that they are more recent developments for which World Bank research and IEG evaluative evidence has not yet been realized or would require new portfolio analysis going beyond the scope of a synthesis report. They are noted as important potential directions of future research and evidence generation. 4For example, a Bank Group database of national standards for defining SMEs suggests that the upper limit for employment distinguishing SMEs from large enterprises varies from a low of 19 employees to a high of 499. SME Finance Forum. 2019 MSME Economic Indicators Database https://www.smefinanceforum.org/sites/default/files/2019%20MSME-EI%20Database.xlsx 5McKenzie, David John and Assaf, Nabila and Cusolito, Ana (2015) The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen. World Bank Policy Research Working Paper No. 7462. https://www.ifc.org/wps/wcm/connect/industry_ext_content/ifc_external_corporate_site/financial +institutions/priorities/ifcs+definitions+of+targeted+sector 6 https://www.enterprisesurveys.org/methodology 7When the threshold is raised to 250 employees, the authors found there is no statistically significant difference between SMEs and large establishments. Finally, when the regression results are weighted by each establishment’s employment share, the d ifference between SMEs and large establishments is found to be insignificant even when the threshold is set at the lower bound of 100 employees. 8Due to the lag between project completion and project evaluation, projects completed after FY15 could not be reviewed. To identify eligible projects, IEG performed a keyword search to identify SME-related projects evaluated on or after FY14, following the Concept Note’s criteria. The search was based on text-based data retrieved from IEG’s Data Mart on project titles, components and objectives. The search yielded 95 World Bank potentially relevant SME projects. In addition, the Evaluation’s Concept Note had already identified 19 World Bank projects, of which 7 overlapped with the results of the keyword search. The net total was 107 potentially SME- relevant World Bank projects. Projects from IFC and MIGA were identified in the Concept Note shared with IFC and MIGA (after eliminating projects that were included in the original evaluation). They totaled 84 projects (33 IFC-IS, 34 IFC-AS and 17 MIGA), for a grand total of 191 potentially relevant SME projects. All project documents were subjected to manual review to assure they were indeed SME support projects. After eliminating false positives, 114 evaluated projects were validated as qualifying for detailed coding and analysis. The micro evaluations 41 (XPSRs/Evnotes/ICRRs/PCRs/PERs/PPARs) were then coded. If a document was subject to both an ICRR and a PPAR, the PPAR ratings and lessons were used. Note that World Bank ASA is not subject to IEG micro evaluation, and therefore could not be treated in the synthesis note at the project level. 9Some false positives, identified at the concept note stage, were dropped. The following were the correctly identified evaluations: 2014. Brazil - Country Program Evaluation, FY2004–11: Evaluation of the World Bank Group Program. 2015. Financial Inclusion - A Foothold on the Ladder toward Prosperity? 2015. World Bank Group Support to Industrial Competitiveness and Its implications for Jobs. 2015. The World Bank Group's Partnership with the Global Environment Facility. Global Program Review. 2016. Knowledge-based Country Programs: An Evaluation of World Bank Group Experience. 2016. Growing the Rural Nonfarm Economy to Alleviate Poverty: An Evaluation of the Contribution of the World Bank Group. 2016 Cluster Country Program Evaluation on Small States (Vol. 1): Regional Program Evaluation of the Organisation of Eastern Caribbean States—Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines. 10IEG (2016) Cluster Country Program Evaluation on Small States (Vol. 1): Regional Program Evaluation of the Organisation of Eastern Caribbean States—Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines. 11While selecting the ‘best’ projects is intuitively appealing, the capacity of the implementing agency to do so can be questioned, especially if projects are evaluated on the basis of their additionality and spillover potential, as we argue should be the case, rather than simply growth potential. On the other end, a competitive selection mechanism could generate more interest if the grants are perceived to be limited and their awarding time-bound. Case study 9 looks at an example of a project with competitive selection. Due to the small number of projects choosing to select grantees with a competitive process, it is hard to assess whether the choice of selection mechanism actually brings any systematic benefits. Hristova, Diana; Coste, Antoine. 2016. How to make grants a better match for private sector development. Washington, DC: World Bank Group. 12IEG. Knowledge-Based Country Programs: An Evaluation of World Bank Group Experience (2016). 13 IEG (2016) Ibid. 42 14The authors further observed that matching gran ts “give SMEs only a temporary incentive to procure more BDS but do nothing to solve the inherent market failure that prevented them from using these services in the first place, the impact of the MG program on the BDS market would be questionable.” (Ibid. 12) 43