Approach Paper “Creating Markets for Sustainable Growth and Development” An Evaluation of World Bank Group Support to Client Countries FY 07-17 February 20, 2018 Background and Context 1. The international development community is increasingly turning to the private sector in its pursuit of the Sustainable Development Goals (SDGs). Involving the private sector as a financier, operator or service provider in SDG relevant areas requires “creating markets”, i.e., putting in place an enabling business environment, overcoming a range of markets constraints, and/or enhancing compe- tition through regulatory reform, pioneering investments or innovation. All these “creating markets” components imply certain roles for the private sector, the government and regulatory authorities. Since the 2002 World Bank Group (WBG) Private Sector Development Strategy creating markets has been a well-established part of the WBG-wide development agenda. 2. The WBG has re-confirmed its commitment to crowd-in the private sector in its most recent strategic Forward Look–A Vision for the World Bank Group in 2030. The Forward Look also introduces the concept of “creating market” which became one of two pillars of IFC’s latest strategy 3.0. Accord- ingly, the creation of markets is being pursued by a systematic focus across all three WBG institutions, using a deliberate and often sequenced deployments of WBG tools and instruments, following the so- called “Cascade”-intervention logic. But not always is a WBG-wide approach required to create mar- kets; also, individual investments may have a potential to create markets by integrating value chains. 3. Considering the heightened emphasis on market creation in WBG’s strategy, this evaluation offers IEG’s first systematic assessment, taking stock of WBG’s experience over the last ten years. It will look at relevant analytical work, the sequencing of up- and downstream work as well as the com- plementarity of WBG tools and instruments, and cover aspects of leveraging synergies across the WBG as well as WBG’s interactions with other MDBs and IFIs. With a learning emphasis, this evaluation seeks to inform the upcoming implementation of the creating market approach. CONTEXT AND ISSUES 1 4. Despite overall progress in achieving the SDGs and the WBG’s twin goals, 767 million peo- ple remain in extreme poverty and progress has been uneven across areas. Billions lack access to health care, education, infrastructure, credit, jobs, and safe food. Globally, 1.06 billion people still live without electricity access, with 80 per cent of them concentrated in just 20 countries. More than half the people without electricity live in sub-Saharan Africa. Without accelerating efforts to address these chal- lenges, gains of the past decades could be reversed, with poverty and fragility becoming more chronic in low income countries (LIC). (United Nations 2017). 5. The international development community acknowledges that the SDGs will not be achieved without the enhanced participation of the private sector. Estimates for investment needs required to achieve the SDGs in developing countries alone range from $3.3 trillion to $4.5 trillion per year. 2 These investments are mainly for basic infrastructure (roads, rail and ports; power stations; water and sanitation), food security (agriculture and rural development), climate change mitigation and ad- aptation, health, and education. Up to 70 percent of the investment gap could come from the private sector, according to international estimates. The annual investment gap of $2.5 trillion could be bridged by the private sector by providing funding of up to $1.8 trillion annually. This, however, would only 1|P a g e be possible if the current rate of annual investments with private participation of $0.9 trillion were to double. 6. The potential for increasing private sector participation is greater in some sectors than in others. Infrastructure sectors, such as power and renewable energy (under climate change mitigation), transport and ICT are natural candidates for greater private sector participation, under the right condi- tions and with appropriate safeguards. Other SDG sectors are less likely to generate significantly higher amounts of private sector interest, either because it is difficult to design risk-return models attractive to private investors, e.g., climate change adaptation, or because they are at the core of public service responsibilities and highly sensitive to private sector involvement, e.g., water, education, or health care (see Figure B.2 in Annex B for a visualization of anticipated private sector contribution). Therefore, public investment remains fundamental and pivotal (United Nations 2015). In addition to the SDG- relevant sectors referred above, the financial sector is essential in achieving the SDGs and the WBG twin goals due to the intermediation it provides between savers and investors, among other functions. Likewise, manufacturing is also essential as a provider of jobs, for example; and so is agriculture due to its additional role in enhancing food security. 7. Engaging the private sector as a financier, operator, service provider, or innovator in the pur- suit of the SDGs requires efficiently functioning and competitive markets. Such markets only emerge when there is a sufficiently conducive enabling business environment in place. At a very basic level, this includes ensuring property rights are introduced and respected and a sound judicial and contract- ing system is in place. To leverage the private sector for growth and development, economies need to overcome a range of additional regulatory and policy challenges: the provisions of services by the pri- vate sector in several of the SDG-related sectors are politically sensitive (for ex-ample, power, transport, or water) or are regarded as being of a public service nature (health or education). At the same time, several of these sectors may not be attractive for private capital, as power or transport are not compet- itive markets or contestable due to (excessive) government interventions, monopolistic behavior and/or the presence of dominant and potentially market-distorting state-owned enterprises. State sub- sidies may also distort markets, for example in the energy sector or state-owned utilities may not be sufficiently financially sound off-takers. Regulatory frameworks may not be sufficiently predictable or transparent or they impose regulatory burdens and administrative procedures incurring costs, all of which deters potential market entrants from investing. Hence, policymakers need to find the right bal- ance between creating a climate conducive to investment and removing barriers to investment in these markets on the one hand, and protecting public interests through regulation on the other (EIB 2016). 8. Lack of competition is another constraint to market creation because of few market players due to risk perception, or market entry barriers or the small size of the market; for example, investments into latest technologies or processes may only be profitable if future sales are large enough, i.e., when markets are large enough (Murphy et al 1989). Larger markets in general tend to attract more sellers; thus, they are more competitive than smaller markets (EIB 2016). 9. Markets may suffer from insufficient integration into value chains, for example in agribusi- ness where integration into regional or international supply chains has become increasingly crucial. Markets may also suffer from a lack of access to input factors, such as energy, finance or transportation with the latter providing the needed physical access to markets. Creating the needed physical and fi- nancial infrastructure as well as functioning capital markets, including capital markets relying on local resource mobilization, is therefore a necessary ingredient to the creating market process, in addition to these sectors being opportunities for market creation themselves. For more details, see attachment B. 10. Creating markets imply a range of activities, from fixing market failures in structurally weak economies to improving underperforming markets by enhancing competition, innovation, integra- 2|P a g e tion and skills. Depending on the maturity of the economy or the respective market within the econ- omy, creating markets may imply different activities: Markets 3 with little to no experience in crowding- in the private sector (i.e. nascent markets) are likely to focus on putting in place the basic legal and regulatory frameworks for a market-based economy. It is likely at this stage that market failures need to be addressed, for example, by creating the institutions and the legal basis for competition through anti-trust laws or abolishing market distorting subsidies, thus allowing new markets to emerge. Slightly more developed economies with still immature or underperforming markets are concerned about broad- ening the scope of market entry by private operators and market competition. While these countries have the basic ingredients for a market economy in place, they may have to overhaul their regular framework to ensure efficiency, i.e., reducing costs of complying with it, transparency, and predictabil- ity. Eventually, consolidating and already developed markets may want to test broadening private sector participation beyond the core infrastructure sectors to sectors where regulatory issues may be more difficult. Such countries are more concerned about deepening the involvement of the private sector within and across sectors. To this end, they focus on investments that allow to scale up the private sector involvement, enhance competition, enable integration into domestic and international value chains, upgrade skills/standards across supply chains, foster improved governance and managerial and process innovation. Creating markets is likely to require a concerted approach involving policy and regulatory reform to enhance the enabling environment along with support to investments. In cer- tain cases, though, markets could also be created by investments alone, when these investments are of strategic nature and provide a missing link in a sector’s value chain. 4 11. Innovation can play a twofold role: Innovation is needed to make the achievement of the SDGs possible by, for example, making economies more resource efficient and less pollution intense or by enabling service delivery with significantly reduced transaction costs, for example through FinTech in the financial inclusion space. But technology can also be “SDG-neutral” and still be a driver in market creation by enhancing competition and market deepening, in more advanced economies and hence contribute to growth in general (EBRD [2014], Mazzucato [2015]). 5, 6, 7 12. Summing up, across all stages of a country’s economic maturity recurring dimensions of creat- ing markets emerge: • Institutional Capacity as an important enabler, comprising:  Public sector capacity, policies and frameworks: To address market constraints and create (or improve) the needed investment climate, governments need to put in place the required policy and re regulatory frameworks, or enhance those in place, for markets to emerge, deepen or function better; and  Private sector institutions and firm level capacity, skills and governance. At the firm level, skills and capacity is needed for enhancing technical, operative and managerial performance, foster innovation, create standards across markets and value chains, and ensure efficient and transparent governance of companies. Private institutions and asso- ciations, (e.g., credit bureaus or stock exchanges) may provide services that underpin markets or set standards facilitating private competitive solutions while complying with good practice regulations. • Market Outcomes characterizing the nature and extent of market creation:  Innovation: new or modified technology, managerial and process improvements that allow the introduction of new products, services or processes yielding new or extending / deepening existing markets; eventually leading to 3|P a g e  Demonstration effects: early-entry investments, first-of-a-kind products or services that demonstrate the robustness of a regulatory environment, for example, or the viability of the business model, spurring replication and further market entry;  Competitive markets: markets where firms can effectively enter, exit, and compete, in- novating and striving for efficiency; and  Integrated markets: enhanced physical and/or financial connectivity, within and across markets and value chains. 13. Countries with limited experience in working with the private sector, such as those with structurally weak economies, Fragile and Conflict States (FCVs) and Least Developed Countries (LDCs) are likely to face the greatest challenges in creating markets. These countries not only have the greatest funding gap, they also face the greatest difficulties in attracting investors due to barriers to investments, lack of or insufficiently developed regulatory frameworks, a business environment that is not conducive to foreign investments, or weak financial infrastructure. The need to make special efforts in LDCs and FCS’ is therefore represented not only in international policy frameworks, but also in more concrete action plans of MDBs as well as of specific investment targets for the WBG. 14. Creating markets is part of the broader private sector development (PSD) agenda. Creating markets focuses on the factors needed to help markets to emerge, often involving changes in policies, rules and regulations, but also involving innovation and integration. Intervention or cluster of inter- ventions may have a market creation potential that range from inducing only incremental enhance- ments to supporting more transformative changes allowing a new market to form or an existing market to significantly function better. Such transformative changes typically address market failures of gov- ernment failures, as outlined above, and may reflect the various aspects of market creation, including privatization, governance and SOE enterprise restructuring, innovations in policies and regulation such as price liberalization, competition and trade policies, provision of the needed infrastructure and finan- cial sector reform (banking, interest rate liberalization and securities markets), or product and process innovations by the private sector that may reduce transaction costs and enable production and ex- change in new products or services or expand qualitatively access to existing products and services. For example, market creation in the power sector would involve moving from a government-controlled market where government departments operate generation, transmission and distribution in a mono- lithic structure to a scheme were these three functions are separated (“unbundled”), regulated by an independent regulator paving the way for the private sector to invest in generation and/or distribution (EBRD 2017). 8 15. In creating such markets, inclusiveness and sustainability matter. The challenge is to address such market constraints in a way that investment opportunities emerge with sufficiently attractive re- turns to private investors while guaranteeing inclusiveness, i.e., accessibility and affordability of services for all as well as integration of smaller commercial farmers and SMEs into domestic and global value chains, for example for the agricultural sector. and sustainability, i.e., reliable service provision meeting environmental and social standards (Tomlinson 2012) (UNCTAD 2014). In addition, markets should withstand economic shocks, i.e., be resilient, to allow them to be developmental impactful. 16. But there are also risks of designing markets poorly and a range of policy challenges that need to be managed carefully. Private investments and sector restructuring can lead to “cream skim- ming”, i.e., privatization of those entities that are profitable, leaving the public sector with the difficult cases to deal with (excessive cost and low revenue). This, in turn, can lead to an increase in the net fiscal costs if cross subsidies from these high profit centers can no longer finance excessive cost or low revenue segments of a market. Furthermore, introducing regulatory frameworks – while required for market 4|P a g e creation – need to consider the political dimensions of the process. Moreover, certain financing arrange- ments used in the context of market creation, such as Public-Private Partnerships (PPPs), bear risks. PPPs, for example, may have negative effects on public budgets because of contingent liabilities not being adequately managed. PPPs are reported as having inadequate risk allocation due to lack of com- petition during the bidding phase, which points at the importance of setting up well-designed local procurement processes in the context of market creation. (Estache 2016; WBI 2012; Hammami et al. 2006; Irwin 2007; Sadka 2006; Ter Minassian 2004). Practitioners must manage their expectations regarding the anticipated level of private sector engagement in some sectors of the economy which, up to date, has lagged expectations (see Attachment B, para 5). WORLD BANK GROUP’S ROLE AND CONTRIBUTION TOWARD CREATING MARKETS 17. Creating markets has been a well-established part of the Bank Group-wide development agenda, evidenced not least by the WB’s 2002 World Development Report, entitled “Building Institu- tions for Markets” that takes an institutional angle toward aspects of competition and innovation (World Bank 2002a). The 2002 Private Sector Development Strategy (PSD strategy) already recognized that “markets and market-type mechanisms play a role in the development process”; more specifically, “functioning markets are a critical mechanism to help reduce poverty”. The 2002 PSD strategy refers to the various ingredients of creating markets, including sound rules for the market and the expectation that such rules be adhered to, and physical access to markets. It introduced the primacy of investment climate influenced by “macro-economic stability, well-defined property rights, a sound judicial and contracting system, a reasonable level of certainty about government policy, functioning financial in- stitutions and a good physical infrastructure, such as a transport system” (World Bank Group 2002b). 9 18. The three WBG institutions have contributed to creating markets through their distinct roles and by deploying a range of tools and services. The traditional set of WBG tools and services have been applied for several years in efforts to engage the private sector in development. These include WB lending, including policy lending, that help to advance PSD-focused upstream reforms in client coun- tries, IFC investments and advisory services. In addition, guarantees (MIGA guarantees and WBG PRG), blended finance, and first loss structures have been applied to manage risks; mobilization was pursued through syndication and the AMC. The latter two groups are likely to gain renewed im- portance now, with the increased emphasis on creating markets in WBG strategies. These tools and services have been supported by diagnostic work, with varying focus on PSD and coverage of relevant aspects of creating markets (box 1). 19. Likewise, creating markets has also been part of IFC’s agenda. IFC’s Articles of Agreement define the Corporation’s mission as furthering “economic development by encouraging the growth of productive private enterprises” and seeking “to stimulate, and to help create conditions conducive to, the flow of private capital, domestic and foreign, into productive investment in member countries.” In the 1980s, IFC created Emerging Market bonds as an asset class. In the 1990s, for example, IFC helped create private markets in the former Soviet Union. 20. For IFC and MIGA, creating markets was even part of the institutions’ results matrix, embed- ded in a broad-based concept of development outcomes. Within the prevailing development outcome framework that IFC and MIGA have been using to articulate their development effects, IFC or MIGA- supported investments tried to contribute to “private sector development” (PSD) by, for example, test- ing regulatory frameworks in client countries and potentially improving these and/or by increasing competition through replication or innovation. 21. Creating Markets has now become a corporate priority—as something to be explicitly consid- ered and pursued in developing and implementing work programs and investment pipelines. What is new about this recently framed concept of creating markets is the systematic application across the 5|P a g e WBG through the so called “Cascade Approach” (“Cascade”) (see IFC Strategy and Business Outlook FY18-20, page 11, box 2), the explicit commitment to assess the potential for creating market of WBG interventions ex ante and the intention to monitor such market creation ex post, over the project lifespan (World Bank Group 2017). Note that the WBG’s response is embedded in a broader global initiative of mobilizing private finance and enhancing the private sector’s role as operator and service provider. See Attachment B for details. Box 1. Private Sector Development Diagnostic Work PSD diagnostics include the existing systematic country diagnostic (SCD), which integrates the PSD agenda in a more explicit manner and AAA work with a PSD focus, for example, (Rural) Investment Climate Assessments (ICAs/RICAs), financial development assessments as part of the Financial Sector Assessment Program (FSAP), and diagnostic trade integration studies (DTIS). Further diagnostic work includes the doing business project, the World Bank enterprise survey, the recently developed PPP country readiness diagnostic tool and the infrastructure prioritization framework, all of which contribute to informing the regulatory business environment of a country, regardless of whether they inform the design of WBG interventions or are used by the client country in its own pursuit of private sector development. In addition, upstream support is provided through IFC’s InfraVenture and IFC’s C3P advisory services that support governments in structuring PPPs. In the agribusiness sector a range of analytical tools are important: Global Agribusiness Deep Dive Diagnostic; Diagnostic for Agricultural Finance; Global Value Chain and Segmentation Analysis Toolkit and Training; and Investor Out- reach for Agribusiness (Sector Scan and Toolkit). The latter two are particularly important for building ‘local capacity to identify a pipeline of bankable projects’. In addition to these existing tools, a new Country Private Sector Diagnostics (CPSD) has been launched as of 2017 to better identify private sector opportunities at country level, starting with Ghana and Ka- zakhstan. Source: IEG. 22. The WBG has re-confirmed its commitment to crowd-in the private sector in its most recent strategic Forward Look– A Vision for the World Bank Group in 2030. In its pursuit to intensify and systemize its commitment to maximizing finance for development (MFD) 10 – to make “Billions-to-Tril- lions” a reality – the March 2017 Forward Look defined private finance mobilization as a core pillar. IFC’s Syndicated Loan Program, the Managed Co-Lending Portfolio Program (MCPP) for infrastructure as well as IFC’s Asset Management Company (AMC) are the key tools for mobilization. 23. The Forward Look also introduces the concept of “creating market” which became one of two pillars of IFC’s latest strategy. According to the Forward Look, IBRD/IDA, IFC, and MIGA are working closely together to enhance the WBG’s value chain – linking reform, investment, and mobilization. (De- velopment Committee 2017, p. 5). Efforts to “create markets” and mobilize private funds would be operationalized through systematic use of a decision tree model, the so-called “Cascade”. 11 Subse- quently, IFC’s latest strategy, Strategy and Business Outlook FY18-20 - Creating Markets and Mobilizing Private Capital (IFC 2017) also builds upon creating markets and mobilization as two main pillars in implementing the Bank Group-wide Forward Look (Box 2). Box 2. The WBG Concept of Creating Markets – An Integral Part of the Maximizing Finance for Development Agenda Within the broader concept of Maximizing Finance for Development (MFD), the Forward Look introduces the notion of “creating markets”. The goal is to “maximize development finance for value-adding investments, promote judicious use of scarce public and concessional resources, crowd-in commercial capital, and minimize the public debt burden”. Creating markets is operationalized through the system- atic use of a decision tree model, the so-called “Cascade”, which provides a logical framework for deriving a sequenced approach to engaging the private sector in development. 6|P a g e Definition: “Enabling the development of new markets or systematic changes to markets that deliver sustainable development impact. This includes developing underdeveloped markets or fostering systemic changes to excising but underperforming mar- kets.” 12 The Cascade. To maximize the impact of scarce public resources, the Cascade first seeks to mobilize commercial finance, enabled by upstream reforms where necessary to address market failures and other constraints to private sector investment at the country and sector level. Where risks remain high, the priority will be to apply guarantees and risk-sharing instruments. Only where market solutions are not possible through sector reform and risk mitigation would official and public resources be applied (see Figure). Figure. The Cascade Approach Note that interventions that do not follow the Cascade approach may also have the potential to create markets, according to IFC. In this context, investments in selected companies that provide the missing linkages in a sector’s value chain are crucial for creating markets. 13 Goals. Interventions that create markets – on a standalone basis or as part of a programmatic approach that may involve interventions by the WBG or others – are those that change the dynamics of markets in a way that promotes the following goals: (i) Competitiveness; (ii) resilience, (iii) integration, (iv) inclusiveness, and (v) sustainability. Criteria. Criteria to assess whether a project – on a standalone basis or as part of a programmatic approach – has the potential to create markets, IFC requires that a clear channel be identified, amongst the following options: (i) putting in place frameworks to enable markets to function; (ii) promoting competition; (iii) offering demonstration effects, including replication in other countries or regions; and (iv) building capacity and skills that open new market opportunities. 14 Sources: IFC Strategy and Business Outlook FY18-20 - Creating Markets and Mobilizing Private Capital (April 2017); Development Committee (2017). The Forward Look - A Vision for the World Bank Group in 2030 – Progress and Challenges, March 2017; IFC’s Anticipated Impact Measurement and Monitoring (AIMM) System: Pilot Framework and Implementation Update, Technical Briefing to the Executive Directors, September 15, 2017, The World Bank Group 2017. 24. A dedicated IDA Private Sector Window (PSW) was established in 2017 to catalyze market creation in structurally weak economies, such as IDA and FCS countries. To mobilize the private sector in countries where the SDG investment gap is the largest and where barriers have so far pre- vented the private sector to play a significant role, i.e., in IDA and FCS, a dedicated mechanism was adapted, the US$2.5 billion IDA IFC-MIGA Private Sector Window will (PSW). The PSW is intended to de-risk private sector investments in IDA-only countries, with a focus on IDA FCS. Essentially, the PSW aims to “de-risk” projects by shifting the risk to publicly-backed institutions, such as IDA or donor agencies willing absorb potential losses. 25. The theory of change in Figure 1 describes the roles of the three WBG institutions and links their interventions with outputs and intended outcomes. In summary, the Bank Group supports local governments through its policy advice and upstream support to put in place the enabling environment, i.e., public sector capacity, policies and frameworks for market creation. These include analytical work (WB 7|P a g e non-lending), WB lending and IFC advisory in support of sector reforms or to enhance the business regulatory environment for private sector participation, including areas such as investment climate, competition and entrepreneurship, and innovation etc. They also encompass support to pipeline devel- opment or to build capacity so the client countries can identify pipelines of potential bankable private sector projects themselves. WBG delivers this support, however, not in isolation, but rather in parallel or coordination with other MDBs (or other development partners). 26. In parallel, IFC’s private sector investments and MIGA guarantees support companies so they can grow and become sustainable service providers and operators in SDG relevant areas, such as infra- structure (energy, ICT, transport, etc.), finance, health, education, agribusiness, and other sectors. Like upstream work, such investments are often co-financed by other MDBs. If such investments and guar- antees have a demonstration effect, subsequent investment follow suit which deepens private sector par- ticipation. Moreover, such investments can lead to increased innovation, including managerial and pro- cess innovation, and enhanced skills and governance at the firm level. Complementary to this, IFC deploys its mobilization tools (syndication, Asset Management Company (AMC), Managed Co-Lending Port- folio Programs (MCPPs), leading to diversification, scaling and leveraging, eventually polling in more private capital, into SDG-relevant areas. In addition, the WBG supports the financial sector through lending, non-lending and investments, putting in place prudent financial sector policies, infrastructure and institutions. Together, these activities contribute to the creation of markets, manifested by increased competition and enhanced integration into domestic and international value chains (intermediate out- come). Secondary features of such created markets are inclusiveness in as much as these markets should provide access to services for marginalized groups (including the poor, women, youth, rural popula- tion) and / or equal economic opportunities / participation; resilience, i.e., able to withstand shocks, and sustainability, i.e., fostering social and environmentally sound practices. 27. Ultimately, market creation is expected to improve access to infrastructure, enhance inclusive growth, and create jobs, while conserving the fiscal space of the client countries, thereby allowing gov- ernments to allocate their scarce public funding to other development priorities (see “final outcomes”). This evaluation will focus on assessing the outputs, immediate and intermediate outcomes captured in the stylized theory of change (Figure 1). 15 28. IEG has evaluated many aspects of the theory of change over the last seven years. This eval- uation builds on a wealth of knowledge from previous IEG work related to creating markets. Those evaluations and learning engagements that directly pertain to the issue of creating markets are as fol- lows: Competitiveness and Jobs, Innovation & entrepreneurship, Investment Climate Reforms, Doing Business, Public Private Partnerships, Financial Inclusion, Support to SMEs, Housing Finance, Capital Markets, Support to Health Financing, IFC's Global Trade Finance Program, Assessing IFC's Poverty Focus and Results, IFC Client Engagement and WBG Joint Projects. In addition, this evaluation builds upon sectoral evaluations that covered private sector engagement, such as Sustained Transport, Information and Communication Technologies, Managing Forest Resources for Sustainable Development, Growth and Productivity in Agriculture and Agribusiness: Evaluative Lessons, Improving Effectiveness and Outcomes for the Poor in Health Nutrition and Population, Environmental Sustainability. Relevant lessons are summarized in Attachment C. 8|P a g e Figure 1. Stylized Theory of Change for World Bank Group Support to Creating Markets Source: IEG. Bolded areas covered by evaluation questions in greater detail. Notes: AS = Advisory Services, AMC = Asset Management Company, GIF = Global Infrastructure Facility, MCPP = Managed Co-Lending Portfolio Program, MDB = Multilateral Development Bank, PPIAF= Public-Private Infrastructure Advisory Facility, WBG = World Bank Group. Objectives and Audience 29. The objective of this evaluation is to distill lessons from the Bank Group’s experience in creating markets to leverage the private sector for sustainable development and growth. Such lessons are intended to inform future program development and the upcoming implementation of the Creating market / Cascade approach. In this regard, the evaluation will obtain evidence-based findings, develop broadly-applicable lessons across the Bank Group, and propose appropriate recommendations. 30. The evaluation, therefore, focuses more on the learning and strategic dimensions of the ex- ercise than on the accountability dimension. The evaluation does not intend to comprehensively as- sess whether the WBG was successful in creating markets in the past. Instead, the evaluation intends to identify the elements of good practice in market creation drawing from the lessons of experience of the body of recent IEG evaluations (see para. 21) and the proposed case studies (see Evaluation Design). 31. The proposed evaluation is IEG’s first systematic assessment of WBG support to creating markets. The notion of creating market is not new to the WBG, as stated earlier. What is new about the current WBG emphasis on creating market, is the systematic focus using a deliberate and sequenced de- ployments of WBG tools and instruments, following the so-called “Cascade”-intervention logic. While previous IEG evaluations have assessed aspects of creating markets, none of them has assessed the 9|P a g e issue in a systematic manner, looking at relevant analytical work, the sequencing of up- and down- stream work, and the complementarity of WBG tools and instruments. Value added is also derived by covering aspects of WBG coordination and leveraging of synergies, keeping the risks of conflict of in- terest between the roles of upstream policy advice and investor in mind, as well as WBG’s interaction with other MDBs and IFIs which will enable the evaluation to better articulate the WBG contribution. 32. The primary audience for this evaluation study are the World Bank Group’s Board of Direc- tors, management, and staff involved in delivering projects and programs that leverage the private sector for sustainable development and growth. Further stakeholders that can benefit from this study include the Bank Group’s client governments, multi-lateral and bi-lateral development banks and do- nors, the private sector, concerned civil society organizations, and the ultimate beneficiaries of policies that enable private sector participation and from increased access to new or better performing markets. External audiences include client countries and other members of the development community aiming at mobilizing the power of private markets to address pressing development challenges. 33. This evaluation is particularly timely as the Bank Group and the international community have made clear the key role that they private sector will play in addressing the $3 trillion yearly funding gap for the SDGs. To address these concerns, the Bank Group has re-confirmed its commit- ment to leveraging the private sector for sustainable development and growth in its most recent strate- gic frameworks. The Bank Group’s “Forward Look” introduced the concept of “creating markets” which was adopted by IFC as one of its two strategic pillars in “Strategy and Business Outlook FY18- 20 - Creating Markets and Mobilizing Private Capital (IFC 2017).” IFC is also in the process of refining its Anticipated Impact Measurement and Monitoring (AIMM) which seeks to explicitly articulate a pro- ject’s potential contribution to market creation. By looking at earlier efforts to leverage the private sec- tor, this evaluation can provide the evidence of what works and why things work and distill lessons from experience at this critical point in time. Evaluation Questions and Coverage/Scope EVALUATION QUESTIONS 34. The evaluation questions for this study have been derived from the review of existing WBG policy and strategies, a preliminary analysis of WBG projects, about 20 expert consultations across the WBG, complemented by a review of 21 previous IEG evaluations. Jointly, this process highlighted areas where operational challenges may persist, knowledge gaps remain, but also rich lessons exist. 16 . 35. The overarching question that IEG seeks to answer in this evaluation is: What lessons can be drawn from previous Bank Group efforts to create markets to leverage the private sector for sustainable develop- ment and growth, and how well is the Bank Group equipped to support countries in future as they create markets to engage the private sector to meet their development needs? This overarching question will be addressed by answering the below subordinated questions in the four areas of: The challenge of “delivering” mar- ket creation, tailoring interventions to country needs, putting in place the enabling environment for market creation, and financing transaction and their potential effects on innovation and demonstration, competition and integration. 1.) Tailoring interventions to countries and sectors and underpinning analytics a) What analytical tools are available to the WBG to design and prioritize its interventions so they reflect a clear understanding of the market gaps and failures? What works about identifying and addressing such market gaps? b) Are the current analytical tools appropriate and sufficient to develop country strategies 10 | P a g e and programs? 2.) Putting in place the public-sector capacity, policies and frameworks ─ WBG policy and up- stream support a) What is the evidence that WBG contributed to the identification of bankable projects for the private sector to invest or has contributed to building institutions and capacity so that the country could identify such a pipeline? Were such institutional and capacity building efforts sustainable? b) What can we learn from WBG policy upstream support regarding its potential to (i) target market gaps and failures and what from cases where this was not the case; and (ii) engage in the private sector as financier, operator or innovator? 3.) Finance, innovation, demonstration, competition and integration ─ WBG downstream work a) What is the evidence that WBG financed projects lead to innovation and what were the driving factors enabling such innovation, including new products, services, business pro- cess and/ or technological innovation? b) Have WBG financed projects had a pioneering character and subsequently induced demonstration and replication? What were the factors that allowed these projects to con- tribute to such demonstration effects? c) Have WBG financed projects lead to enhanced skills and governance at the firm level and what were the drivers of success? d) What is the evidence that supported companies and operators provide access to services of sufficient quality and in a sustainable and affordable manner, including for the Base of the Pyramid? e) Ultimately, what is the evidence that WBG financed projects contributed to increased com- petition and integration, thus creating markets and/or making these more efficient? What were the factors that allowed to enhance competition and integration and how could such experience be applied elsewhere? f) What is the evidence that WBG created markets that are resilient, inclusive and fostering environmental sustainable practices? 4.) The delivery challenge ─ factors that determine the success of implementing the creating mar- ket approach a) What lessons can we draw from complementary upstream and downstream approaches about WBG coordination, collaboration and leveraging of synergies? Was the sequencing of WBG support or interventions ad hoc or intentional and at what stage of planning or implementation was it introduced? How were conflicts of interest addressed? b) How has the WBG considered other MDB strategies and operations from conceptualiza- tion to implementation and how did this impact results? What factors contributed to or prevented such coordination? c) Does the WBG have the right processes, tools and incentives in place to engage in inter- ventions with market creation potential and do these incentives foster responsible risk tak- ing and is it commensurate with the WBG objectives? 17 11 | P a g e 36. The proposed evaluation questions cover defined areas of the Theory of Change. Starting on the left of the Theory of Change (Figure 1), the first set of questions, addresses aspects of delivery and coordination, followed by a set of questions on the analytical underpinnings of the creating market approach. Subsequently, a set of questions addressed the immediate outcome dimension of creating markets: putting in place public sector capacity, policies and frameworks; demonstration and innova- tive character of investments; followed by questions on intermediate outcomes in creating markets re- lated to competitiveness and integration. Those areas in the Theory of Change that will be covered by this evaluation in greater detail are highlighted in bold in Figure 1. Final outcomes are not covered. 18 EVALUATION SCOPE 37. This evaluation plans to derive lessons that are relevant for the entire World Bank Group portfolio. The evaluation will apply a comprehensive and sequenced mixed-methods approach com- prising of 15 cases studies allowing for an in-depth treatment of three SDG-relevant sectors, coupled with a portfolio review and analysis within these sectors. All of this will be complemented by the min- ing of lessons from more than 20 previous IEG evaluations relevant to creating markets across a range of relevant sectors, including energy and electricity, ICT, transport, water, health, education, forestry, finance and capital markets, complemented by a review of the literature. Jointly this approach will en- able the evaluation to arrive at a comprehensive treatment of all WBG interventions, including WB lending, non-lending, ASA, IFC investments, IFC advisory services, and MIGA and WB guarantees, approved FY07-17. 19 WBG interventions channeled through the recently approved IDA PSW are ex- cluded from the scope of the evaluation. The FY07-17-time window will allow this evaluation to look at market creation even when it involved relatively long-term transformation processes. Given that the evaluation portfolio encompasses a large share of IFC’s and MIGA’s operations and a sizeable portion of WB support to business regulations and sector reforms, this evaluation does not aim at capturing this underlying portfolio in a comprehensive manner; moreover, a large share of the evaluation portfo- lio has already been assessed through previous IEG evaluations (see above). 38. To derive comprehensive lessons, this evaluation will focus on a program of support to create markets for private sector in three selected sectors. Sectors of relevance for the pursuit of SDGs with the highest potential for private sector participation are: power/energy, agriculture (as one of the ena- blers for food security), telecommunication, transport, and finance, followed by health, water and edu- cation with a somewhat diminished potential for private sector participation (UNCTAD 2014 and 215, see Figure B.2. in Annex B). This evaluation will focus on three of these sectors. The selection criteria are the following: (i) achieving a balance across IFC’s focus industries, per IFC’s Strategy 3.0. (IFC 2017); (ii) providing a strong link to the pursuit to the SDGs; (iii) addressing at least one infrastructure sector to be able to study the regulatory challenges associated with such cases, while covering both type of cases: those that lend themselves to the application of the Cascade Approach, which offers opportuni- ties to study the WBG-wide dimension of creating markets; and cases of IFC standalone investments. In addition to these three sector deep dives, the evaluation will cover aspects of the financial sector to the extent it affects market creation through its effects on supply and demand in each real economy sector, including the design and implementation of new financial instruments. For instance, the provi- sion of long-term finance for infrastructure development in infrastructure sectors or the provision of local currency finance; financial services to the rural population, MSMEs and farmers associated with the agribusiness; or innovative student loan programs to enhance access to education services. 20 39. The selection of cases within these sectors will focus on markets where WBG interventions con- tributed to significant shifts in how markets are designed, regulated or function. As outlined above, this will allow to better differentiate market creation from general PSD. Comparison across cases ─ within sectors across different maturity levels as well as across sectors within the same maturity level ─ will 12 | P a g e enhance the internal and external validity. Looking at cases of success and failure will enable the team to identify factors that contributed to positive and not so positive outcomes in market creation 40. This evaluation will assess aspects of coordination and collaboration across the WBG as well as with other MDBs. As the application of the cascade approach calls for a collaborative effort across IBRD, IDA, IFC and MIGA, the evaluation will place a special emphasis on distilling lessons from joint projects. Focusing on the second and third step of the cascade (see Figure in Box 3), the evaluation will assess questions of coordination and sequencing of interventions. Given that market creation and max- imizing finance for development is an endeavor that all major MDBs subscribed to (see G-20, 2017), this evaluation will assess the contributions of the WBG in context of the role of the other MDBs. Evaluation Design and Evaluability Assessment 41. The evaluation design benefited from valuable interactions with stakeholders and subject- matter experts and from a careful review of the WBG’s recent strategic documents that reference the Creating Markets concept. During the early phases of the review, IEG interacted with World Bank Group strategy teams and with staff working on public private partnership areas and on priority sectors such as infrastructure (energy, transport, ICT), finance, and agribusiness. These interactions, together with a review of relevant literature and the most recently published strategies – Forward Look (WBG, 2017) and Creating Markets and Mobilizing Private Capital (IFC, 2017) – informed the evaluation ap- proach by highlighting important concepts and frameworks that would facilitate the design of the an- alytical framework and the selection of evaluation methods. 42. The evaluation will adopt a theory-driven, case-based approach to analyze selected aspects of the evaluation’s theory of change across multiple levels (i.e. sectors and levels of maturity). 21 This ap- proach will allow the evaluation to open the “black box” between intervention and outcome to provide information on whether the program succeeded and on “how” and “why” it did so, with a view of improving future program effectiveness and program theory. The analysis will be case-based and will cover multiple-levels, namely: three levels of maturity (nascent, immature, developed) across three se- lected sectors. See also Figure 2 and Attachment D. Figure 2. Stylization of the evaluation’s multi-level analytical framework Sources: IEG Review and interviews with World Bank Group subject-matter experts and management. 43. The evaluation will adopt a sequenced mixed-methods approach whereby findings from methodological components progressively build on each other. The key methodological components include sector-focused case studies, literature and document reviews, semi-structured interviews, and data analysis, following the below outlined sequence. Below is a summary of the selected methods, their objectives, and sources of information (Table 1, see also annex D for details). 13 | P a g e Table 1. Summary of Evaluation Components and their Descriptions Evaluation Component Description Portfolio review and analy- The evaluation will conduct a systematic desk review and assessment of projects along the sis (PRA) evaluation’s dimensions of creating markets to identify design features and characteristics, re- sults indicators, and drivers of success and failure. Embedded country / sector Up to 15 cases will be assessed to learn about the World Bank Group’s efforts to leverage the case studies 22 private sector for sustainable development and growth in client countries. Cases will aim at identifying “how” and “why” specific interventions were or were not successful in delivering the intended results. Lessons mining in IEG Ma- Systematic desk review of IEG’s portfolio of Major Evaluations with a focus on (i) the enabling jor Evaluations environment, (ii) financial markets, and (iii) specific sectors (e.g. electricity access, transport, ICT, water, health, agriculture, and forest). A qualitative meta-analysis, the review will identify and classify the factors identified by these evaluations as facilitating or constraining the imple- mentation of their interventions. Reconstruction of the The- Reconstruction of how the outcomes sought by the World Bank Group in leveraging the private ory of Change sector for sustainable development and growth were expected to happen. Literature reviews Structured review of academic, evaluation, and other literature on leveraging the private sector for sustainable development and growth across each of the selected sectors. Semi-structured interviews Semi-structured interviews with subject matter experts within IEG, the broader World Bank Group, and external stakeholders such as governments, donors, non-governmental agencies, academics, and private sector entities. Review of data and indica- The evaluation will identify and utilize indicators aligned with the evaluation questions and se- tors lected sectors to identify sector priorities and changes over time. Indicators sources include WBG and external sources such as EIU, WEF, UN agencies, among others. Portfolio data will be mapped against these indicators. 44. The evaluation will conduct a Portfolio Review and Analysis (PRA). The Portfolio Review and Analysis (PRA) will analyze the potential of WBG interventions to create markets in the three sec- tors and identify results drivers. The PRA will allow to shed light on WBG interventions covering both WBG upstream support to a sector’s reform agenda and associated public investments as well as IFC’s / MIGA’s standalone investment projects and their potential to create markets. As it is impossible to capture all creating markets-relevant interventions, the PRA will focus on the portfolios of the case study countries (see below). 45. In addition, the evaluation will conduct up to 15 case studies of countries where the World Bank Group has attempted to support market creation. These cases will cover examples of creating markets across the various maturity stages of an economy (nascent, immature, and developed). This approach will facilitate the drawing of lessons within and across the three deep-dive sectors as well as across and within countries of the same maturity level. Of these 15 case studies, six to nine case studies will be supported by field missions; the remaining will be desk-based cases, enriched by market data analysis and interviews with counterparts. 23 These cases will take a comprehensive approach by con- sidering the “eco-system” needed to develop markets. Each case will therefore cover WBG’s support to setting up a favorable business environment in general, through interventions in the space of competi- tiveness, investment climate, doing business, trade and FDI policies, institutions building, macro poli- cies, and regulatory frameworks etc. In addition, each case will cover the role of important and relevant input factors, i.e. access to finance (see para 37 for details) and access to physical infrastructure and hence access to markets, as well as the role of other market players, beneficiary and clients. 24 14 | P a g e 46. Cases will focus on deriving lessons on what works regarding creating markets, in areas where WBG has been active. Internal and external validity will be enhanced by conducting in total nine cases across three different sectors and three different maturity level and comparison across these cases. The selected cases will be used to generate insights into which sectors would benefit from market creation in each country and derive lessons on how the various policy areas inter-link. The case ap- proach will also allow for insights whether WBG has missed to create markets in the given countries; but the proposed case design does not lend itself to comprehensively assess the extent to which the WBG has missed opportunities in creating markets as this would require a comprehensive capture of all cre- ating markets activities which, for reasons outlines above, is not possible. 47. Case selection and design will be informed by a systematic review of IEG major evaluations and of relevant literature to establish a logic that would allow for generalizability across the levels depicted in the evaluation’s analytical framework. By using carefully constructed case protocols, the evaluation will be able to test findings against the established logic and to compare them across sectors, maturity stages, and dimensions of creating markets (see annex D figure 4). 48. Case selection will be systematic but purposeful. The following selection model will ensure the evaluation adequately balances the tradeoffs between depth and breadth of analysis while making sure the cases are selected in a systematic and transparent manner. Specifically, the evaluation will: • Use external data to classify countries across sector maturity stages • Examine the WBG’s portfolio of support in each country/sector to identify a short-list of coun- tries that meet the evaluation’s selection criteria (annex D) • Consult with WBG internal and external experts to identify the countries / cases and within these cases the relevant interventions that offer the richest opportunities for learning • For each focal sector, develop sector specific theory of change to reflect the context and county factors adequately, including suitable matrices for assessing market creation. • Strive to cover both type of creating market cases, those where the Cascade interventions logic was applied, and cases of standalone investments. 25 DESIGN LIMITATIONS 49. Several factors might constrain the evaluation; these fall broadly into two categories: limita- tions from conscious choices about scope and from the availability and quality of existing data and documentation. To manage the tradeoff between breadth and depth of analysis, the evaluation ap- proach makes the necessary choice of focusing the analysis on three key sectors. The combination of focused PRA, desk and mission-based case studies will allow to derive robust lessons, noting the lim- ited generalizability of a case-based approach. This choice was informed by the literature and by initial stage stakeholder consultations. Data and documentation constrains will include, among others: (i) identification of cases will rely on external data (which may have caveats of its own and may not be complete for the full range of countries) and on WBG portfolio coding systems (which are not always accurate) and (ii) strategy-, policy-, technical and/or project-level documentation (which is not always available nor is it always consistent). To the extent, gender disaggregated data are available, the team will present its findings reflecting gender issues as appropriate. Quality Assurance Process 50. This approach paper has been peer reviewed to ensure relevance of evaluation questions and issues covered, adequacy of scope of the evaluation, and appropriateness of methodology. The four peer reviewers come from outside IEG: Anne Marie Chidzero (from the Financial Sector Deepening- Mozambique Fund, and Board member of the Africa Enterprise Challenge Fund and the Advisory 15 | P a g e Board of Women’s Banking); Antonio Estache (Professor of Economics at Université Libre de Bruxelles, Belgium); Mattia Romani (Managing Director for Economics, Policy and Governance at the EBRD); and Simon Gaviria (former General Director of the National Planning Department, Colombia). Expected Outputs, Outreach, and Tracking 51. Planned Reporting Vehicle. The primary output of the evaluation will be the report to the Board’s Committee on Development Effectiveness (CODE), which will contain the main findings and recommendations. The finished evaluation will be published and disseminated both internally and ex- ternally. IEG will develop working papers, presentations, blogs, videos, and other products as appro- priate for other audiences for the evaluation, including key stakeholders. 52. Regular stakeholder interaction will be sought to enhance the evaluation process. This will in- clude consultation while the evaluation is under way and dissemination and outreach once the study is complete. During evaluation preparation, the team will solicit feedback from stakeholders, World Bank Group management, investment and sectoral practitioners, and government agencies, to improve the evaluation’s accuracy and relevance. Such stakeholder interaction will contribute valuable infor- mation and qualitative data to supplement interviews, case studies, and other research. Consultations will also be held during field missions with stakeholders, including government counterparts, Bank staff, nongovernmental organizations, donors, private sector actors, and beneficiaries. 53. Outreach strategy. In addition to outreach during the evaluation process, IEG will implement an outreach plan once the evaluation is completed. IEG will launch the report in Washington, DC, and at a major international conference. The efforts will target key stakeholders, including staff at head- quarters and country offices, other multilateral development banks and donors, government authori- ties, civil society organizations, and counterpart officials. Resources 54. Timeline and budget. The evaluation will be submitted to CODE by the end of Q3 FY19. The budget for the study is estimated at $1,000,000 an amount consistent with other major IEG sector studies and include dissemination and outreach activities. 55. Team and Skills Mix. The skills mix required to complete this evaluation includes expertise in private sector development management, evaluation experience, and knowledge of IEG methods, in- cluding case study analysis and portfolio analysis; familiarity with the policies, procedures, and oper- ations of IFC, MIGA, and the World Bank; and knowledge of the Bank Group and external sources. The evaluation will be prepared by a team led by Stefan Apfalter (task team leader), and composed of Kon- stantin Atanesyan (senior evaluation officer and co-lead), Jacqueline Andrieu (evaluation officer), Ma- ria Elena Pinglo (evaluation officer), Arianna Ranuschio and Victor Malca (PSD consultants). In addi- tion, specific sector expertise from within IEG will be leveraged and external consultants be included as needed. This team has substantial knowledge and experience in key subject matters and about the respective institutions of the Bank Group, and methodology. Emelda Cudilla provided administrative and logistical support. The report will be prepared under the direction of Stoyan Tenev, Senior Man- ager, IEGFP; and José Carbajo Martínez, Director, IEGSP. 16 | P a g e Attachment A BIBLIOGRAPHY Banerjee, Abhijit and Ester Duflo, (2007). The Economic Lives of the Poor, Journal of Economic Perspectives 21, 141–67 Bator, Francis M. (1958). "The Anatomy of Market Failure," Quarterly Journal of Economics Collins, Darryl, Jonathan Morduch, Stuart Rutherford, and Orlanda Ruthven (2009) Portfo- lios of the Poor: How the World’s Poor Live on $2 a Day, Princeton University Press. Cunningham S. (2011). Understanding market failures in an economic development context, Meso- partner Monograph 4, Ed 1, July 2011 Development Committee (2017). The Forward Look - A Vision for the World Bank Group in 2030 – Progress and Challenges, March 2017 EBRD (2014) EBRD Transition Report 2014 – Innovation in Transition EBRD (2017) Transition indicators methodology, EBRD Office of the Chief Economist, retrieved Dec 2017, source: http://www.ebrd.com/cs/Satellite?c=Con- tent&cid=1395237866249&pagename=EBRD%2FContent%2FContentLayout EIB (2016) Breaking Down Investment Barrier at Ground Level, European Investment Bank, Sep- tember 2016 Estache (2016) Institutions for infrastructure in developing countries: what we know … and the lot we still need to know. EDI Working paper series WP16/6.I G-20. (2017). Principle of MDBs’ strategy for crowding-in Private Sector Finance for growth and sustainable development, April 2017 Hammami, Ruhashyankiko, Yehoue (2006) Determinants of Public-Private Partnerships in In- fra-structure, IMF Working Paper, WP/06/99. IFC (2011). International Finance Institutions and Development Through the Private Sector, A joint report of 31 bilateral development finance institutions, International Finance Corpo- ration, Washington DC, 2011 IFC (2017). IFC Strategy and Business Outlook FY18-20- Creating Markets and Mobilizing Private Capital (April 2017) IFC (2017b) Technical Briefing – IFC’s Anticipated Impact Measurement and Monitoring (AIMM) System, September 5, 2017 Irwin (2007) Government Guarantees: Allocating and Valuing Risk in Privately Financed Infra- structure Projects. World Bank. Krueger, Anne O. (1990). "Government Failures in Development," Journal of Economic Per- spectives, Lin, J. Y. (2011) New structural economics: A framework for Rethinking Development, The World Bank Research Observer 26:193-221, July 2011 17 | P a g e Mazzucato (2015) From Market Fixing to Market Creating – A new framework for economic policy, ISI Growth, Working Paper 2/2015 October McMillan J. (2002), Reinventing the bazaar: a natural history of markets, New York 2002 Murphy, K M, Shleifer, A, Vishny R (1989) Income distribution, market size, and industrializa- tion, The Quarterly Journal of Economics 104(3), pp 537-64 Orbach, Barak (2013). "What Is Government Failure," Yale Journal on Regulation Theodore Panayotuo (2000), The Role of Private Sector in Sustainable Infrastructure Develop- ment, Harvard Institute for International Development, Yale E&S bulletin 101. Tomlinson B. (ed.) (2012) Aid and the Private Sector: Catalyzing Poverty reduction and Develop- ment? The Reality of Aid International Coordinating Committee Roth, Alvin E. (2007). “The Art of Designing Markets”, Harvard Business Review Sadka (2006) Public-Private Partnerships: A Public Economics Perspective, IMF Working Paper WP/06/77. Schmidt-Traub, G (2015) Investment Needs to Achieve the Sustainable Development Goals – Un- derstanding the Billions and Trillions. SDSN Working Paper, Nov 2015 Stiglitz, Joseph E. (1998). "The Private Uses of Public Interests: Incentives and Institutions," Jour- nal of Economic Perspectives Stiglitz, Joseph E. (2009). "Regulation and Failure," in David Moss and John Cisternino (eds.), New Perspectives on Regulation Ter-Minassian (2004) Public-Private Partnerships, International Monetary Fund. UNCTAD (2014), World Investment Report 2014. UNCTAD (2015) Access to financial services as a driver for the post-2015 development agenda. Pol- icy Brief, No 35, September 2015 United Nations (2015) Investing in Sustainable Development Goals. Action Plan for Private In- vestments in SDGs, UNCTAD United Nations (2017) The Sustainable Development Goals Report 2017, The United Na- tions, New York, 2017 World Bank (2002a) World Development Report. Building Institutions for Markets, Oxford University Press, 2002 World Bank Group (2002b) Private sector Development Strategy – Directions for the World Bank Group, April 9, 2002 World Bank (2016) Poverty and Shared Prosperity 2016: Taking on Inequality. World Bank Group (2017) IFC’s Anticipated Impact Measurement and Monitoring (AIMM) System: Pilot Framework and Implementation Update, Technical Briefing to the Ex- ecutive Directors, September 15, 2017, The World Bank Group 2017 The World Bank Institute (2012) Public-Private Partnerships Reference Guide, Version 1.0, The World Bank Institute and Public Private Infrastructure Advisory Facility Zerbe Jr., Richard O. and McCurdy, Howard E. (1999). "The Failure of Market Failure," Jour- nal of Policy Analysis and Management. 18 | P a g e Attachment B CONTEXT – CREATING MARKETS, THE SDGS, MFD AND MOBILIZATION Toward Achieving the SDGs Progress has been made in achieving the SDGs and the WBG’s twin goals of end- ing extreme poverty by 2030 and boosting shared prosperity. 1.1 billion fewer people were living in poverty worldwide in 2013 compared with 1990. The incomes of the poorest people in 60 out of 83 countries reviewed increased between 2008 and 2013. (World Bank 2016). Nevertheless, 767 million people remain in extreme poverty and progress in other areas has been uneven. Billions lack access to health care, education, infrastructure, credit, jobs and safe food. Globally, 1.06 billion people still live without electricity access, with 80 per cent of them concentrated in just 20 countries. More than half the people without elec- tricity lived in sub-Saharan Africa. Without accelerating efforts to address these challenges, gains of the past decades could be reversed, with poverty and fragility becoming more chronic in Low Income Countries (LIC). The latest The Sustainable Development Goals Report 2017 concludes that, while considerable progress has been made over the past decade across all areas of development, the pace of progress observed in previous years is insufficient to fully meet the SDGs by 2030: Faster and more inclusive progress is needed. (United Nations 2017). Achieving the SDGs has significant resource implications. Estimates for invest- ment needs in developing countries alone range from $3.3 trillion to $4.5 trillion per year, mainly for basic infrastructure (roads, rail and ports; power stations; water and sanitation), food security (agriculture and rural development), climate change mitigation and adapta- tion, health, and education. The SDGs will require a step-change in the levels of both public and private investment in all countries. At current levels of investment in SDG-relevant sec- tors, developing countries alone face an annual gap of $2.5 trillion (Figure B.1). In develop- ing countries, especially in LDCs and other vulnerable economies, public finances are cen- tral to investment in SDGs. However, they cannot meet all SDG-implied resource demands. Therefore, the development community looks at the private sector to fill the gap. 19 | P a g e Figure B.1. The SDG Investment Gap and Potential Private Sector Contributions [Trillions of US$] Sources: UNCTAD 204 and UNCTAD 2015. CC=Climate change Up to 70 percent of the investment gap could come from the private sector, per in- ternational estimates. The annual investment gap of $2.5 trillion could be bridged by the private sector by providing funding of up to $1.8 trillion annually. This, however, would only be possible if the currently rate of private participation of $0.9 trillion were to double. While this is considered feasible at aggregate level, structurally weak economies, fragile and conflict affected countries and LDCs, would need special attention. Their investment needs are significantly higher than those of other economies and investors are more cautious about allocation funds there. However, so far, the participation of the private sector in investment in SDG- related sectors is relatively low. Only a fraction of the worldwide invested assets of banks, pension funds, insurers, foundations and endowments, as well as transnational corpora- tions, is in SDG sectors. Their participation is even lower in developing countries, particu- larly the poorest ones. In LDCs, a doubling of the growth rate of private investment would be a desirable target. Developing countries as a group could see the private sector cover ap- proximately the part of SDG investment needs corresponding to its current share in invest- ment in SDG sectors, based on current growth rates. In that scenario, however, they would still face an annual gap of about $1.6 trillion. In LDCs, where investment needs are most acute and where financing capacity is lowest, about twice the current growth rate of private investment is needed to give it a meaningful complementary financing role next to public investment and overseas development assistance (ODA). But the increasing trend of private sector flows give reasons to hope that private capital will play a more key role in funding the pathway toward the SDGs. Over the last 25 years, private investment flows have gained importance compared to ODA: While ODA has increased from US$55 billion in 1990 to US$161 billion in 2014, private investment flows to developing countries have grown exponentially, with Foreign Direct Investment (FDI) 20 | P a g e growing 30-fold over the same period. (IFC 2017). Though net FDI in LICs and Middle-In- come Countries (MIC) was nearly US$650 billion in 2015, much larger pools of private fi- nance have yet to be meaningfully engaged. The potential for increasing private sector participation is greater in some sectors than in others. Infrastructure sectors, such as power and renewable energy (under climate change mitigation), and transport are natural candidates for greater private sector participa- tion, under the right conditions and with appropriate safeguards (Figure B.2). Other SDG sectors are less likely to generate significantly higher amounts of private sector interest, ei- ther because it is difficult to design risk-return models attractive to private investors (e.g. cli- mate change adaptation), or because they are at the core of public service responsibilities and highly sensitive to private sector involvement (e.g. education and health care). There- fore, public investment remains fundamental and pivotal (United Nations 2015). Figure B 2. Potential Private Sector Contributions to the SDG Investment Gap, by Sector [Billion US$] Source: UNCTAD 2014 and UNCTAD 2015. CC=Climate change In addition to the SDG-relevant sectors referred to in Figure B 2, the financial sec- tor is essential in achieving the SDGs and the WBG twin goals. Adopting financial sector policies to enable access to financial services for the poor as well as to allow a sound finan- cial sector and capital markets to develop are essential. This is particularly true as the finan- cial sector represents a market itself as well as it provides the necessary financial infrastruc- ture for the other parts of the (real) economy to function. This includes efforts to develop an efficient financial sector, including capital markets, as well as initiatives to advance access by poor families and microenterprises to financial services, i.e. financial inclusion. Capital markets and bank finance, provide funding and help households and firms to better manage risks associated with long-term investments. As integral parts of the financial framework, banking and capital markets help finance growth and thereby have the potential to alleviate 21 | P a g e poverty. On the retail end of the financial markets, financial inclusion is regarded a driver for development and achieving the SDGs (UNCTAD 2015). Financial inclusion has the po- tential to benefit the poor through an array of channels both directly or indirectly. For exam- ple, affordable and reliable payment systems have the potential to facilitate day-to-day fi- nancial transactions such as remittances or through credits and savings to smooth consumption (Collins et al. 2009, Banerjee and Duflo 2007). HOW TO ATTRACT THE PRIVATE SECTOR AND CREATE MARKETS Attracting private sector funding for the SDGs requires overcoming a range of regulatory and policy challenges. The provision of services in several of the SDG-related sectors are sensitive (Figure B.2, power, transport or water, for example) or are regarded of a public service nature (health or education). At the same time, several of these sectors may not be attractive for private capital, as power or transport are not competitive markets or contestable due to (excessive) government interventions, monopolistic behavior and / or the presence of dominant and potentially market-distorting state-owned enterprises. Hence, policymakers need to find the right balance between creating a climate conducive to invest- ment and removing barriers to investment in these markets on the one hand, and protecting public interests through regulation on the other. The challenge is to find mechanisms to pro- vide sufficiently attractive returns to private investors while guaranteeing accessibility and affordability of services for all. Mechanisms are required to channel private sector support so it reduces poverty in an inclusive manner and considers the needs and constraints of the poor and marginalized groups including women (Tomlinson 2012). This points at the need of balancing liberalization and the right to regulate. Box B.1 summarizes the ingredients for functioning markets. In addition, the mobilization of private funds must ensure that these are being channeled into investments in SDG-relevant sectors while maximizing their sus- tainable development impact and minimizing risks or drawbacks involved (UNCTAD 2014). Box B.1. Well-Functioning Markets – What Do We Know from The Theory The concepts of markets and market economies are fundamental to all economic theories and therefore under- pin most aspects of thinking on development. A “market” is generally defined as a process that facilitates trade and enables the distribution and allocation of resources in a society, and by which the prices of goods and services are established; and as a system with a structure of a perfect competition. The theory of perfect com- petition is key for definition of the structure of a well-functioning market. Well-functioning markets of the real world are never perfect, but can be characterized by the following criteria: (a) ease of entry and exit; (b) absence of significant monopoly power; (c) widespread availability of information; (d) absence of market externalities; and (e) achievement of public interest objectives. The basic institutions of the market economy can be subdi- vided into five categories: (i) private property; (ii) free markets; (iii) competition; (iv) division and combination of labor; and (v) social cooperation. These are mutually dependent institutions: each implies the other, and makes it possible. 22 | P a g e Economic theory traditionally focuses on the study of market structure and the efficiency of market equilibrium; when the latter is not efficient, it means that a market failure has occurred - a situation in which the inefficient allocation of goods and services is caused by exogenous systems. Market failures are often associated with such factors as: time-inconsistent preferences, information asymmetries, non-competitive markets, principal– agent problems, externalities, or public goods. As most markets do not inherently satisfy all the conditions nec- essary for a well-functioning market, and are characterized by varying degrees of market imperfection, govern- ments examine the type and significance of the market imperfections, and the need for various kinds of market intervention. Thus, existence of a market failure is often the reason for interventions by self-regulatory organizations, gov- ernments or supra-national institutions. However, government policy interventions, such as taxes, subsidies, bailouts, wage and price controls, and regulations (including poorly implemented attempts to correct market failure), may also lead to an inefficient allocation of resources, sometimes called government failure (or non- market failure). Government failure occurs when (i) government intervention causes a more inefficient allocation of goods and resources; (ii) the government performs inadequately; and (iii) it fails to intervene or does not intervene sufficiently. There are many indices and ratings that measure and rank various aspects of market performance, composition, openness, and competitiveness (WBG Doing Business, WEF Global Competitiveness, etc.) A useful tool devel- oped by the Heritage Foundation - the Index of Economic Freedom - ranks annually all countries based on 12 quantitative and qualitative factors, grouped into four broad pillars of economic freedom (equally weighted): (i) Rule of Law (property rights, government integrity, judicial effectiveness); (ii) Government Size (government spending, tax burden, fiscal health); (iii) Regulatory Efficiency (business freedom, labor freedom, monetary free- dom); and (iv) Open Markets (trade freedom, investment freedom, financial freedom). From this “wholesale” perspective, countries considering economic reforms may find opportunities for improving economic perfor- mance in those factors in which they score the lowest, and which may indicate significant binding constraints on economic growth and prosperity. Sources: IEG based on Bator, Francis M. (1958). "The Anatomy of Market Failure," Quarterly Journal of Economics; Krueger, Anne O. (1990). "Government Failures in Development," Journal of Economic Perspectives; Orbach, Barak (2013). "What Is Government Failure," Yale Journal on Regulation; Roth, Alvin E. (2007). “The Art of Designing Markets”, Harvard Business Review; Stiglitz, Joseph E. (1998). "The Private Uses of Public Interests: Incentives and Institutions," Journal of Economic Perspectives; Stiglitz, Joseph E. (2009). "Regulation and Failure," in David Moss and John Cistern- ino (eds.), New Perspectives on Regulation; Zerbe Jr., Richard O. and McCurdy, Howard E. (1999). "The Failure of Mar- ket Failure," Journal of Policy Analysis and Management. Creating investment-friendly environments requires certain activities. Many scholars have argued that markets remain the most efficient way to coordinate transactions in societies simply because of their decentralized nature and the self-interest of the actors involved. (McMillan J. 2002). Markets often do not work properly for a range of reasons. For some goods, like public goods, for example health, and merit goods, for example education, markets tend to not form spontaneously due to lack of incentives, requiring the public sector to step in and / or provide the incentives. In other cases, information is not available or only available to some market participants, hence markets do not form or work efficiently; prop- erty right are not sufficiently protected, side-effects on third parties are not accounted for (externalities) or not priced in, competition in the market is not fostered, in particular in markets that tend to establish natural monopolies, and market participants don’t trust each other due to lack of common value or low level of enforcement of the law. Institutions are required in such cases to address these “market failures”, such as regulatory institutions to 23 | P a g e establish and enforce anti-trust, competition laws, and transactions laws; develop suitable trade policies and facilitation regimes; rate the creditworthiness of market participants (companies and individuals); and enable a transparent management of property right in- cluding through land or collateral registers (Cunningham 2009). The literature suggests a phased approach in addressing market constraints, from fixing market failures to enhancing how underperforming markets work. Panayotuo (2000) lays out a three-stage concept: a. Nascent countries with little to no experience in crowding-in the private sector are likely to focus on putting in place the basic legal and regulatory framework for a mar- ket-based economy. It is likely at this stage that market failures need to be addressed, for example, by creating the institutions and the legal basis for competition through anti-trust laws. b. Slightly more developed economies with still immature or underperforming markets are then concerned about broadening the scope of private entry and competition. While these countries have the basic ingredients for a market economy in place, they may have to overhaul their regularly framework to ensure efficiency and transparency. c. Eventually, consolidating and already developed markets may want to test broadening private sector participation beyond the core infrastructure sectors to sectors where regulatory issues may be more difficult. Such countries are more concerned about deepening the involvement of the private sector within and across sectors. To this end, they focus on investments that allow to scale up the private sector involvement, enable integration into domestic and international value chains, upgrade skills/standards across supply chains, foster managerial and process innovation, improved govern- ance, and enhance competition. d. 26 See also (Lin, J. Y. 2011). Such a phased approach is also suggested by a joint report of 31 multilateral and bi- lateral development finance institutions and is reflected in IFC’s latest strategy (IFC 2011, IFC 2017). 27 Countries with limited experience in working with the private sector, such as those with structurally weak economies, FCS’ and LDCs are likely to face the greatest dif- ficulties. These countries not only have the greatest funding gap, they also face the greatest difficulties in attracting investors due to barriers to investments, lack of or insufficiently de- veloped regulatory frameworks, a business environment that is not conducive to foreign in- vestments, or weak financial infrastructure. The need to make special efforts in LDCs and FCS’ is therefore represented not only in international policy frameworks, but also in more concrete action plans of MDBs as well as of specific investment targets for the WBG. THE DEVELOPMENT COMMUNITY’S RESPONSE The WBG’s response is embedded in a broader global initiative of mobiliz- ing private finance and enhancing the private sector’s role as operator and service 24 | P a g e provider. The international development community has developed principles and strategies to crowd-in the private sector in the pursuit of the SDGs since their adop- tion in 2015: The UNCTAD Action Plan for Private Investment in the SDGs contains a range of policy options to respond to the mobilization, channeling and impact chal- lenges (UNCTD 2015). In a similar vein, based on the Addis Ababa Agenda for Ac- tion, MDBs embarked on an effort to help countries maximize finance for develop- ment, as expressed in the Joint MDB Statement of Ambitions for Crowding in Private Finance. 28 Enhancing domestic revenue mobilization through, for example, predictable and efficient taxation as well as improving public spending are important ingredients to these frameworks. However, the emphasis of these frameworks is on mobilizing private finance, and enhancing the private sector’s role as operator and service provider across a range of sec- tors, including infrastructure, agriculture, finance, climate change, education, health, logis- tics and manufacturing. More specifically, MDBs commit to increasing overall private sector mobilization by 25-35 percent over the following three years. To crowd-in the private sector the international development community adopted a set of principles (Box B.2). Box B.2. The “Hamburg Principles” to Crowd-in the Private Sector for Growth and Sustainable Development The G-20 “Principles of MDBs Strategy for Crowding-in Private Sector Finance for Growth and Sustainable Development”, the so-called “Hamburg Principles” set out six principles (G-20, 2017). (i) Recognizing the pri- macy of country ownership so countries remain fully responsible for the support they seek from MDBs. (ii) Creating an investment-friendly environment through upstream reforms, capacity and institution building, pipe- line development and financial and capital market development. One of the most frequent constraints faced by potential investors is the lack of concrete proposals of impactful and bankable projects. Hence, developing standardized and wholesale solutions to reduce transaction costs for private investments and create a pipeline of commercially viable projects is needed as well. (iii) To enhance opportunities for the private sector for public objectives and lower income countries, the G-20 adapted as third principle the expansion and standardization of credit enhancement through guarantees, insurance products, blended finance, equity and liquidity finance backup facilities. (iv) To contribute to the optimal use of scarce public resources, commercial funding would be prioritized, per the G-20; and (v) deployment of blended concessional resources and private capital to close the gap between economic/social and commercial returns. These principles would be implemented pending a reg- ular (iv) review of incentives for crowding-in private sector resources. Source: G-20 (2017). 25 | P a g e Attachment C RELEVANT LESSONS FROM PAST IEG EVALUATIONS IEG conducted a preliminary review of its own major evaluations to uncover lessons of experience regarding the creating markets and cascade approaches. During the approach paper preparation, IEG reviewed twenty-one major evaluations with at least one recommen- dation related to private sector participation, creating markets, or the cascade approach. These evaluations cover a wide range of topics that can be clustered into business enabling environment, financial sector, and real sector evaluations (Figure C.1). Figure C.1. Coverage of IEG Major Evaluations clustered by topic area Source: IEG. Note: This figure shows a subset of the 21 reviewed evaluations plus an additional subset of evaluations that will be re- viewed during the evaluation (as they were delivered recently). Lessons from this review informed the design of the evaluation’s theory of change and evaluation questions, highlighting areas where previous experience provide rich les- sons for future program implementation and where knowledge gaps remain. By uncover- ing lessons on the constraints and opportunities faced by the Bank Group in supporting pri- vate sector participation across a wide range of sectors, this review informed the left-hand side of the evaluation’s theory of change – i.e. that which relates to the needed activities and immediate outcomes. These lessons can be categorized along two broad areas, namely (i) on how the Bank Group (and the broader DFI community) can organize itself to deliver inter- ventions that together help leverage the private sector for sustainable development and growth and (ii) on the challenges and opportunities in delivering such interventions (that support the enabling environment and that deal with direct investments). By contrast, this 26 | P a g e review revealed that knowledge gaps remain with regards to the right-hand side of the the- ory of change – i.e., that which relates to the dimensions of creating markets – i.e., public sector capacity, policies, and frameworks, enhanced skills and governance, demonstration and innovation, and increased competition and enhanced integration. Figure C.2. Lessons map: a preliminary lessons mining exercise of IEG major evaluations Delivering investments: Organizing to deliver PSP interventions Supporting the enabling environment structuring and finance Assessing the Country readiness through Efforts to reform the relevant sectors and Integration of PSP and PPP pipeline into manage public finance and to develop a High quality structuring advice essential realistic project timeframes and focused Institutions and capacity is essential to Coordination with other MDBs / IFIs as Government with clear objectives and to introduce the legal basis for PSP / National Infrastructure Plans (NIPs) Competition, market structure, and Staying engaged beyond financial The scope / width of reform efforts well as across WBG is essential Early stakeholder involvement Poverty-Centered Intervention Assessing fiscal implications Choice of WBG instrument a PSP / PPP diagnostics Transparent procedures Incentives for WBG staff cost-recovery tariffs investment climate Conflict of Interest strong leadership Sponsor quality project design Strategy closure matter M&E Competitiveness and Jobs 2017      Support to Electricity Access 2016        WB Lending for Financial Inclusion 2016       Capital Markets 2016       Support to Health Financing 2015     Public Private Partnerships 2015                       Poverty Focus of Country Programs 2015     WBG Assistance to Low-Income FCS 2014         SMEs 2014    Innovation & entrepreneurship 2014      Sustained Transport 2014       WBG Support to ICT 2013          Managing Forest Resources 2013       IFC's Global Trade Finance Program 2013      IFC's Poverty Focus and Results 2013     Investment Climate Reforms 2013     Youth Employment 2013      Agriculture and Agribusiness 2011     Health Nutrition and Population 2009      Environmental Sustainability 2009      Doing Business 2009      Source: IEG. This review identified several broad areas of interest for this evaluation. These ar- eas relate to the delivery challenges of WBG support, encompassing coordination within the Bank Group (IBRD, IFC, and MIGA) as well as coordination and alignment with other MDBs and development partners. These lessons point at the need for MDBs and IFIs to de- liver an integrated and comprehensive “solutions package”, posing questions of sequencing, standardization of products and services, and leveraging of synergies across the delivery chain (upstream policy advice, AAA, advisory services, sector reform efforts, downstream lending and investments). But also, questions of incentives and risk management surfaced 27 | P a g e to what extent these are fit for the challenge of creating markets. The need for putting in place the right enabling environment, through sector reform measures and the introduction of the legal basis for private sector participation. A sustained engagement across the WBG appears warranted given that such reforms often take longer than WBG’s project cycles. Such efforts to create markets, a more likely to succeed when government have clear objec- tive and WBG can rely on a government champion that is committed to the needed reforms. Past evaluation has not sufficiently covered aspects of risk management, i.e., the capacity of WBG institutions to manage risk, their risk appetite, underlying processes and incentive sys- tems that guide the existing risk management. Likewise, the extent to which current WBG analytical tools are fit for the purpose has not yet been holistically assessed. These prelimi- nary lessons mining effort was summarized along a lessons map that aligns lessons to spe- cific evaluations which reveals that coordination and sector reform are the two (Figure C.2 above). The subsequent paragraphs summarize each of these lesson areas in more detail. Lessons around how the Bank Group (and broader DFI community) can organize itself to deliver interventions focused on the following aspects: • Use of diagnostics to make strategic choices: assessing the country’s readiness through analytics and diagnostics (e.g. PSD diagnostics) was described as a vital com- ponent in ensuring the Bank Group makes informed strategic choices. Such assess- ments can help the Bank Group identify sectors or circumstances where engaging the private sector is feasible and determine the role of the Bank Group in this context (in- cluding synergies in program delivery). Diagnostics were also described as important in helping assess whether It was also described as important is assessing whether long-term finance is available to help limit the uncertainty of project finance. • Selection of WBG instrument should fit country context: the review highlighted the importance of selecting the right Bank Group instrument (e.g. Bank DPOs vs IPFs, IFC Advisory vs. Investment) to achieve the desired result, considering the country’s read- iness as per the diagnostic work. • Focus of engagement should cover inclusiveness and poverty concerns: engage- ments that support private sector participation should have a clear intention to pro- vide services in an inclusive and poverty-focused manner and should therefore take into considerations concerns such as affordability and geographic reach. • Coordination within the Bank Group and with other development partners should deliver synergies: coordination, both internal and with other development partners was described as essential for project finance and resource mobilization. The Bank Group was appreciated for its capacity to bring “solution packages” described as the most comprehensive when comparing with other MDBs; hence internal coordination to deliver these solutions is also critical. • Incentives for Bank Group staff are needed: despite the Bank Group’s reputation delivering effective “solution packages,” the review found that improving the incen- tives structure for Bank Group staff would be critical to ensure team leads and staff 28 | P a g e engage in time-consuming PSP projects and to ensure Bank Group coordination is achieved over time. Incentives were also described as playing a significant role in en- couraging smart risk-taking and in pursuing more complex or riskier PSP projects and programs. • Managing conflict of interest: the review identified conflict of interest as an area that needs to be managed carefully with upstream advice and transaction-level advice and finance. • Monitoring and evaluation: ensuring PSP engagements are equipped with smart M&E systems that capture poverty and inclusion aspects and that last beyond finan- cial or program closure to ensure feedback loops are maintained and results are com- municated down the line. Lessons on interventions that support the enabling environment can be summa- rized as follows: • Governments with clear objectives and strong leadership: strong and long-term gov- ernment commitment to reform is a pre-requisite for success in reform efforts; govern- ment champions were found to be essential; by contrast, political uncertainty and power shifts or shift of government priorities lead to delays or non-implementation of reform efforts or enhanced private sector involvement • Competition, market structure and investment climate: in cases where SOEs were dominant, this slowed down sector reform and attempts to roll out a private sector participation agenda; thus, privatization of SOEs was often described as a precursor to encouraging the private sector to engage in a sector; notable features of such reform include restructuring, financial soundness, and corporatization of public utilities • Institutions and capacity essential to manage public finance and to develop a pipe- line of bankable projects: capacity is crucial so that potential deals get adequately assessed, controlled, budgeted and disclosed, including management of contingent liability. Building capacity and enhancing financial and technical skills at the country level is crucial so country agencies can develop a pipeline of bankable projects where private sector can invest, not limited to the PPP space, but including supply chain, manufacturing, agribusiness; such skills include the development of essential prepar- atory analysis (e.g. VfM, risk analysis, etc.) • Transparent procedures and responsibilities, including the public and private sec- tor side: For project preparation, bidding and award were described as required; standardization of transactional documents, procedures, and contracts increased effi- ciency and often delivered spill-over effects. • Sector reform and legal basis: sector reform efforts potentially take longer than an- ticipated, are complex, and often fail due to political pressures and other reasons; however, experience shows that launching a PPP agenda need not wait for the “per- fect regulation” in place; the existence of legal basis to allow the private sector to en- gage and defined process coupled with institutional responsibilities is a prerequisite 29 | P a g e • Scope and width of reform efforts: narrowly focused reform efforts tend to better achieve their objectives while complex or broad reform efforts tend to fail • Early stakeholder involvement: engagement with the civil society, raising public awareness about the pros and cons of private sector involvement are essential to achieve a consensus and ensure buy-in from all sides • Integration of PSP pipeline into National Infrastructure Plans: such plans are essen- tial to align public investment priorities and reduce uncertainty about government commitment Lessons on delivering interventions (structuring and financing) can be summa- rized along the following lines: • High quality structuring advice: such advice was found to be essential to turn deals into sustainable projects, balance private sector revenue expectations and public ob- jectives, optimize risk allocation across parties involved in the deal, and minimizing revenue fluctuations (for example, through sound off-take agreements and PRI) • Challenge with cost-recovery tariffs: cost-recovery was problematic and therefore suggests that, in cases where tariffs cannot cover costs, a system that covers costs for the private operator be put in place. Cross-subsidization may help improve the reach of operations into areas where users would not be able to afford PPP tariffs (from rev- enues generated from the more affluent users); but such systems were delicate and their success hinges upon public and private sector actors adhering to their commit- ments. • Assessing fiscal implications: in such assessments, those which deal with contingent liabilities and where government guarantees are needed, it is essential to avoid that they are treated as “off-balance sheet.” • Realistic project timeframes and focused project design: such realism proved to be beneficial for positive outcomes, for World Bank investment projects that contained a PPP finance component • Sponsor quality: high sponsor quality with technical, operational and country-spe- cific know how was described as important • Staying engaged beyond financial closure: to ensure the sustainability of outcomes, it is important to stay engaged beyond financial closure to assist with mid-course cor- rections and re-negotiations and to monitor longer term outcomes. Such engagement would also help assess demonstration effects as well as broader sector effects (includ- ing in the enabling environment because of such investments) 30 | P a g e Attachment D METHODOLOGICAL APPROACH, EVALUATION DESIGN AND DESIGN MATRIX The evaluation design benefited from valuable interactions with stakeholders and subject-matter experts and from a careful review of the WBG’s recent strategic documents that reference the Cascade model and Creating Markets concept. During the early phases of the review, IEG interacted with World Bank Group strategy teams and with staff working on public private partnership areas and on priority sectors such as infrastructure (energy, transport, ICT), finance, and agribusiness. These interactions, together with a review of rele- vant literature and the most recently published strategies – Forward Look (WBG, 2017) and Creating Markets and Mobilizing Private Capital (IFC, 2017) – informed the evaluation ap- proach by highlighting important concepts and frameworks and by revealing industry cod- ing, system flags, and keywords that would facilitate the design of the analytical framework and the selection of evaluation methods. Reflecting on the multi-dimensional nature of the evaluation subject, leveraging the private sector for growth and sustainable development, the analysis will be carried out at multiple levels. The analysis will cover multiple levels across the evaluation’s dimen- sions of creating markets, namely: (i) public sector capacity, policies, and frameworks, (ii) enhancing skills and governance (firm-level), (iii) demonstration and innovation, and (iv) increasing competition and enhancing integration. It will also cover multiple sectors across a three-stage maturity model (see Figure D.1). Figure D.1. Stylization of the evaluation’s multi-level analytical framework Sources: IEG Review and interviews with World Bank Group subject-matter experts and management. The evaluation will adopt a theory-driven approach to analyze the causal steps identified in the intervention logic. 29 The underlying theory will be developed by review- ing the available literature on private sector participation and will be complemented by semi-structured interviews with internal and external experts and a review of project- and country-level documentation. This approach will allow the evaluation to open the “black box” between intervention and outcome to provide information on whether the program 31 | P a g e succeeded and on “how” and “why” it did so, with a view of improving future program ef- fectiveness. This approach should result in an improved program theory that can be incor- porated into the broader body of knowledge. Underpinning the analytical framework is a sequenced mixed-methods approach. The key methodological components include sector-focused case studies, literature and doc- ument reviews, semi-structured interviews, and data analysis, following the below outlined sequence (see Figure D.2). In the initial stages of the evaluation, three components will be carried out in parallel – i.e. the initial stakeholder interviews, preliminary literature review, and exploratory data analysis (external data and portfolio). Once these three components have been completed, they will inform the second stage of the review – i.e. the case selec- tion, design, and analysis, and the continued stakeholder engagement and data analysis. Figure D.2. Stylization of the evaluation’s sequenced mixed-methods approach Source: IEG Review and interviews with World Bank Group subject-matter experts and management Sector-focused case studies The evaluation will conduct up to 15 case studies in countries where the World Bank Group has delivered a program of support to leverage the private sector in selected sectors. These cases will focus on three focal sectors, equally covering various maturity stages of the underlying economy or sector (nascent, immature, and developed). Up to nine of these 15 cases will be supported by field mission with the remaining being conducted from headquarters, based on available documentation and market data, complemented by interviews with WBG staff and counterparts, as needed. Given that cases are defined as a sector within a country, one country may therefore be selected for one or more cases. For ex- ample, country “A” may have a nascent energy sector and an immature finance sector and therefore both may be selected for case study under this framework. Sectors of relevance for the pursuit of SDGs with the highest potential for private sector participation are: power/energy, agriculture (as one of the enablers for food security), 32 | P a g e telecommunication, transport, and finance, followed by health, water and education with a somewhat diminished potential for private sector participation (UNCTAD 2014 and 215, see Figure B.2. in Annex B). Cases will focus on three of these sectors, with the choice being made based on these considerations: (i) achieving a balance across IFC’s focus industries, per IFC’s Strategy 3.0. (IFC 2017), (ii) providing a strong link to the pursuit to the SDGs, (iii) addressing at least one infrastructure sectors to be able to study the regulatory challenges associated with these cases, while (iv) covering both, cases that lend themselves to the appli- cation of the Cascade Approach which offers opportunities to study the WBG-wide dimen- sion of creating markets, as well as cases of IFC standalone investments. These cases will take a comprehensive approach, i.e. will consider the “eco-sys- tem” needed to develop markets. Each case will therefore cover WBG’s support to setting up a favorable business environment in general, through interventions in the space of com- petitiveness, investment climate, doing business, trade and FDI policies, institutions build- ing, macro policies etc. In addition, each case will cover the role of important and relevant input factors, i.e. access to finance and access to physical infrastructure and hence access to markets. In doing so, case will consider, in addition to systemic factors mentioned above, also consumer needs, beneficiary effects, other market participants as well as other IFIs / MDBs. Case will focus on one of three focal sectors which will help deriving lessons relevant to market creation in SDG-relevant areas. Within this sector the cases will look at the sector specific interventions to build a conducive enabling environment through, e.g. putting in place ICT relevant regulatory framework, oversight, unbundling, SOE restructuring, PPP policies, or purchasing power agreement (PPA) standards for the energy sector etc. And very importantly, each case will look at specific transactions (lending, investments, guaran- tees) to the extent they materialized and evaluation their contribution to market creation. Cases will focus on deriving lessons on what works regarding creating markets, in ar- eas where WBG has been active. Internal and external validity will be enhanced by conduct- ing in total 15 cases across three different sectors and three different maturity level and com- parison across these cases. The selected cases will be used to generate insights into which sectors would benefit from market creation in each country and derive lessons on how the various policy areas inter-link. The case approach will also allow for insights whether WBG has missed to create markets in the given countries; but the proposed case design does not lend itself to comprehensively assess the extent to which the WBG has missed opportunities in creating markets as this would require a comprehensive capture of all creating markets activities which, for reasons outlines above, is not possible. Case selection and design will be informed by the “lessons mining exercise” (at- tachment C) of relevant literature to establish a logic that would allow for generalizabil- ity across the levels depicted in the evaluation’s analytical framework. A preliminary re- view of IEG major evaluations and of relevant literature was carried out during the 33 | P a g e preparation of this approach paper. This review helped identify patterns of regularity of what works across different sectors and issue areas. Findings from this review were used to develop the evaluation questions and will be used to develop the case protocols for compar- ative analysis. By using carefully constructed case protocols, the evaluation will be able to test findings against the established logic and to compare them across sectors, maturity stages, and dimensions of creating markets. In addition, cases will look at the country’s ex- perience beyond the WBG and will include support provided to the country by other MDBs, donors, and the private sector. Specifically, the Bank Group’s engagement will be assessed at three levels: • Country: covering context and overarching enabling environment (e.g. macro condi- tions, quality of institutions and regulations, depth and quality of financial markets); country priorities; and WBG response at a strategic level (e.g. SCDs, CPFs, country- level ASA) • Sector: covers history of WBG engagement in the sector and closely related enabling conditions (e.g. PPP regulations, financial sector regulations, agri-sector reform) • Dimensions of creating markets: in-depth review of interventions within the coun- try/sector portfolio leading to the dimensions of creating markets to better under- stand their effectiveness and the factors that facilitated or constrained their implemen- tation Case selection will be systematic but purposeful. The following selection model will ensure the evaluation adequately balances the tradeoffs between depth and breadth of analysis while making sure the cases are selected in a systematic and transparent manner. For more on the case selection methodology, please see “Case Selection Methodology” be- low. • Using external data, the evaluation will classify countries based on their sector’s ma- turity – nascent, immature, and developed. • Once sectors are classified, the evaluation will examine the WBG’s portfolio of support in this sector to identify a short-list of countries that meet the evaluation’s selection criteria. These criteria may include WBG support along the intervention logic (either as stand-alone projects or as part of a programmatic approach) with interventions across the dimensions of creating markets. • Other criteria include: (i) Support of WBG institutions in putting in place the enabling environment and (ii) Presence of evaluation documentation (ICRs, XPSRs, PCRs, PERs). • The evaluation team will also consult with internal and external experts to identify the countries that offer the richest opportunities for learning. The evaluation’s case design and case selection criteria will allow learning from both the common case as well as the critical case. Each case is intended to contain one 34 | P a g e country/sector where the WBG provided programmatic support in a key sector at a certain maturity stage and the selection process thereby facilitating the drawing of lessons within and across sectors. The potential for horizontal comparative case analysis (across maturity stages) is expected to yield more valid and robust lessons. To facilitate comparison across sectors and maturity stages, the evaluation will employ the same data collection methods and protocols in all cases. These methods in- clude: (i) a review of literature on private sector participation in the sector, (ii) a review of literature on the country’s enabling conditions for private sector participation, (iii) a detailed review of WBG country strategies, diagnostics, and relevant analytical works, (iv) a detailed desk review of WBG’s portfolio of support across the three levels (enabling conditions and target sector reform and structuring and finance), and (v) semi-structured interview with project and non-project stakeholders (i.e., government, MDBs, private sector, NGO/CSOs, academics). In cases where quality data is available, case authors may use such data for analysis of relevance or to test the program’s effectiveness. Figure D.3. depicts the expected data collection method for each level of case review. Figure D.3. Data collection methods for each of the case levels Lit. on enabling Country & Semi-structured Level conditions for WBG Strate- PRA Data analysis interviews PSP gies • Country ✓ ✓ ✓ • Sector ✓ ✓ ✓ ✓ ✓ • Dimensions of creating mar- ✓ ✓ ✓ kets Source: IEG Portfolio Review and Analysis (PRA) The evaluation will carry out a two-stage PRA of projects within the selected cases. The first stage will aim to identify the key characteristics of the portfolio across the two ana- lytical levels (enabling environment and reform and structuring and finance) to facilitate cross-sector and cross-maturity comparison of the portfolio characteristics. The second stage will involve in-depth content analysis of purposefully selected portfolio subsets (e.g. to un- derstand how a specific intervention works or how a certain factor drives success or failure and to understand country experiences through clusters of projects and their co-portfolios) using custom protocols and relying on semi-structured interviews and project-level docu- mentation (e.g. PADs, Board Reports, ICRs, ICRRs, XPSRs, PCRs, and their EvNotes). World 35 | P a g e Bank ASA will be reviewed in this context on a purposive basis and using a simplified ver- sion of the PRA review protocol. Literature Reviews The early stage of the evaluation will employ a structured review of relevant (inter- nal and academic) literature on leveraging the private sector for sustainable development and growth across each of the selected sectors. The objective is to understand the character- istics of this support and the role of complementary or sequential interventions that may in- fluence its impact (e.g. role of the capital markets or investment climate). The review aims at generating insights in this regard and is intended to provide the theoretical basis for the evaluation to establish causal links between policies in support of PSP in the sector and to formulate the models adopted to validate the causal relationship of the WBG portfolio in leveraging the private sector to promote sustainable development and growth. Semi-structured interviews The evaluation team will carry out semi-structured interviews throughout the evalu- ation’s lifecycle. At an early stage, the evaluation will carry out such interviews with a view of better understanding the underlying theory, getting to know the institutional priorities (past, present, and future), and developing a set of preliminary hypotheses. During case studies, the team will carry out semi-structured interviews to gain deeper understanding of the program’s features, its effectiveness, and lessons on what works. For each set of inter- views, the evaluation team will develop an interview guide to ensure key questions are asked consistently across interviews while maintaining the flexibility needed to follow topi- cal trajectories that stray from the guide where appropriate. A wide range of stakeholders will be identified for interview as part of the early stage theory building exercise and in case studies; these include, among others, WBG staff at headquarters and in the field, govern- ment agencies, multilaterals, donors, non-governmental agencies, civil society, academics, and private sector entities. Lessons Mining in IEG Major Evaluations The evaluation will carry out a systematic desk review of IEG’s portfolio of Major Evaluations with a focus on (i) the enabling environment, (ii) financial markets, and (iii) spe- cific sectors. A qualitative meta-analysis, the review will identify and classify the factors identified by these evaluations as facilitating or constraining the implementation of their in- terventions. The preliminary results of this exercise a summarized in Annex C and helped the design of the evaluation questions. 36 | P a g e Review of Databases and Indicators The evaluation will identify and utilize indicators aligned with the evaluation questions and selected sectors to identify sector priorities and changes over time. Indica- tors will be selected from data warehouses such as WDI (WB) and datasets such as Infra- scope from the Economist Intelligence Unit (EIU), Global Competitiveness Index from World Economic Forum (GCI-WEF), Doing Business from the World Bank (WB-DB), Coun- try Policy and Institutional Assessment from the World Bank (WB-CPIA), among others (for more details please see Figure A.4 below). Portfolio data will be mapped against these indi- cators. Design Limitations Several factors might constrain the evaluation; these fall broadly into two catego- ries: limitations from conscious choices about scope and from the availability and quality of existing data and documentation. To manage the tradeoff between breadth and depth of analysis, the evaluation approach makes the necessary choice of focusing the analysis on three key sectors and of selecting a case-based (and portfolio-within-case), acknowledging that this approach will have limitations about the extrapolation of results. This choice was informed by the literature and by initial stage stakeholder consultations. Data and docu- mentation constrains will include, among others: (i) identification of cases will rely on exter- nal data (which may have caveats of its own and may not be complete for the full range of countries) and on WBG portfolio coding systems (which are not always accurate) and (ii) strategy- and project-level documentation which is not always available nor is it always con- sistent. In addition, though all lending operations in the World Bank are subject to self-eval- uation and IEG validation, IFC Investments and IFC Advisory are evaluated on a sample ba- sis (approximately half of the population). Therefore, the size of the evaluated portfolio for these will be smaller than for World Bank lending operations. Case Selection Methodology IEG’s case and portfolio-within-case selection methodology relied on the use of external data and Bank Group portfolio summaries aligned with the evaluation’s multi- level analytical framework. As described in the earlier section on case methodology, the se- lection of the 15 case studies will be based on the use of external data, the review of WBG strategy, sector, and project-level documentation, and on interactions with internal and ex- ternal subject-matter experts. External data will enable the evaluation to classify countries into maturity stages as per the multi-level analytical framework – i.e. nascent, immature, and developed. The evaluation will use static data (latest available) to create these categories. For example, the 37 | P a g e EIU’s Infrascope and its sub-indicators may be used as a proxy for a country’s readiness for private sector participation while the World Bank’s Systems Approach for Better Education Results (SABER) and its sub-indicators may be used to segment countries based on the share of students enrolled in primary or tertiary private institutions. In addition, and to the extent that data is available, the evaluation will use time series data to visualize how a country and sector’s trajectory has changed over time. Such visualizations will give the team a view of which countries and sectors may provide better opportunity for learning; this may be whether countries have moved upwards or downwards along the maturity stage or whether they have remained stagnant over time. Figure D.6 provides details of this strategy. Figure D.4. Data as an identification strategy: selected indicators Sources: IEG Review and interviews with World Bank Group subject-matter experts and management The case selection methodology will also examine the WBG’s portfolio within coun- tries to better understand the evolution of the portfolio for the selected sector. To identify the relevant portfolio within country, IEG will employ the following steps: (i) retrieve pro- jects using WBG systems and their codes (e.g. sector, thematic, industry, or product codes), (ii) manually review these portfolio subsets to systematically categorize projects and de- velop a unified picture of their features and characteristics. Cases will cover countries with WBG-wide engagement, reflecting the nature of the Cascade approach. This suggests cases where at least two of the three WBG institutions pro- vided support or where WBG support was delivered upstream (to enhance policy or regula- tory frameworks) and downstream (in form of investments), regardless of the institution. In addition, the study will look at cases where WBG provided only upstream or downstream support, should these offer particularly valuable lessons, including cases with IFC standalone projects. For more on the system codes used to identify the portfolios-within- countries, see Figure D.5. 38 | P a g e To finalize the selection, the evaluation team will consult with internal and exter- nal experts to identify the countries that offer the richest opportunities for learning. By interacting with the sector’s portfolio and with subject matter experts, the evaluation will also ensure the cases selected provide the richest opportunity for learning. Figure D.5. World Bank Group system codes used to identify the portfolios-within-countries World Bank Lending and Advisory (ASA) Source: OPCS Sector and Theme Codes: http://www.worldbank.org/projects/sector?lang=en&page= http://www.worldbank.org/projects/theme?lang=en&page= File from World Bank Business Intelligence and Analysis for Office (BI / AO) Sector and Agribusiness: AB, AJ, AH, AF, AT, AI, AL, AZ, AK, YA, YB Theme Codes Education: EC, EF, EL, EP, ES, ET, EW, EZ, 65 Energy: LT, LN, LZ, LD, LP, LE, LB, LI, LH, LU, LW, 86, 83 Finance: FA, FD, FE, FF, FK, FL, FP, FS, FY, 31, 32, 33 ICT: CB, CC, CD, CF, CI, CS, CZ, 26 Notes: Cut-off point: project can have multiple sector and theme codes. Each sector and theme code is assigned a percentage from 1-100. Projects were tagged if they contained at least 20 percent of the relevant code. IFC Investment and Advisory Source: File from iDesk (MIS Extract) and ASOP (Project Product Detailed Listing). Sectors Agriculture: Agriculture and Forestry (A-A, A-B, A-C, A-D) Education: Education Services (T-AA through T-AC) Energy: Electric Power (V-A through V-I) and Utilities (C-A and C-C) Finance: Finance & Insurance (O-AA through O-MG) Collective Investment Vehicles (P-BA through P-GJ) ICT: Information (N-AA through N-CA) Industries Agribusiness: Agribusiness & Forestry (Investment Education: Education (Basic Education, Higher Education, Other Education) only) Energy: Infrastructure (Electric Power and Utilities- District Heating and Cooling) Finance: Financial Markets, Funds, Collective Investment Vehicles, Payments & FinTech, Venture Investing ICT: Telecom, Media, and Technology MIGA Source: MIGA Sector Codes: https://www.miga.org/Pages/Projects/AdvSearch.aspx Sectors: Agribusiness: Agribusiness Energy: Power Finance: Banking, Capital Markets, Financial Markets, Financial Services, Leasing ICT: Telecommunications Sources: IEG Review and interviews with World Bank Group subject-matter experts and management Evaluation design by evaluation questions The table below aligns evaluation questions with the evaluation design. Each check mark represents the strength of the method to answer the questions; i.e., one check mark suggests that the method will provide some data to answer the evaluation question while three check marks suggest the method is expected to provide a great deal of data to answer the evaluation question. 39 | P a g e Semi- Sector Litera- Data Lessons struc- QUESTIONS case PRA ture analy- mining tured in- studies review sis terviews 1. Tailoring interventions to countries and sectors and underpinning analytics • What analytical tools are available to the WBG to design and prioritize its interven- tions so they reflect a clear understanding of the market gaps and failures? What ✓✓✓ ✓ ✓ ✓✓ ✓✓✓ works about identifying and addressing such market gaps? • Are the current analytical tools appropriate and sufficient to develop country strategies ✓✓✓ ✓ ✓✓✓ and programs? 2. Putting in place the public-sector capacity, policies and frameworks -- WBG policy and up- stream support • What is the evidence that WBG contributed to the identification of bankable projects for the private sector to invest or has contrib- uted to building institutions and capacity so ✓✓✓ ✓ ✓ ✓✓✓ that the country could identify such a pipe- line? Were such institutional and capacity building efforts sustainable? • What can we learn from WBG policy up- stream support about its potential to (i) tar- get market gaps and failures and what from ✓✓ ✓✓✓ ✓ ✓ ✓✓ ✓✓✓ cases where this was not the case; and (ii) ✓ engage in the private sector as financier, operator or innovator? 3. Finance, demonstration, innovation, competition and integration -- WBG downstream work • What is the evidence that WBG financed projects lead to innovation and what were the driving factors enabling such innovation, ✓✓✓ ✓ ✓✓ ✓✓ ✓✓✓ including new products, services, business process and/ or technological innovation? • Have WBG financed projects had a pio- neering character and subsequently in- duced demonstration and replication? What ✓✓✓ ✓ ✓✓ ✓✓ ✓✓✓ were the factors that allowed these projects to contribute to such demonstration effects? • What is the evidence that WBG financed projects lead to enhanced skills and gov- ernance at the firm level and what were the ✓✓✓ ✓✓ ✓✓✓ drivers of success? 40 | P a g e Semi- Sector Litera- Data Lessons struc- QUESTIONS case PRA ture analy- mining tured in- studies review sis terviews • What is the evidence that supported com- panies and operators provide services in a sustainable, reliable and affordable manner, ✓✓✓ ✓✓ ✓✓✓ including for the Base of the Pyramid? • Ultimately, what is the evidence that WBG financed projects contributed to increased competition and integration, thus creating markets and/or making these more effi- cient? What were the factors that allowed to ✓✓ enhance competition and integration and ✓✓✓ ✓ ✓ ✓✓✓ how could such experience be applied else- ✓ where? • Were such created markets resilient, inclu- sive and fostering environmental sustaina- ble practices? 4. The delivery challenge – factors that determine the success of implementing the creating market approach • What lessons can we draw from comple- mentary upstream and downstream ap- proaches about WBG coordination, col- laboration and leveraging of synergies? Was the sequencing of WBG support or ✓✓✓ ✓ ✓ ✓✓✓ ✓✓ interventions ad hoc or intentional and at what stage of planning or implementation was it introduced? How were conflicts of interest addressed • How has the WBG considered other MDB strategies and operations from conceptu- alization to implementation and how did ✓✓✓ ✓ ✓✓ this impact results? What factors contrib- uted to or prevented such coordination? • Does the WBG have the right processes, tools and incentives in place to engage in interventions with market creation poten- tial and do these incentives foster respon- ✓✓✓ ✓ ✓ ✓✓✓ sible risk taking and is it commensurate with the WBG objectives? 41 | P a g e Endnotes 1 For more details, see Attachment A. 2 These projections seem to consider expected demographic changes, i.e. population growth as well as changes in the distribution of age groups and resulting needs (Schmidt-Traub, G (2015). For example, in education, changes in the population of school age children over time is factored in. Likewise, in energy, where demand is driven by population and income growth and distribution. 3 This Approach Paper acknowledges the need to differentiate between countries, economies and markets. They can be at the same maturity level about opening and engaging with the private sector, or may be at various stages. Hence the AP refers to markets more specifically. 4 For example, investments in agribusinesses may provide the needed link within a supply chain or provide market access by processing specific agriculture produce that was not mar- ketable without this processing. 5The literature also makes a case of innovation-led growth. More advanced economies (OECD and EU countries, for examples) would be pursuing an innovation-led growth which would require public policies that aim to create markets, rather than just ‘fixing’ mar- ket failures; public agencies would not only ‘de-risked’ the private sector, but also lead the way in terms of shaping and creating new technological opportunities and market land- scapes. (Mazzucato 2015) 6 This evaluation therefore adopts a market maturity concept for the stratification of coun- tries to select cases of analysis. 7 Such a phased approach is not only suggested by the literature, but also reflected in a joint report of 31 multilateral and bilateral development finance institutions as well as in IFC’s latest strategy (Panayotuo 2000), (Lin, J. Y. 2011). (IFC 2011, IFC 2017) 8For an example of an indicators system that attempts to capture the various levels and di- mensions of market creation, see EBRD, retrieved Dec 2017: http://www.ebrd.com/cs/Sat- ellite?c=Content&cid=1395237866249&pagename=EBRD%2FContent%2FContentLayout 9 World Bank Group 2002b, pages 2, 3 and 7 10This AP uses the terminology regarding MFD, Cascade Approach and Creating Markets as per the referenced official WBG documents. Note that MFD is increasingly used by WBG staff in an inter-changeable manner with “Cascade Approach”. Note however as well that the official IFC strategy 3.0 contains the Cascade approach as an explicit tool to Create Mar- kets. The team is aware of the evolving nature of these concepts. 11This Approach Paper tries to use the notion of “creating markets” as reflected in the broader literatures. Regarding the WBG concept of MFD and the relationship of MFD with “creating markets”, the Approach Paper tries to reflect the relationship as described in WBG’s own documents (the Forward Look and IFC 3.0). The relationship of these two con- cepts is not always clear in WBG documents, however. It can be deduced safely from WBG 42 | P a g e documents that MFD and Creating markets are at least linked through the “Cascade Ap- proach”. According to the IFC’s latest strategy, “IFC 3.0 requires IBRD/IDA, IFC and MIGA to move from isolated examples of best practice to systematic, scaled-up collaboration. Cre- ating Markets demands an intensified focus on achieving structural reforms that facilitate private investment, address market and institutional failures, or strengthen the rules gov- erning competition.” Subsequently, the IFC 3.0 strategy introduces the Cascade approach as a tool to operationalize “creating markets” (see IFC 3.0, p. 10-11, Box 2). Likewise, the Cas- cade approach is central to the MFD concept (see Development Committee, Sept 17, 2017, p. 2, Box 1). Accordingly, “MFD complements the IFC’s strategy to “Create Markets” and MIGA’s 2020 strategy by strengthening regulatory or policy frameworks, promoting compe- tition, and achieving demonstration effects, […]”. One of the purposes of this evaluation is to bring clarity to these WBG / IFC concepts. 12Some IFC documents also refer to Creating Markets as “Emergence of yet undeveloped markets or game-changing expansion and upgrading of already exiting market structure”. Source: Creating Markets, IFC Presentation by CEDVP, Jan 24, 2018 Reference is made by IFC to the example: “Restoring Afghanistan’s Raisins Potential” 13 where the capacity increase of a raising processor would contribute to integration of the market. Source: Creating Markets, IFC Presentation by CEDVP, Jan 24, 2018 14 The reference document (WBG 2017) does not stipulate how many of these criteria ought to be met for a project to be classified as “having the potential to create markets“. 15 Embedded in the theory of change is also a metrics to assess market creation, akin to some of the measures currently proposed in the recently introduced IFC Anticipated Impact Measurement and Monitoring (AIMM) System (IFC 2017b). Immediate outcomes also mat- ter; these are measured by the extent to which WBG interventions could put in place sus- tainable public-sector capacities and policy frameworks, or by the extent to which private investments had a replication effect, spurred innovation and /or helped to enhance skills and governance at the firm level. The team will elaborate indicator systems during the im- plementation of the study that account for the specificity of context and case study. 16 The preliminary review of these 21 previous IEG evaluations identified several broad ar- eas of interest relevant for this evaluation. These 21 evaluations cover a range of sectors and areas, including infrastructure, financial sector and capital markets, trade, competitiveness, investment climate, innovation, PPPs, and health. Areas of interest relate to the delivery chal- lenges of WBG support, encompassing coordination within the Bank Group (IBRD, IFC and MIGA); and the coordination and alignment with other MDBs and development partners. These lessons point at the need for MDBs and IFIs to deliver an integrated and comprehen- sive “solutions package”, posing questions of sequencing, standardization of products and services, and leveraging of synergies across the delivery chain (upstream policy advice, AAA, advisory services, sector reform efforts, downstream lending and investments). The lessons also point at the extent in which current incentives and risk management practices are fit for the challenge of creating markets. The need to put in place the right enabling environ- ment, through sector reform measures and the introduction of the legal basis for private sec- tor participation. A sustained engagement across the WBG appears warranted given that such reforms often take longer than WBG’s project cycles. Such efforts to create markets, a more likely to succeed when government have clear objective and WBG can rely on a gov- ernment champion that is committed to the needed reforms. A lessons map in Annex C aligns 43 | P a g e lessons to specific evaluations. Past evaluations have not sufficiently covered aspects of risk management, i.e. the capacity of WBG institutions to manage risk, their risk appetite, under- lying processes and incentive systems that guide the existing risk management. Likewise, the extent to which current WBG analytical tools are “fit for purpose” has not yet been as- sessed in a comprehensive way. 17When assessing incentives, the team will consider also recent development and efforts im- proving incentives across the WBG even though they may not have been in place when the interventions were implemented. Conclusion will be eventually based on both, past inter- ventions, considering recent efforts to improve systems and processes. 18While the evaluation will not be able to assess the effects of market creation on the SDGs, the evaluation will cover linkages of relevant WBG interventions to the SDG by assessing relevant dimensions of the underlying interventions (access to services and quality of these, affordability, and inclusion/distributional considerations), to the extent that project level evaluation evidence contains such data or data can be obtained readily during the conduct of the case studies. 19Trust fund-supported interventions by the Public-Private Infrastructure Advisory Facility (PPIAF), the Global Infrastructure Facility (GIF) and InfraVenture will be covered to the ex- tent that evaluative evidence, validated by independent sources, is readily available as their interventions are not subject to regular project level evaluations. 20Note that the financial sector was not chosen as one of the three focal sectors as previous IEG evaluations have assessed many relevant aspects of capital markets and securitization, financial inclusion, SME finance and trade finance. 21Chen, 2012. "Purpose of theory-driven evaluation is not only to assess whether an inter- vention works or does not work, but also how and why it does so." He differentiates theory- driven evaluation from "black-box" which mainly assesses whether an intervention has an impact on outcomes and from "methods-driven" evaluation which uses a research method as a basis for conducting an evaluation. 22 Embedded case design: https://www.sagepub.com/sites/default/files/upm-bina- ries/41407_1.pdf 23Given that cases are defined as a sector within a country, one country may therefore be se- lected for one or more cases. Maturity levels will be used both for the economy in total, based on general indicators on competitiveness, doing business regulations and institutional capacity as well as for the sectors specifically, based on sector indicators on regulatory ro- bustness and institutional capacity. 24In addition to assessing the eco-system conducive to creating markets, each case will focus on one of the three focal sectors. This will help deriving lessons relevant to market creation in SDG-relevant areas. Within this sector the cases will look at the sector specific interven- tions to build a conducive enabling environment through, e.g. putting in place ICT relevant regulatory framework, oversight, unbundling, SOE restructuring, PPP policies, purchasing power agreement (PPA) standards etc. And very importantly, each case will look at specific transactions (lending, investments, guarantees) to the extent they materialized and evalua- tion their contribution to market creation. 44 | P a g e 25To utilize IEG’s budget in a cost-effective manner, mission-based case studies are likely to focus on clusters of interventions that follow the Cascade logic (involving upstream policy support and downstream investments) while desk-based studies are likely to focus on standalone interventions. Desk-based will cover a review of the needed documents, includ- ing project document and country strategies, associated analytical work, market data and interviews with WBG staff and counterparts. 26The literature also makes a case of innovation-led growth. More advanced economies (OECD and EU countries, for examples) would be pursuing an innovation-led growth which would require public policies that aim to create markets, rather than just ‘fixing’ mar- ket failures; public agencies would not only ‘de-risked’ the private sector, but also lead the way in terms of shaping and creating new technological opportunities and market land- scapes. (Mazzucato 2015). 27 This evaluation therefore adopts a market maturity concept for the stratification of coun- tries to select cases of analysis. http://www.bundesfinanzministerium.de/Content/DE/Downloads/G20-Dokumente/ 28 Hamburg_Genannte_Berichte/ Joint-MDB-Statement-of- Ambitions.pdf?__blob=publicationFile&v=1 29Chen, 2012. "Purpose of theory-driven evaluation is not only to assess whether an inter- vention works or does not work, but also how and why it does so." He differentiates theory- driven evaluation from "black-box" which mainly assesses whether an intervention has an impact on outcomes and from "methods-driven" evaluation which uses a research method as a basis for conducting an evaluation. Version Feb 20, 2017, 6pm, for CODE 45 | P a g e