Annual Report Transforming Evidence FY19 into Better Outcomes About IEG The Independent Evaluation Group (IEG) is an independent unit within the World Bank Group. It reports directly to the Boards of Executive Directors, which oversee IEG’s work through the Committee on Development Effectiveness. IEG is charged with evaluating the activities of the World Bank (the International Bank for Reconstruction and Development and the International Development Association), the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA). The goal of IEG’s evaluation work is to improve the Bank Group’s ability to achieve development outcomes by providing impartial, evidence-based assessments and lessons on drivers of success and failure. IEG uses rigorous methods that triangulate findings from a cross-section of sources to provide credible evaluations with new insights based on the choice of topic and methodologies used. Sharing these findings, strengthening client evaluation practice, and engaging stakeholders ensures that insights are understood and necessary improvements made. All evaluations discussed in this report are available on IEG’s website at http:// ieg.worldbank.org. Contents Message from the Vice President and Director-General, Evaluation  2 Evaluation Insights for a More Resilient World 4 Synthesizing Past Lessons for Future Gains 10 Results and Performance of the World Bank Group 2018 16 Strengthening Client Evaluation Practice  18 FY19 Highlights  24 Maximizing Strategic Focus to Increase World Bank Group Effectiveness  32 Appendixes  36 Message from the Vice President and Director-General, Evaluation The close of the fiscal year marks the end of my first six FY19 saw IEG becoming more engaged and creating to the delivery of the Bank Group’s mission. We are also months as Director-General, Evaluation, at the Independent space and demand for the range of work we do—from increasing our focus on the Bank Group’s contribution Evaluation Group (IEG). Two specific things have struck evaluations to learning engagements—while retaining our to development outcomes, bringing greater attention me. The first is that there is significant appetite for relevant, independence. IEG’s work program was aligned with the to country development results and what is needed to timely, and useful independent evaluation. During my first World Bank Group’s strategic priorities, the SDGs, and the successfully tackle complex and systemic problems that months, I engaged with members of the Board of Executive commitments made in the capital increase packages for affect client countries. To achieve these ends, IEG must look Directors, World Bank Group senior management, and key the International Bank for Reconstruction and Development carefully at what we evaluate and how we can do better to stakeholders to better understand what is working well (IBRD) and International Finance Corporation (IFC), together deepen our evaluative evidence across our streams of work, and where there are opportunities for IEG to create even with the replenishment of the International Development while seeking to innovate to deliver more timely evaluations more value. We have converted many of these ideas into a Association (IDA). While IEG evidence confirmed that without losing rigor. We are also looking to expand our new work program for fiscal years (FY)20–23. The second the Bank Group continues to be an effective partner in activities in evaluation capacity development, building on is the need for evaluation to move beyond confirming the development progress of most of its client countries, the successes to date but with an even greater emphasis on development truisms to better understanding not only “what several evaluations during the year highlighted the promoting better monitoring and evaluation practices and works” but why, how, and for whom. This means asking persistent challenges of including poor and vulnerable capacity in client countries. not only whether the Bank Group is doing things right but groups in sustainable economic growth, tackling historically We have a huge agenda ahead of us, and I am excited to be whether it is doing the right things to support the twin goals high levels of forced displacement, and creating markets in at IEG. and maximize the contribution it and its partners can make low-income settings, among others. to the Sustainable Development Goals (SDGs), both in client Alison Evans IEG’s FY20 work program concentrates even more squarely countries and globally. Director-General, Evaluation on the development effectiveness questions that are central 2 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 3 Evaluation Insights for a More Resilient World In a year marked by change throughout the World Bank Group, IEG’s commitment to rigorous analysis, innovative methodological approaches, and the sharing of knowledge and lessons remains its foundation. Though topics ranged from forced displacement to creating markets, the evaluations presented similar stories about development and the effectiveness the World Bank Group, providing guidance on improving outcomes. Common themes included building resilience, monitoring who benefits, and increasing private sector engagement. In FY19, IEG published five major evaluations. Conflict-Induced Displacement Urban Resilience Regional Integration Since 2016, the World Bank Group has stepped up its More than 1 million people move into cities each week, and Development issues often transcend borders. Natural engagement in situations of conflict-induced forced such rapid growth strains institutions and infrastructure. Cities disasters, climate change, pandemics, and other crises do displacement, responding to the steep increase in with unmanaged growth cannot respond appropriately to not respect geography. Tackling issues regionally allows the numbers of internally displaced persons and shocks, such as natural disasters, or chronic stresses, such for both immediate responses and long-term planning to refugees and the extreme poverty and human suffering as violence and crime. These challenges can be overcome increase resilience over time. In Two to Tango: An Evaluation associated with this increase. and recognizing that the by investing in and implementing policies that build urban of the World Bank Group Support to Fostering Regional humanitarian model by itself is not a sustainable long-term resilience. In Building Urban Resilience: An Evaluation of Integration, IEG determined that the Bank Group fosters approach. IEG’s evaluation, World Bank Group Support the World Bank Group’s Evolving Experience (2007–17), regional integration by enabling clients through advisory in Situations of Conflict-Induced Displacement, found IEG noted that because urban needs and capacities vary and analytical work, financing projects through policy that a complementary and simultaneous development greatly, the Bank Group must constantly innovate and learn and investment loans, and convening state and nonstate response starting from the onset of a crisis helps build by doing. However, it can be difficult to extract lessons actors for coordination and collective actions. Interventions the conditions that enable the forcibly displaced to move from innovations because the Bank Group lacks a shared generally improved regional connectivity, public goods, toward independence, dignity, and hope. The development understanding of “urban resilience,” and no institutional or institutions. Although the portfolio includes a spectrum approach can ease the impact on host communities by framework exists to assess how projects contribute in of practice areas, sectors, regions, and subregions, most bringing together development partners to build institutions the medium or long term. At the operational level, better efforts across the World Bank (69 percent), IFC (51 percent), and policies that promote economic opportunities. Private coordination is needed among the Bank Group institutions and MIGA (89 percent) focused on infrastructure. Bank sector investment presents an underused pathway to and among the Global Practices. At the system level, more Group Regions also receive disparate amounts of support economic growth that could increase the participation systematic assessment of resilience risks is warranted to because the Bank Group lacks high-level commitment or of the forcibly displaced in the market. The evaluation recommends that the Bank Group build on its analytical identify and strategically respond to the most critical risks. strategic priorities on regional integration in all Regions work to create context-specific interventions, strengthen The Bank Group’s approach also needs to expand beyond except Africa, which has a Regional Integration Strategy. country-level coordination, and identify and support addressing acute shocks to tackling chronic stresses. The Project success reflects these two areas of concentration. opportunities for private sector development. evaluation recommends increasing multisectoral and cross- The report recommends that the Bank Group increase institutional coordination, identifying system-level risks commitments outside Africa by tailoring regional strategies evenly across the portfolio, and systematically incorporating and rebalance projects to emphasize regions that are found resilience characteristics throughout the project cycle. to have high integration potential. 6 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 7 Creating Markets Knowledge Flow and Collaboration The international community recognizes that the SDGs Achieving the Bank Group’s ambitious goals requires an will not be achieved without participation from the private effective operating model that allows for adequate knowledge sector, but engaging the private sector requires competitive sharing and collaboration among institutions and units. In markets that function efficiently. The 16 case studies Knowledge Flow and Collaboration under the World Bank’s reviewed for “Creating Markets” to Leverage the Private New Operating Model, IEG recognized that the creation Sector for Sustainable Development and Growth revealed of Global Practices (GPs) and overarching Global Theme that IFC and MIGA have contributed significantly to market Groups in 2014 improved knowledge flow across Regions, creation. In addition to expanding services to small and but a systematic approach to managing and investing in medium markets, IFC has made markets competitive and knowledge is not present in all GPs. GPs with coherent sustainable by enhancing public and private sector capacity, fostering innovation, generating demonstration effects, and knowledge strategies address gaps through cross-support, promoting the integration of value chains. However, the staff rotation, knowledge hubs, and training. Knowledge is Bank Group could greatly amplify its contribution by fully transferred to operations through help desks, safe-space understanding the sectors and areas it gets involved in, meetings, and other platforms. However, interviews with using more diagnostic approaches. For example, enhancing Country Management Units revealed that larger country access to markets for the underserved groups, including the programs receive more knowledge than smaller programs. poor, will require better understanding how market creation As for collaboration, although the GPs reduce siloing among specifically affects each group. For IFC to engage more Regions, heavy transaction costs and competition among GPs in creating markets, it needs to adapt its overall business complicate collaboration. The 2014 model lacks the strong model and its incentives for staff to work in nascent leadership needed to encourage collaboration except in cases private sector markets, taking more risks and allocating of crisis response or in high-profile projects. The evaluation more resources up front. IEG recommends enhancing recommends improving incentives for knowledge production, the understanding of market creation opportunities, curation, and sharing; strengthening links among GPs and including through the recently introduced Country Private Sector Diagnostics, and ensuring that such knowledge is Regions to improve coordination; and empowering Program adequately reflected in the Country Partnership Framework. Leaders to connect staff and knowledge across boundaries to find integrated solutions. 8 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 9 Synthesizing Past Lessons for Future Gains The Bank Group’s twin goals of ending extreme poverty and achieving shared prosperity demand a clear focus on who is benefiting from growth and development; specifically, what is happening to the most vulnerable people in society—those at higher risk of falling into or remaining in poverty. In FY19, IEG extracted findings and lessons from past evaluations on topics relevant to the poorest, the often-excluded, and those affected by crises to clarify Bank Group support in a specific area from different perspectives. Of the four synthesis reports, three cover IDA-related topics, including special windows. The fourth considers inclusive growth, an important part of achieving the twin goals. Learning from IDA Experience IDA Crisis Response Window IDA’s 18th Replenishment, the largest in the institution’s IDA’s Crisis Response Window, put in place in 2008 with 56-year history, was expected to speed progress toward IDA’s 15th Replenishment, originally focused on economic the international community’s 2030 agenda. Special themes shocks but later expanded to include natural disasters included jobs and economic transformation; gender and and public health crises. The window is designed to offer development; climate change; fragility, conflict, and violence rapid response, transparency and predictability, support (FCV); and governance and institutions, each of which is for resilience in natural disaster response, and support for examined in Learning from IDA Experience: Lessons from long-term development objectives. IDA’s Crisis Response IEG Evaluations, with a Focus on IDA Special Themes and Window: Lessons from IEG Evaluations found that half Development Effectiveness. Achievement of the objectives of the funding from 2010 to 2018 went to the Africa varied across themes, with the strongest results seen in Region, and about half responded to natural disasters. gender and jobs and economic transformation. This review Projects financed through the window performed better of evaluations from FY12–17 showed that development than the average IDA operation in delivering project effectiveness of IDA operations is improving overall but outcomes. Success was associated with building on still lagging in FCV countries and that while project-level existing operations that were already good performers, ratings improved, country program ratings remain below close coordination with partners and other international the corporate target. A key message in the report is that institutions in the process of crisis response, taking a more monitoring and evaluation design and country data gaps flexible approach to delivery, and putting future resilience at need to be substantially strengthened so appropriate the centre of operational design. However, because Crisis indicators can be selected, tracked, and used to inform Response Window funds were frequently pooled with other decision-making in real time. Training in results reporting at IDA financing, it remains challenging to separately evaluate the corporate level and the use of systemwide approaches the achievement of all of the window’s objectives. to build statistical capacity show promise in ameliorating these problems. 12 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 13 IDA Regional Window Inclusive Growth IDA’s Regional Window Program funds regional integration– The Bank Group has pursued an inclusive agenda for related projects in low-income countries. Since the decades, and strategies increasingly focus on equity. window’s inception in 2003, support to such projects Considering evaluations completed between FY09 and has nearly tripled. Evidence in IDA Regional Window FY18, Inclusive Growth: A Synthesis of Findings from Program 2003–17: Lessons from IEG Evaluations indicates Recent IEG Evaluations found that the Bank Group was that Regional Window support works both upstream to a leading source of analytical work used to understand strengthen the institutional capacity of regional institutions, the drivers of inclusive growth, and its interventions promote regional policy reform, and set up new regional supporting various related aspects (for example, economic institutions, and downstream to finance cross-border assets and opportunities, human development, and power, transport, and other predominantly infrastructure social protection) have effectively achieved their expected projects. The Africa Region received the largest share development outcomes. However, projects did not always of support through the window. Outside of Africa, the clearly articulate the mechanisms through which they report found that the window has been used especially helped the poorest or the often-excluded, and several well in landlocked countries, small states, and FCV recommendations called for the systematic consideration states; however, support is less prominent in regions and of these groups in the design of strategies and operations. subregions with high potential and demand for integration. The evaluation noted that the Bank Group made significant Although the Regional Window guidelines specify that strides in extending access to basic social, infrastructure, projects should generate benefits that spill over country and financial services to large segments of the population, boundaries, the definition of what this means is unclear, though lack of data on the poorest groups and the often- making it difficult to find evidence of such effects. excluded is a major constraint. Access is not the same as use, however, and reaching true inclusion will require a deeper understanding of the needs of those being served. The role of the private sector, though acknowledged, also requires further investigation. 14 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 15 Results and Performance of the World Bank Group 2018 IEG Ratings by Region, Institution, and Practice Group Outcome ratings for World Bank projects in IBRD, IDA, and FCSa countries, and development outcome ratings for IFC investment Outcome ratings by Region and advisory projects CY06–08 CY07–09 CY08–10 CY09–11 CY10–12 CY11–13 CY12–14 CY13–15 CY14–16 CY15–17 CY06–08 CY07–09 CY08–10 CY09–11 CY10–12 CY11–13 CY12–14 CY13–15 CY14–16 CY15–17 100 100 East Asia 100 100 and Pacific The flagship report, the Results and Performance of the World Bank Group (RAP), is Europe and Results and Performance of theWorld Bank Group 2018 80 Central Asia 80 IBRD 80 South Asia IBRD 80 IEG’s annual review of the development effectiveness of the Bank Group. This year’s report IDA IDA Rated MS+ (%) Rated MS+ (%) Rated MS+ (%) Rated MS+ (%) Africa FCS Latin America FCS showed that outcome ratings and Bank performance ratings 60 for World Bank lending have 60 and the Caribbean Middle East and 60 60 North Africa continued on an upward trajectory by number and lending 40 volume. Ratings increased in all IFC Investment 40 40 IFC Investment 40 IFC Advisory IFC Advisory Practice Groups, with standout improvements in the Human Development Practice Group, Results and Performance of 20 20 20 20 FY06–08 FY07–09 FY08–10 FY09–11 FY10–12 FY11–13 FY12–14 FY13–15 FY14–16 FY15–17 FY06–08 F the World Bank Group 2018 AN INDEPENDENT EVALUATION and across all Regions, except for the Latin America andFY06–08 FY07–09 FY08–10 FY09–11 FY10–12 FY11–13 FY12–14 FY13–15 FY14–16 FY15–17 the Caribbean Region, which FY06–08 FY07–09 FY08–10 FY09–11 FY10–12 FY11–13 FY12–14 FY13–15 FY14–16 FY15–17 Year of closing Year of closing (World Bank) or evaluation (IFC) is largely explained by weaker project outcomes in Mexico and Brazil. In countries with Outcome ratings by Practice Group (percent rated MS+) fragile and conflict-affected situations, a modest decline in ratings was seen, mainly owing FY12–14 FY15–17 FY12–14 FY15–17 FY12–14 FY15–17 FY12–14 FY15–17 to weak project outcomes in Afghanistan and the Republic of Yemen. IFC development Equitable Growth, Finance, and Institutions Human Development Infrastructure Sustainable Development outcome ratings continued the decline that began in 2008. Although the latest cohort of 65 70 75 84 69 78 69 75 evaluated projects coincided with difficult economic conditions globally, the decline in Poverty and Equity Social Protection and Labor Transport and Digital Development Social, Urban, Rural, and Resilience 50 100 89 93 73 84 77 80 ratings remains strongly associated with persistent challenges in IFC’s work quality, which Finance, Competitiveness, and Innovation Education Energy and Extractives Agriculture the institution is now seeking to address. MIGA’s development outcome ratings increased 71 75 65 89 65 70 70 75 Macroeconomics, Trade, and Investment Health, Nutrition, and Population Environment and Natural Resources modestly, with the strongest showing in environmental and social effects. 68 73 78 75 59 74 Governance Water 54 64 61 67 Trade and Competitiveness 69 62 Source: Independent Evaluation Group. Note: Different time frames are applied to assessing the performance of projects across the institutions. IFC investment projects are sampled, evaluated, and reported by calendar year. World Bank, IFC advisory services, and Multilateral Investment Guarantee Agency projects are reported by fiscal year. CY = calendar year; FCS = fragile and conflict-affected situation; FY = fiscal year; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association, and the IDA category includes IDA blend; IFC = International Finance Corporation; MS+ = mostly satisfactory or above (World Bank) / mostly successful or better (IFC). a. IEG assigned FCS status based on the harmonized list of FCS situations. b. Although the FY18 restructuring of EFI phased out the Trade and Competitiveness (T&C) Global Practice, some projects mapped to T&C remained in World Bank business systems as of October 3, 2018, the data cutoff date for this report. Therefore, T&C is still reported as a Global Practice in this report. Some T&C-mapped projects are expected to be remapped to Finance, Competitiveness and Innovation, others to Macroeconomics, Trade, and Investment. 16 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 17 Strengthening Client Evaluation Practice IEG is a key player in supporting evaluators both in IEG and in client countries through several evaluation capacity development programs. IEG drives two capacity building initiatives, the International Program for Development Evaluation Training (IPDET) and the Centers for Learning on Evaluation and Results (CLEAR) Initiative. IEG also invests in the skills and capabilities of its own staff through the IEG Academy. Evaluation Capacity Development Brazil and Lusophone Africa South Asia Developed and delivered, Worked with the state One important part of IEG’s mandate is to assist national, regional, and local governments, as well as civil society together with CLEAR government in Tamil Nadu to and nongovernmental organizations, in strengthening their monitoring and evaluation (M&E) capacity in support of Francophone Africa, a establish a comprehensive better-designed public policies and increased accountability for results. It is particularly important in the context foundations of M&E course system of M&E throughout of the SDGs and Agenda 2030, which require governments to monitor progress and understand what accelerates for policy makers in Guinea- the government and support advancement toward them. Bissau as part of ongoing the use of evidence by state CLEAR is a global network comprising six centers that provide capacity building support for M&E in six different regions of efforts to improve the officials in public program the world. The objective is to deepen the M&E culture in these regions and help establish the needed legal and regulatory effectiveness of public policy in design. framework, systems, procedures, and skills. In FY19, CLEAR demonstrated its commitment to evaluation capacity a fragile state. development with projects in each region. Anglophone Africa Francophone Africa East Asia Latin America Worked with members of Conducted a workshop with Trained nearly 500 M&E Led, with support from the Tanzanian Parliament to Niger’s Finance and Budget professionals with the objective the International Fund for increase the use of evidence Committee to deepen the of growing the pool of highly Agricultural Development, in parliamentary discussions country’s commitment to skilled local evaluators. the delivery of the Program and decisions. evaluation and build capacity in Rural M&E (PRiME) in performance-based course to 158 rural sector budgeting. professionals from over 50 countries to enhance the use of M&E by government officials working in the agricultural sector. 20 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 21 CLEAR is also a strong advocate for promoting the use of evidence in policy making globally. In June 2019, CLEAR IEG Academy launched the gLOCAL Evaluation Week. This event recognized that today, more than ever before, global knowledge is shaping local evaluation practices while, at the same time, local experiences are contributing to strengthening global In FY19, the IEG Academy took on its second year in full force. Two all-staff dedicated Learning Days highlighted the evaluation thinking. In this context, over 270 events were organized in more than 90 cities across 38 countries to share importance of professional growth and development to IEG. Courses covered a wide range of topics, including the global and local M&E knowledge and experiences. foundations of evaluation design, problem-solving techniques, and design thinking. In addition, over the course of the year, IEG Academy hosted 47 classes and provided on-the-job tools and coaching opportunities. IPDET is a global executive program in M&E. In July 2019, IEG supported—with its partners the Center for Evaluation in Germany and the University of Bern, Switzerland—the second year of IPDET. This year’s program benefited from closer collaboration with partners and instructors from the Global South. A total of 230 participants from 85 countries took the A highlight of FY19 was the release of the World Bank Group Evaluation course this summer to enhance their M&E knowledge and skills. Principles, co-led by the IEG Methods Adviser in collaboration with IFC, MIGA, Methods Advisory and the World Bank. Based on the evaluation framework established in FY18, the document delineates core principles for evaluation and underlying principles IEG’s methods advisory team aims to advise and influence IEG evaluators in implementing high-quality methodological for planning, conducting, and using evaluations at the Bank Group. Informed by approaches in their evaluations and introducing methodological innovations within IEG. To that end, the methods advisory team worked with other IEG teams to apply a broader range of methodological approaches, including process international evaluation principles and good practice standards, the common tracing, social network analysis, Delphi techniques, structured literature reviews, and protocol-based portfolio analysis, in principles are designed to enhance development results by strengthening evaluations occurring in FY19. A highlight of FY19 involved the release of the World Bank Group Evaluation Principles, co- led by the IEG Methods Adviser. Finally, other work involves updating and developing methodological guidelines, unifying accountability and learning. The principles align the Bank Group’s evaluative and professionalizing the work done by IEG’s data analysts, collaborating with World Bank M&E colleagues on bringing efforts with global challenges and with its strategic focus; clarify the roles and external experts to community of practice learning events (the Results Measurement and Evidence Stream), and guiding responsibilities of key actors and encourage synergy among them throughout M&E training with the IEG Academy. Regarding the latter, the methods advisory team successfully organized and delivered courses for IEG staff (and, increasingly, with participation from colleagues from other international DC-based institutions the evaluation process; and ensure that all Bank Group evaluations are robust, such as the Inter-American Development Bank): Foundations of Evaluation Design; Theory of Change; and several Portfolio of high quality, and credible. Review and Analysis training modules. 22 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 23 FY19 Highlights 5 Major Evaluations 2 Meso Evaluations Project Performance Assessment Reports 43 Reports 1 Corporate Evaluation 4 Synthesis Reports 35 Countries 4 Sector and Thematic Evaluations 7 Learning Engagements 62 0 Country Projects Program Evaluations Total Validations Implementation Completion and Results Report Reviews Project Completion Reports 443 Reports 75 Reports 228 Reports 89 Countries Expanded Project 96+ Countries Supervision Reports Completion and Learning Report Review 104 Reports 18 Reports MIGA Validations 18 Countries 18 Reports ANNUAL REPORT FY19 25 Evaluating Globally PPAR PPAR + ICRR ICRR PPAR + CLRR CLRR CLRR + ICRR ALL Note: Map clearance provided by the World Bank Cartography Unit. CLRR = Completion and Learning Report Review; ICRR = Implementation Completion and Results Report Review; PPAR = Project Performance Assessment Report. 26 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 27 Most-Downloaded Reports Most-Read Blogs LE A R N ING PR O DUC T Public-Private Partnerships in Health Hope or Hype? Attracting Investors to Emerging Markets and Developing Economies 3,509 World Bank Group Engagement in Health PPPs Carbon Markets for Greenhouse Gas An IEG Synthesis Report World Bank Group Support to Health Services Emission Reduction in a Warming World Engaging Citizens for Results and Performance of the An Evaluation of the World Bank Group’s Achieve me nts a nd Cha lle nge s Better Development Results World Bank Group 2017 Support to Carbon Finance AN INDEPENDENT EVALUATION AN INDEPENDENT EVALUATION AN INDEPENDENT EVALUATION Uncovering Factors of Project Success: A Literature Review 3,316 Citizen Engagement 2017 RAP Carbon Finance Health PPP LP Health Services Evaluation and the Sustainable Development Goals: Unpacking the Issues 2,067 1,583 1,481 1,067 946 864 Why Growth Alone Is Not Enough to Reduce Poverty 1,957 A Return to the World Bank Group 1,806 Knowledge Flow and Collaboration under the Grow With the Flow ‘Creating Markets’ to Leverage the World Bank’s New Operating Model An Evaluation of the World Bank Group’s Private Sector for Sustainable Development Support to Facilitating Trade 2006–17 and Growth A N I N D E PE N D E N T E VA LUAT I O N An Evaluation of the World Bank Group’s Experience Through 16 Case Studies Knowledge Flow Regional Integration Facilitating Trade Creating Markets and Collaboration 847 825 649 548 28 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 29 Total Spending, FY15–19 Summary of IEG Expenses, FY19 27% $10.1K 5% $1.9K 60 13% $4.9K 37.9 37.6 FY 4% $1.4K 40 34.1 34.6 37.4 19 23% $8.5K 33.4 ($, millions) 1% $0.5K 8% $3.1K 20 2% $0.8K 17% $6.6K 0 FY15 FY16 FY17 FY18 Planned Actual FY19 Administration Evaluation Capacity Development Major Evaluations Country-Focused Evaluations Knowledge and Communications Meso Evaluations Validations Learning Engagements Project-Level Evaluations 30 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 31 Maximizing Strategic Focus to Increase World Bank Group Effectiveness In FY20, IEG will position itself to focus on the development effectiveness questions that most concern the institution and its clients. IEG’s work program for FY20 is built around several work streams that align with Bank Group strategic priorities, the IBRD and IFC capital packages, and recent IDA replenishments. The work program seeks to provide even greater impact by building evidence across evaluations, providing more timely and flexible evaluation formats, and closing the feedback loop through more systematic engagement throughout the evaluation cycle. To increase the benefits of its strategic topic selection, meaningful, of these is outcomes. IEG will work closely with IEG will seek lessons from multiple perspectives at the Bank Group management in FY20 to take the institution’s country, client, and beneficiary levels, and will offer outcome orientation agenda to the next level. This means improved knowledge packaging and customized outreach finding ways to generate better information on the results for different audiences. For example, Country Program and impacts of Bank Group operations, extending beyond Evaluations will be geared toward maximizing their potential simple attribution to a concrete analysis of contribution, to inform the design of subsequent Country Partnership which will encourage an even stronger push for results. IEG Frameworks while contributing more broadly to enhancing will contribute by reviewing project-level validations and the Bank Group support for development impact more evaluations for quality, use, cost, methodology, and value. broadly. Demand-driven learning engagements will be held Improving evaluation practice is a cornerstone of IEG’s work. on M&E frameworks in FCV situations, the World Bank’s In FY20, IEG intends to expand its role as an incubator of organizational effectiveness to deliver on its jobs agenda, innovative practices and methods for evaluation and build and South-South exchanges. At the corporate level, IEG on its unique position to bring technical experts together by plans to provide timely inputs to the IDA19 midterm review expanding its role as an incubator of innovative practices and and the IDA20 discussions by evaluating areas relevant to methods for evaluation. Overarching themes include systems the IDA18 and IDA19 Replenishment Themes. approaches, case-based methods (especially relating to IEG will also increase its engagement with operational causal analysis), using new data (for example, social media counterparts throughout the evaluation life cycle to enrich or geospatial data), and new approaches for data analytics content and use. At the beginning of the process, this will and visualization. provide evaluators with informed, context-specific inputs. As IEG introduces more flexible approaches to answer At the end, the increased attention to dissemination will strategic evaluation questions, it will take a correspondingly boost uptake and allow evaluators to see their lessons put adaptable approach to presenting its findings. FY20 will into play, providing more insights for future evaluations. see shorter, more focused evaluative products and more All facets of Bank Group projects should be examined and willingness to consider different product styles designed learned from, but the most visible, and perhaps the most for specific audience needs. We are excited to push the evaluation practice in a fresh, meaningful direction. 34 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 35 Appendixes Work Program, FY20–23 SDG FY20 FY21 FY22 FY23 SDG FY20 FY21 FY22 FY23 Urban spatial growth (TE) Gender 5 Gender equality Gender strategy PSEA and GBV Growth and 2 8 9 Agricultural Jobs and workers (TE) Migration in FCVs (E) (TE) Transformation Structural transformation (TE) transformation (TE) Lagging regions Financial (cluster CPE) SOE reforms (TE) inclusion (TE) 10 11 Building national Fragility, 16 Post-conflict IFC/MIGA in IDA/FCV Resource-Rich FCV strategy Crisis preparedness (TE) innovation systems (E) Conflict, and reconstruction and countries (TE) FCV countries (E) Disruptive technology Violence reengagement (E) World Bank Group CPEs of FCV CPEs of FCV Readiness (TE) CPEs of FCV countries (CPE) countries (CPE) countries (CPE) Outcome RAP RAP RAP Tackling extreme 16 1 IFC additionality MIGA strategic poverty Climate Change 13 15 6 Natural resources Solid waste Water resources Climate action in middle-income objectives Shared prosperity degradation (TE) management (TE) management (E) plan countries (TE) achievement (E) (Update) Disaster risk Forest action Results agenda RAP 14 7 reduction (E) plan Review (E) Capital package AIMM and IMPACT Program-for Results Maximizing 17 Mobilization of private Domestic resource Blended finance Catalyzing Institutional World Bank Group Building effective and Change decentralization accountable institutions Other lending and Finance for capital (TE) mobilization (E) (TE) private finance effectiveness (E) (TE) guarantee instruments Development IDA Private Sector Foreign direct [will include corruption] Environmental and Social Window (E) investments Framework Procurement Human Capital 2 3 4 Aging (TE) Stunting (TE) High fertility (TE) Social services SCD/CPF Update Social protection in delivery Knowledge management FCV countries (E) Human Capital Knowledge strategy Project and RASs Index Learning from failures GBV = gender-based violdence; PSEA = Protection from Sexual Exploitation and Abuse; RAP = Results and Performance of the World Bank Group; Note: Evaluations particularly relevant to the Bank twin goals are shown in green. AIMM = Anticipated Impact Measurement and Monitoring; RAS = reimbursable advisory services; SCD = Systematic Country Diagnostic; SOE = state-owned enterprise; T = thematic evaluation. CPE = Country Program Evaluation; CPF = Country Partnership Framework; E = other evaluation; FCV = fragility, conflict, and violence; 36 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 37 IEG Deliverables and Spending (number, unless otherwise indicated) IEG Staffing (number) Deliverables FY15 FY16 FY17 FY18 FY19a Staffing FY15 FY16 FY17 FY18 FY19 Planned Actual Percent Total Staff 105 114 118 113 109 Total Major Evaluations 7 9 8 9 10 7 70 By Grade Corporate Evaluations 2 3 3 3 2 1 50 UC 1 2 1 1 4 Sector and Thematic Evaluations 4 2 5 4 6 5 83 GA-GD 23 23 24 23 21 Country Program Evaluations 1 2 — 2 2 0 0 GE 7 8 11 6 5 Meso Evaluations — — — 3 4 2 50 GF 21 27 22 20 20 Synthesis Reports — — 2 — 4 4 100 Impact Evaluations — 2 — — — — — GG 30 30 36 38 37 Learning Products — 6 — — — — — GH 19 17 20 21 17 Learning Engagements — — 11 7 9 7 77 GI and above 3 4 4 4 3 Project Evaluations and Global Program Reviews Other (ETT, ETC, UC) 12 3 — — 2 Project Performance Assessment Reports 36 51 50 43 64 43 67 Total Validations 571 442 431 427 515 443 86 Implementation Completion and Results Reports 392 267 285 273 250 228 91 Completion and Learning Report Review 13 22 18 11 20 18 90 Project Completion Reports 72 47 54 38 102 75 73 Expanded Project Supervision Reports 78 88 69 95 123 104 85 MIGA Validations 16 18 14 10 20 18 90 Note: — = not available. a. For validation products, IEG makes a rough estimate of the planned numbers at the beginning of the year, but actual delivery is determined by how many reports the World Bank Group provides to validate. FY18 delivery of Project Performance Assessment Reports was below target values because of one-time effects from substantial turnover in staff and management. Internal coordinators for these reports have been put in place, and a robust pipeline has been developed for FY19 delivery. In the case of the Completion and Learning Report (CLR) Reviews, actual delivery to the Board is determined by the timing of the Country Partnership Framework Board discussion. All CLRs received by IEG are reviewed and delivered to the Board on schedule in line with the Policy and Operations unit of the Secretariat lead time requirements. In other words, the percentage does not reflect slippages in CLR Review delivery but slippages or delays in Country Partnership Framework Board discussions. 38 INDEPENDENT EVALUATION GROUP ANNUAL REPORT FY19 39 Credits Photos Production Cover, Sergi Lopez Roig / shutterstock Amanda O’Brien Page 2, IEG / World Bank Design Pages 4–5, A’Melody Lee / World Bank Luísa Ulhoa Page 6, Allison Kwesell / World Bank Page 9, Arne Hoel / World Bank Pages 10–11, Dominic Chavez / World Bank Pages 12–13, Yosef Hadar / World Bank Page 15, Stephan Gladieu / World Bank Pages 18–19, Abbas Farzami / Rumi Consultancy / WorldBank Page 24, Arne Hoel / World Bank Page 27, Sarah Farhat / World Bank Pages 32–33, Eugene Salazar / World Bank Pages 34–35, World Bank 1818 H Street NW, Washington, DC 20433 http://ieg.worldbank.org ieg@worldbank.org