World Bank Group Country Engagement An Early-Stage Assessment of the Systematic Country Diagnostic and Country Partnership Framework Process and Implementation A N I N D E PE N D E N T E VA LUAT I O N © 2017 International Bank for This work is a product of the staff RIGHTS AND PERMISSIONS Reconstruction and Development / of The World Bank with external The material in this work is subject to The World Bank contributions. The findings, copyright. Because The World Bank 1818 H Street NW interpretations, and conclusions encourages dissemination Washington, DC 20433 expressed in this work do not of its knowledge, this work may be Telephone: 202-473-1000 necessarily reflect the views of The reproduced, in whole or in part, for Internet: www.worldbank.org World Bank, its Board of Executive noncommercial purposes as long as Directors, or the governments they full attribution to this work is given. Attribution—Please cite the report represent. as: World Bank. 2017. World Bank Any queries on rights and licenses, Group Country Engagement: An The World Bank does not guarantee including subsidiary rights, should be Early-Stage Assessment of the the accuracy of the data included in addressed to World Bank Systematic Country Diagnostic and this work. Publications, The World Bank Group, Country Partnership Framework 1818 H Street NW, Washington, DC Process and Implementation. The boundaries, colors, 20433, USA; fax: 202-522-2625; Independent Evaluation Group. denominations, and other information e-mail: pubrights@worldbank.org. Washington, DC: World Bank. shown on any map in this work do not imply any judgment on the part Cover Photo: © kentoh and Toria/ of The World Bank concerning the Shutterstock legal status of any territory or the endorsement or acceptance of such Cover Design: Crabtree + Company boundaries. World Bank Group Country Engagement An Early-Stage Assessment of the Systematic Country Diagnostic and Country Partnership Framework Process and Implementation AN INDEPENDENT EVALUATION Careful observation and analysis of program data and the many issues impacting program efficacy reveal what works as well as what could work better. The knowledge gleaned is valuable to all who strive to ensure that World Bank goals are met and surpassed. contents abbreviations  v acknowledgments  vi overview  vii management response xvi management action record  xxii chairperson’s summary: committee on development effectiveness xxix 1. Introduction 1 Background and the Objectives of the New Model 1 The New Country Engagement Model  3 The IEG Evaluatio 4 References6 2. The Systematic Country Diagnostic 7 Background  Results and Performance of the 7 World Bank Group 2015 Evidence Base and Framing of the Issues  Growth, Poverty and Shared Prosperity, and Sustainability  8 9 Identification of Priorities for Achieving the Twin Goals  12 AN INDEPENDENT EVALUATION Process and One World Bank Group  14 Emerging Messages  17 References18 3. The Country Partnership Framework 19 Background  19 Identification of Priority Objectives for World Bank Group Support  19 Selectivity  24 The Results Framework  25 One World Bank Group  28 Risk Analysis  30 The Use of Country Engagement Notes  32 Emerging Messages  33 References34 Careful observation and analysis of program data and the many issues 4. Institutional and Cross-Cutting Issues in the New Country Engagement Model impacting 35  program efficacy reveals what Background  35 as what works as well Internal Processes  could work35 better. The One World Bank Group—The Involvement of IFC and MIGA  knowledge 38gleaned is Partnerships  valuable to all 40who strive to ensure that World Cross-Cutting Themes and Issues in SCDs and CPFs  42 Bank goals are met and Emerging Messages  46 surpassed. References47 Independent Evaluation Group | World Bank Group iii 5. Findings and Recommendations`  48 A New Approach  48 Main Findings  48 Recommendations  53 Reference  54 Bibliography  55 Boxes Box 1.1. Egypt’s Need for Stronger Poverty Data and Analysis  9 Box 2.2. Impact of Economic Growth on the Poor in Indonesia  11 Box 2.3. Indonesia’s Prioritization Process  12 Box 2.4. Use of Analytic Approaches in the Selection of Priorities in the Chad SCD  13 Box 2.5. Treatment of Governance in Myanmar  14 Box 2.6. Albania’s Consultation Process  16 Box 3.1. Addressing the Need for Enhanced Poverty Monitoring  22 Box 3.2. Divergences Between the SCD and CPF Priorities  22 Box 3.3. Matching the CPF Program with the Long-Term Priorities of the SCD  25 Box 3.4. Examples of Ambiguous Indicators  27 Box 3.5. World Bank Group Collaboration in the CPFs  29 Box 4.1. The Corporate Review Process for the Serbia SCD  36 Box 4.2. Country Partnership of Proposed CPF Priorities Has Been Good but Is Not Assured  41 Box 4.3. Responding to Government Priorities on Climate Change in Bangladesh  43 Figures Figure 3.1. CPF Selectivity Filters  20 Figure 4.1. Elapsed Time for SCD-CPF Process 37 Tables Table 3.1. Comparison of Risk Ratings by Category for 22 Countries  31 Table 4.1. Comparison of Elapsed Time for the CAS, SCD, and CPF  37 Table 4.2. Comparison of Budget Used by the CAS, SCD, and CPF  38 Appendixes Appendix A. Evolution of Country Assistance Strategies  63 Appendix B. Methodology  70 Appendix C. List of Country Case Studies  78 Appendix D. Poverty Reduction and Shared Prosperity in Systematic Country Diagnostics  82 Appendix E. Results Frameworks in CPFs  85 Appendix F. Country Engagement Notes  92 Appendix G. IFC and MIGA Integration in the SCD ans CPF Process  103 Appendix H. Gender Integration in the New Country Engagement Model  121 Appendix I. Assessment of Governance and Political Economy Issues in SCDs and CPFs  145 Appendix J. Integration of Fragility Assessments in SCDs, CPFs, and CENs  151 iv World Bank Group Country Engagement | Contents abbreviations CAS country assistance strategy CCSA cross-cutting solutions area CEN Country Engagement Note CMU Country Management Unit CPF Country Partnership Framework CPS country partnership strategy FCS fragile and conflict-affected situation IBRD International Bank for Reconstruction and Development IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation MIGA Multilateral Investment Guarantee Agency OPCS Operations Policy and Country Services PLR Performance and Learning Review PSD private sector development ROC Regional Operation Committee SCD Systematic Country Diagnostic SME small and medium-size enterprises SORT Systematic Operations Risk-Taking Tool All dollar amounts are U.S. dollars unless otherwise indicated. Independent Evaluation Group | World Bank Group v acknowledgments This report of the Independent Evaluation Group (IEG) was prepared by a team led by Ismail Arslan, with major contributions and background papers provided by Iradj Alikhani, Zeljko Bogetic, Mauricio Carrizosa, Anis Dani, Shahrokh Fardoust, Juan Jose Fernandez-Ansola, Nils Fostvedt, Gisela Garcia, Monika Huppi, Basil Kavalsky, Paul Levy, Xue Li, Andres Liebenthal, Neeta Sirur, Aurora Medina Siy, Kate Steingraber, Clay Wescott, and Kendra White. Yasmin Angeles, Aimée Niane, and Carla F. Chacaltana were responsible for all administrative aspects of the evaluation. William Hurlbut and Caroline McEuen provided editorial support. The team gratefully acknowledges the support of task teams and task team leaders for Systematic Country Diagnostics and Country Partnership Frameworks throughout the World Bank Group. Colleagues in the World Bank Operations Policy and Country Services (OPCS) Vice Presidency, International Finance Corporation Country Framework Engagement, and Multilateral Investment Guarantee Agency Economics Unit were also very helpful, as were the many IEG staff and managers who offered ideas and perspectives at various points during the process. The team also acknowledges the advice received from Brenda Barbour, Elena Bardasi, Soniya Carvalho, Unur Demberel, Javier Lanza, Lauren Kelly, Bahar Salimova, Stoyan Tenev, and Josef Leonardus Vaessen. Peer reviewers were James Adams (former World Bank vice president for OPCS and the East Asia and Pacific Region) and David Dollar (former World Bank country director for China). The evaluation was conducted under the guidance of Nick York, director; Marie Gaarder, acting director; Mark Sundberg, manager; Anjali Kumar, acting manager; and Caroline Heider, director- general, evaluations. vi World Bank Group Country Engagement | Acknowledgments overview A New Model for Country Engagement U N D E R T H E W O R L D BA N K G R O U P ST R AT E GY released in 2013, country programs will focus on accelerating progress toward sustainably reducing poverty and building shared prosperity (the “twin goals”). To this end, the World Bank Group established a new model for country engagement consisting of two instruments. The Systematic Country Diagnostic (SCD) uses data and analytic methods to help country clients and World Bank Group teams identify the most critical constraints to, and opportunities for, reducing poverty and building shared prosperity. The Country Partnership Framework (CPF) determines focus areas for World Bank Group support that align with the country’s development agenda, address the key constraints and opportunities identified in the SCD, and reflect the comparative advantages of the World Bank Group. Responding to a request from the Board of Executive Directors, the Independent Evaluation Group (IEG) has reviewed the processing and early implementation of the new country engagement model. This evaluation seeks to answer their expressed need for an early look at how well the SCD/CPF process is working and identifies ways to strengthen the new country engagement model as it continues to unfold. There are no country-level implementation results yet available from the new model. IEG has used evidence (desk reviews, consultations, 10 field visits, and comparisons with the previous country assistance strategies) from the production of SCDs and CPFs as a basis for the evaluation, reviewing the first 22 CPFs (up to end of April 2016) discussed at the Board and their associated SCDs. Another 10 CPFs with SCDs were completed by end of November 2016. vii The Systematic Country Diagnostic The new SCD instrument has added considerable value to the quality of the World Bank Group’s engagement with countries. The instrument has been well received inside and outside the World Bank Group, including by governments and development partners. Most of the SCDs reviewed provided a strong or reasonable analytical basis for the main issues, constraints, and opportunities they identified. They recognized country characteristics relevant to the analysis of growth, poverty, and inequality. They presented links between growth and poverty reduction, often with discussion about different measurements of poverty and trends in income inequality. In a few cases, special analytical frameworks were used to examine key variables of economic growth and poverty. The SCDs depended heavily on available analytical material and data. While the reports benefited from work by the World Bank and other organizations, gaps in data inevitably meant that some SCDs suffered from weaknesses in their analyses of the future needs for achieving the twin goals. Many SCDs identified knowledge and data gaps that needed to be addressed to improve the evidence base for future policy making. The identification of opportunities for economic growth was the strongest of the SCD dimensions explored. Some SCDs argued for shifts in prevailing growth models or identified areas of growth potential. The majority analyzed the drivers of economic growth and their impact on poverty. The impact of economic growth on the poor was analyzed extensively, with a focus on the drivers of poverty reduction, such as job creation and labor income, social assistance, and equality of opportunity in education and health. However, quantification of possibilities for economic growth, poverty reduction, or shared prosperity was rare. Appropriate focus was given to the analysis of shared prosperity and the bottom 40 percent of the population, but SCDs generally did not analyze the bottom 40 percent when extreme poverty was near or greater than that threshold. Regarding sustainability, some key issues did not always get sufficient attention. Fiscal, environmental, and social sustainability were addressed adequately by the SCDs. But while the threats to social and political stability were well analyzed in several cases, especially in fragile countries, not all SCDs provided a robust analysis of relevant social or political risks. All the SCDs identified priorities for achieving the twin goals, but clearer guidance is needed on prioritization. Some SCDs discussed policies and policy alternatives, while others focused largely on structural issues. Consultative processes were often used to identify priorities, but there was significant variation in the methodology used to select them and in the number and nature of priorities identified. In most cases, these were at a high level of generality—areas to be addressed rather than specific actions to be taken. This meant that more specific policy actions would need to be identified through subsequent policy dialogue. Some SCDs identified such a large number of priorities that they undermined their value in contributing to selectivity in the related CPFs. The SCD and CPF teams undertook broad consultations with a wide range of stakeholders. The International Finance Corporation (IFC) played an important role in outreach to private sector viii World Bank Group Country Engagement | Overview stakeholders. The consultations were useful in identifying issues for the SCDs, but in some cases there were complaints that the consultation process was pro forma, with no subsequent feedback to stakeholders. Many of the stakeholders consulted also found the difference between the SCD and CPF consultation processes to be somewhat blurred, and some participants found them lengthy and duplicative. Nonetheless, consultations have been effective in reaching out to key stakeholders to obtain inputs and identify priorities for the SCDs and CPFs. Governments and development partners noted that the SCDs were useful in the formulation of their own country programs, and the document often covered new analytical ground for a country. Also, the SCD, beyond its function as an input to the CPF, provides a unique opportunity to help define an agenda for dialogue on key development issues in a country, and thus to enhance broader country ownership of agreed strategic priorities and, among stakeholders, for the country’s own development programs. The reports therefore have a significant public good aspect. However, sharing of the SCD findings with country stakeholders has fallen short of serving the potential public good nature of the document. IEG’s consultations with stakeholders indicated significant weaknesses in distribution of the report once the SCD was completed. It is important for the World Bank Group to disseminate the SCD findings widely, beyond meeting its own disclosure requirement. The Country Partnership Framework Unlike the SCD, the CPF is a well-established World Bank product. However, the new engagement model, which builds on the SCD’s analysis of the country’s constraints and priorities, has led to modifications in the traditional country program documents. The expected link between the World Bank Group program and the achievement of the twin goals is a departure from the more generalized support for growth and poverty reduction found in previous country assistance strategies (CASs). World Bank Group guidance identifies three criteria for the selection of program priorities: alignment with SCD priorities, alignment with the government’s own priorities, and areas where the World Bank Group has a comparative advantage. Most of the CPFs reviewed referred to these criteria, and some discussed each of them. But in practice, country teams also used priorities based on the World Bank Group’s ongoing operational program—whether or not these were consistent with the three criteria. While most CPFs have a table or a paragraph comparing the SCD and CPF priorities, they usually did not clearly explain the reasons for the variances among them. Aligning the program priorities of the CPF to the constraints and priorities identified in the SCD was challenging. In part this is because the SCD and CPF have very different purposes. The SCD’s purpose is to assess what steps the country needs to take to achieve the twin goals; the CPF, by contrast, is the World Bank Group’s own program of support over a five-year period, focusing on areas where it can maximize the value it adds to the government’s program. Independent Evaluation Group | World Bank Group ix The SCDs identified knowledge and data gaps, but the CPFs did not address these gaps effectively. The guidance to SCD teams recommends that they identify the gaps in data needed to assess the incidence of poverty and the constraints to achieving the twin goals. The CPF guidance calls for an indication of how the gaps will be filled and in what time frame, and also about the arrangements and capacity for enhanced poverty monitoring. But many CPFs did not explain how, or by whom, the gaps most relevant to CPF objectives would be filled. They were particularly weak on the need and capacity for enhanced poverty monitoring. All CPFs indicated that the World Bank Group’s comparative advantage was a criterion for the selection of priority objectives in the program. In practice, however, they referred to absolute advantages, such as convening power or the ability to combine analytic work with operational support. The World Bank Group’s comparative advantage depends not just on its own capacities, but on those of the government and development partners. Only a few CPFs systematically analyzed what other donors were doing and explored the implications for the World Bank Group program. In preparing the CPFs, it proved difficult in a number of cases to relate the SCD priorities to the ongoing lending program. This became an issue where the SCDs identified new priorities that were not prominent in the ongoing program. Some country teams responded by defining a long list of priorities that would be consistent with almost any set of World Bank Group–supported programs, while others stated CPF objectives in general terms that would accommodate both the ongoing program and the SCD priorities. The evaluation identified as good practice the CPFs that clearly separated the ongoing program from what was needed to support the new program, whether through modifications of the existing program, new operations in the outer years, or through a redesigned program of analytic work and technical support. The definition of the results chains in CPFs has improved, but remains work in progress. The results chains can best be described as truncated, with several items missing. In general, the “upstream” part of the results chain was handled well, although there were still gaps between achievement of objectives and achievement of indicators. However, the role of the World Bank Group’s program in contributing to the achievement of these outcomes was often not well thought through and explained. It would be useful to identify core activities that will drive the achievement of an indicator in the results chains, separating these from other activities that will contribute more generally to the objective over time, and to establish these links more explicitly in the discussion of the results chains. The identification of monitorable and measurable indicators for the results was a strong point in many of the CPFs reviewed. However, the tension between the World Bank Group’s ongoing program and the SCD priorities was also evident in the results framework. The requirement for a measurable and monitorable results framework led to indicators being drawn mainly from the existing World Bank Group program. Consequently, a number of results frameworks confined themselves to indicators for the first three years and did not reflect new activities. This sent the wrong signal by implying that new operations were not important for achievement of strategic objectives. The results frameworks need to include both ongoing and new work, while making a clear distinction between ongoing operations and the new program to be implemented in the future. x World Bank Group Country Engagement | Overview Overall, the contribution of Advisory Services and Analytics to the World Bank Group program was handled unevenly in the CPFs reviewed. Long-term commitments to build capacity and institutions were poorly reflected in the results framework. These programs need to be delineated separately, with identified markers that allow monitoring of progress to ensure the programs are on track. Nonlending activities also tended to be poorly integrated into the results frameworks. Sometimes their contribution to outcomes was not reflected at all in the frameworks. Another weakness in most CPFs was the inadequate discussion of the country’s capacity to monitor the elements of the results framework. The risk assessments were systematic, but had significant weaknesses. All CPFs used the Systematic Operations Risk-Rating Tool (SORT), which was introduced in 2014 to provide a more structured method of rating risks. This tool was originally designed for use in projects, and some elements have little relevance for country-level risk analysis. The country teams have been conservative in their risk assessments. However, the descriptions of risks were often generic, and the risk mitigation or risk adaptation strategy was often vague, possibly because of the broad way in which the risks themselves had been framed. The presentation and discussion of risks also needs to draw a clearer distinction between World Bank and IFC programs, since these have very different risk profiles. Overall, the SORT may need to be adjusted to better support the CPF through more focus on country-level risks and how the World Bank Group can address systemic operational risks. Use of Country Engagement Notes World Bank Group guidance advises teams to use a Country Engagement Note (CEN) only in limited circumstances, when a country is facing a conflict or political crisis, or when the World Bank Group is reengaging after a prolonged hiatus and lacks the knowledge needed to formulate detailed objectives or a medium-term program. IEG found a number of apparent deviations from the expectations expressed in the guidance, leading to the conclusion that the current practice needs to adhere more stringently to the guidance that limits the use of CENs and to put in place the elements to be able to monitor its performance. Half of the CENs reviewed lacked basic elements that would allow an analysis of performance and lessons. Because CENs are often prepared in situations where the World Bank Group has knowledge gaps, they are not required to be preceded by SCDs, and they are not expected to contain as much documentation and analysis as a CPF. But they are expected to lay the basis for the preparation of SCD/CPFs by putting in place efforts to fill the knowledge gaps. Most of the CENs provided little indication of how this would be done, and it is therefore not clear if most of the CEN countries will be able to graduate to an SCD/CPF mode in the next cycle. Internal SCD and CPF Processes Internal World Bank Group processes for SCD and CPF preparation and review generally worked well. The corporate review process often helped to enhance the quality of the reports. The Global Independent Evaluation Group | World Bank Group xi Practices were heavily engaged in the processes, through multisectoral consultations, participation in the large country teams, and formal review meetings. However, task team leaders interviewed by IEG indicated that engagement with the Global Practices on SCDs was complex, and in some cases identification of a limited set of priorities was made more difficult by pressure from special sectoral or thematic interests. There were also cases where task teams felt pressure from country management units to identify priorities in line with current or already planned work programs. The timing and processing of the new engagement model—SCDs and CPFs combined—inevitably took longer than the previous CAS preparation cycle. The time spent on production of the CPF alone has been less than that needed for the previous CAS, but preparation of the SCD was additional, so the average time elapsed from concept review to Board presentation increased from 231 days for the last CAS (in the same 22 countries) to 388 days for the two documents combined under the new model. In the future, the average processing times may decline somewhat as the World Bank Group moves up the learning curve. The new processes clearly required additional resources. The evaluation compared budgets (World Bank only) charged to each SCD and CPF with the expenses on the last CAS in the same set of 22 countries. Actual expenditures charged to the CPF code were, on average, marginally higher than those spent on the previous CAS, despite the shortened time spent on preparation of the CPF. The expense for the SCD was additional. These costs did not include IFC and the Multilateral Investment Guarantee Agency (MIGA) costs. IFC does not yet have a system for tracking costs related to individual country SCD and CPF engagements. MIGA uses a time recording system code to track overall staff time and cost of its SCD and CPF engagement, which is funded through its regular budget. However, this does not permit identification of such costs at the country level. One World Bank Group The integration of IFC and MIGA into the CPF process has significantly improved under the new country engagement approach. Overall, IFC and MIGA participation was more evident in the SCDs and CPFs than under the previous CAS approach. IFC was an integral part of the team in all 22 SCDs and CPFs reviewed by IEG, even in countries where it had limited operations, and there is now an IFC structure and established work flow for the new approach. As a result, the integration of the IFC program in the SCDs and CPFs was broader and more substantive than under the previous CAS approach. The new approach improved coordination and collaboration among World Bank Group entities, but the budget transparency issues remain. IFC had greater involvement in shaping the overall country strategy and gained more knowledge and learning from stakeholder consultations. The new approach also provided MIGA with better engagement opportunities within the World Bank Group. However, for IFC and MIGA staff, the approach has not been supported by dedicated budget resources for SCD and CPF preparation in individual countries. For the SCDs, IFC provided inputs on private sector prospects and issues, enriching the potential strategic focus of the World Bank Group. IFC’s role in promoting better corporate governance and xii World Bank Group Country Engagement | Overview improved labor and working conditions also provided useful additional perspectives. SCDs were written by World Bank staff, with inputs and comments from IFC and MIGA. IFC also led the private sector consultations in several SCDs. Task team leaders found considerable IFC engagement in the SCD processes, stronger in some countries than in others, despite different operational modalities. MIGA’s small size, lack of country presence, and operational model often did not allow extensive participation in this process. Coordination of the strategic focus among the World Bank Group has improved in the CPFs, but further improvement is needed. Most CPFs discussed the strategic direction and content of the IFC and MIGA operations and how they contribute to the overall program, but this discussion was often not adequately reflected in the results framework, which frequently lacked information on their planned operations. In the future, it would be useful for IFC and MIGA to at least identify intended areas of concentration and lines of activities in the CPF results matrixes. The discussion in the CPFs of private sector development (PSD) tends to be driven by the World Bank. A review of PSD discussions in the results frameworks of the 22 CPFs found that they did not fully integrate the perspectives of all three World Bank Group institutions. The discussions tended to give a government-centric, policy-oriented view and did not adequately draw on the private sector perspective that IFC, in particular, could have provided. Private sector analyses in future SCDs need to go beyond a discussion of general policy constraints to include a more business-oriented and granular analysis of a country’s private sector. Cross-Cutting Themes Overall, the 22 reviewed SCDs did a reasonable job of addressing cross-cutting themes—gender, climate change, and fragility—identified in the guidance, while job creation has become a regular topic in a number of the SCDs. The majority of the SCDs discussed gender issues. These discussions reflected an evolution beyond human development toward economic empowerment, especially related to labor force participation, skills development, and jobs. Among the 22 SCDs, 19 discussed gender issues and 8 identified key gender gaps to be addressed. Several SCDs offered good examples of gender integration, identifying gender gaps related to labor markets, skills development, and school-to-work transitions. However, when priorities were identified in the SCDs, their gender linkages were often not explicitly spelled out. Compared with previous CASs, CPFs were better at articulating World Bank Group actions to address specific gender inequalities, and included many more gender-disaggregated indicators. But the alignment between the actions proposed and indicators in the results frameworks was often weak. World Bank Group Gender Strategy (FY16–23): Gender Equality, Poverty Reduction and Inclusive Growth, finalized after the current guidance for the SCD/CPFs, provides more specific directions on how gender can be incorporated in SCDs and CPFs. The guidance will need to be updated to incorporate these expectations. Independent Evaluation Group | World Bank Group xiii Climate change is an emerging agenda that was often addressed in conjunction with environmental sustainability. While half of the SCDs explicitly discussed climate change, others discussed environmental sustainability in the context of sector operations. Most of the SCDs contained a substantial discussion of climate change or environmental sustainability, and included these issues directly or indirectly among SCD priorities, although some SCDs in vulnerable countries (such as Mali) did not discuss climate change. Most of the CPFs, including in countries where there had been little discussion of these matters in the SCDs, paid attention to these issues. Among the 22 CPFs reviewed, 18 specified objectives or subobjectives addressing climate change or environmental sustainability in relation to specific sector operations, such as energy or land management. Fragility assessments were useful for the preparation of SCD/CPFs in fragile situations. The evaluation reviewed the contribution of fragility assessments to SCD/CPFs and CENs in the six countries where they were available. In this sample, both SCDs and CPFs made good use of the available fragility assessments. The availability of these assessments in countries with fragile situations appears to have made it easier for SCDs, CPFs, or CENs to integrate fragility within the World Bank Group strategy for those countries. Most SCDs and CPFs take governance into account in both analysis and program recommendations, but the measures proposed to address governance and capacity constraints were not always commensurate with identified shortcomings. The guidance for SCDs points out that political economy factors are a crucial part of country context, and should be used in identifying constraints, solutions, and opportunities. Governance was highlighted in most SCDs as a cross-cutting theme or prerequisite for achieving the twin goals. However, a clear approach to governance was missing in some SCDs, which could have benefited from more systematic cross-country comparisons on governance indicators and more explicit discussion of capacity constraints and plans to build institutional capacity in line with country-specific governance challenges. Recommendations Recommendation 1: The World Bank Group should enhance its guidance and ensure more consistent and focused priority setting in SCDs and more explicit discussion of strategic priorities in CPFs. CPFs could be strengthened by a systematic discussion of the basis for the strategic priorities in the country program and indicating its alignment with or the reasons for divergence from each of the priorities identified in the SCD. Recommendation 2: The World Bank Group should disseminate the SCD to further enhance its potential use by other stakeholders in the country, with specific steps to be determined by each country team. With appropriate dissemination, the SCDs have the potential to underpin the development dialogue both with and in borrower countries. Recommendation 3: The CPFs should explicitly indicate how the SCD-identified knowledge and data gaps, which are most relevant to CPF objectives, will be addressed. SCDs depend heavily on available analytical material and data. The guidance would benefit from providing adequate xiv World Bank Group Country Engagement | Overview discussion in the CPF showing its expected follow-up to the identified knowledge and data gaps. Recommendation 4: The World Bank Group should strengthen the results frameworks in CPFs to ensure that they more clearly reflect the ongoing and new work programs and results of all types of interventions. This could be addressed in four ways: first, by requiring CPFs to distinguish clearly between the ongoing programs and the new programs that reflect the strategic directions in the country, whose results will begin to materialize primarily in the outer years of the CPF; second, by more explicit inclusion in the results framework of nonlending advisory services and activities, as well as institutional development and capacity-building programs; third, by more explicit inclusion in the results frameworks of the current and future areas where IFC and MIGA intend to focus their support, considering constraints from their business models; and fourth, by providing corresponding training on results frameworks to country teams. Recommendation 5: The World Bank Group should modify the country engagement guidance to ensure consistency with the new Gender Strategy. The guidance on the country engagement model needs to be reconciled with the directions provided in the strategy to ensure its consistent application in SCDs and CPFs. Recommendation 6: The World Bank Group should improve resource management and budget transparency to strengthen One World Bank Group participation in the new country engagement model. Budget allocation and resource management systems could be strengthened at the country level by improving budget-recording practices in the World Bank for greater consistency and to capture the full cost of preparing the SCDs and CPFs; and in IFC and MIGA, more clearly allocating, tracking, and managing resources used on the new country engagement products. Independent Evaluation Group | World Bank Group xv management response Introduction T H E WO R L D B A N K G R O U P welcomes this early stage assessment by the Independent Evaluation Group (IEG) of the Systematic Countr y Diagnostic (SCD) and Countr y Par tnership Framework (CPF) processes and implementation. The findings are based on a limited sample of the first 22 completed CPFs and their associated SCDs; and, as recognized by IEG, it is still too early to draw conclusions on the overall effectiveness of the new process. Nonetheless, this early assessment and the continuing accumulation of experience with the new approach are valuable in informing World Bank Group’s ongoing efforts to fur ther strengthen the countr y engagement process. The management of the World Bank Group institutions appreciates IEG’s general finding that (i) the new country engagement process brings improvement to the quality and focus of the CPF process compared with the former Country Assistance Strategy (CAS) and Country Partnership Strategy (CPS) process, and (ii) SCDs have added considerable value to World Bank Group engagement with member countries.1 Management also broadly concurs with most of the report’s recommendations and several suggestions for improving the approach to country engagement. Many aspects highlighted by the report have been addressed by recent and ongoing efforts to strengthen and improve the country engagement model; these include the following points: „„Revision  of SCD and Country Engagement Guidance. Guidance notes for country engagement products are updated periodically to account for emerging experiences with implementation and other developments. Most relevant is the rollout of the first full SCD Guidance note in December 2016.2 This addresses many issues raised in the IEG report, notably, (i) strengthening advice on prioritization  of constraints for meeting the World Bank Group’s twin strategic goals; (ii) identifying key knowledge and data gaps; and (iii) addressing cross-cutting issues in the analysis. A revision of the broader country engagement guidance will likely be adopted in fiscal year (FY)18. This will provide an opportunity to address CPF-specific issues raised by the IEG report, including the CPF’s role in defining selectivity of the proposed World Bank Group program and aligning it with SCD priorities, strengthening results frameworks, and reducing processing time and cost across World Bank Group. „„The  World Bank Group has been undertaking two complementary analytical exercises with the aim to take stock and reflect on the early lessons of implementation of the country engagement process. The first stocktaking exercise of the SCD process, led by the SCD Advisory Group, was completed in December 2016. It assesses accumulated SCD experience in terms of design, quality, and process. The analysis is based on quantitative surveys and in-depth interviews among xvi World Bank Group Country Engagement | Management Response task teams. A separate, ongoing exercise based on desk reviews of completed SCD reports reviews emerging patterns in priority areas and up-to-date data and knowledge gaps identified by SCDs. The second exercise, a CPF stocktaking conducted jointly by IFC and World Bank, has recently been launched. Its main purpose is to systematically assess critical aspects of the CPF implementation experience, including (i) alignment of CPF objectives and SCD prioritization areas; (ii) CPF process requirements, including those related to enhanced collaboration across World Bank Group institutions; and (iii) results frameworks. This will complement the main findings of the IEG report and inform periodic updates of Guidance Notes and Procedures. „„  of country engagement processes. Management is committed to minimizing Simplification process and time requirements for preparing country engagement documents. A revised country engagement procedure that eliminated some clearance steps was rolled out in December 2016. Although the “agile pilots” do not yet extend to country engagement processes, an increasing number of teams have introduced agile features. The revised SCD Guidance also incorporates recommendations on simplifying SCD preparation and review processes, including short SCD concept notes using PowerPoint format as an option with a clearly defined scope and a maximum length of 80 pages. Training ƒƒ programs and support to SCD and CPF teams. Management believes that one of the most effective ways to strengthen country engagement products is by fostering cross-team learning and sharing of experience. For SCDs, this is championed by the SCD Advisory Group, supported by a technical secretariat tasked with curating technical and informational resources, including specific guidance notes, presentations, and databases for SCD task teams, and collecting and sharing knowledge and experiences on SCD preparation. Larger events are organized about once a year to share diagnostic experiences and lessons more broadly with external partners. Learning activities on CPF products for World Bank Group staff are conducted throughout the year through biannual CPF Academiescoordinated by Operations Policy and Country Services and IFC Country Engagement Unit as flagship events. Systematically gathered feedback shows that teams are benefiting from these resources and from just-in-time advice and support from the technical secretariat, SCD Advisory Group, and Operations Policy and Country Services. The IEG report offers an and to refine processes and methodological approaches, including through the efforts mentioned above. Management broadly concurs with IEG’s recommendations (see responses in the Management Action Record). Beyond these recommendations, the IEG report contains other suggestions that touch on issues critical for strengthening country engagement, such as CPF Results Frameworks, incorporating cross-cutting themes in SCDs and CPFs, involvement of IFC and MIGA, consultations and dissemination, the use of country engagement notes (CENs), and treatment of risks in CPFs. These are discussed in the following sections. Overall Effectiveness of the New Country Engagement Process IEG rightly notes that no results are yet available from implementation of the new country engagement process. Hence, it seems too early to assess the overall effectiveness, including Independent Evaluation Group | World Bank Group xvii cost-effectiveness, of the new country engagement process. Findings are based on a small sample of 22 CPFs and SCDs as well as 8 CENs. It is also important to recognize that the cases reviewed were early adopters of the new model. Full assessment of effectiveness can only be conducted once a critical mass of CPFs has been fully implemented and their results evaluated. IEG’s reporting of the comparison of time and cost incurred for SCDs and CPFs with those for the preceding strategy products (that is, CASs and CPSs) finds anincrease on both counts. Management believes this point could be misleading. First, this comparison would need to take into account that the SCD is an additional product devised with the specific purpose of improving the analytical basis of the CPF. It is to be expected that introducing such a new product requires additional time and resources. Second, the timeline and cost of the former CASs and CPSs would not have captured the additional time and resources spent on analytical work that often preceded the drafting of CASs but was not considered explicitly as a part of the CAS or CPS process. It would be necessary to examine how far SCDs have replaced or added to other analytical products such as the Country Economic Memorandum (CEM) or poverty assessments. Third, several country teams have engaged in a new CPF cycle by starting the SCD process well in advance. This would show as an apparent increase in time taken for the process when timelines for SCDs and CPFs are added up. However, an early start to SCD preparation is considered good practice as it allows sufficient time for preparing a high-quality SCD and for clear sequencing with the CPF preparation process, as intended by the new country engagement model. Finally, since the SCDs and CPFs reviewed can be regarded as early adopters, they may not be representative of the cost and time compared with those that follow. It is also expected that second-generation SCDs would involve less cost and time given that they would be, to a large extent, updates of the first SCD produced for a country. CPF Results Framework Management appreciates the IEG finding that the presentation of results chains has improved through the introduction of a new results framework methodology for CPFs. According to the report, the explicit introduction of an intervention logic in the results framework improved links between objectives and indicators. The report indicates that the results framework, like many early product introductions, is still a work in progress. The report identifies some inherent tensions that exist in the application of the new results framework, namely, (i) the need to build results frameworks mostly around results of existing World Bank Group interventions, since new interventions are often not ready to contribute specific results at the time of CPF preparation or have a time horizon that goes beyond the CPF timeframe; (ii) difficulties in the identification of concrete and attributable results of knowledge products; and (iii) the IFC and MIGA business models that, unlike the World Bank, do not allow an identification of specific projects over the four-year CPF timeframe. xviii World Bank Group Country Engagement | Management Response There is also the broader challenge of setting CPF objectives at a level that represents a significantly ambitious impact on a country’s progress toward the World Bank Group twin goals while ensuring that corresponding results are attributable to World Bank Group interventions. Recognizing these challenges, management will continue to strengthen the CPF results framework. It will continue to support CPF teams in designing the results framework through training, advice, and other means. A more in-depth assessment of the experience with results frameworks will be needed to address broader challenges. Treatment of Cross-Cutting Issues in SCDs and CPFs Management concurs with the report that SCDs and CPFs do a reasonable job in addressing cross-cutting issues. Existing SCD and country engagement guidance stresses the need for adequate consideration of cross-cutting issues in SCDs and CPFs, notably those related to gender equality; fragility, conflict, and violence; and climate change. The country engagement guidance is clear about the need to address gender inequality issues in SCDs and CPFs in all of their key dimensions—endowments, economic opportunities, and voice and agency. This is reinforced by the recently released SCD Guidance that fully incorporates the main directions of the World Bank Group Gender Strategy and gender requirements under Operations Policy (OP) 4.20, Gender and Development. The next revision of the Country Engagement Guidance will include a similar update to strengthen gender considerations. Support has been enhanced to SCD and CPF teams on incorporating gender concerns in analysis, programming and results matrices such as, for example, efforts like the Gender Innovation Lab in the Africa Region. The IEG report states that SCDs have adequately addressed climate change considerations, yet it notes that there could have been a more in-depth treatment of this issue. The new SCD guidance highlights the importance of assessing climate change in terms of how it can influence trends in growth and poverty reduction and their sustainability. This would include examining the interplay between these trends and policies adopted in the context of the nationally determined contributions (as per the 2015 Paris Agreement to implement climate pledges). Efforts to support teams in incorporating climate change considerations in SCDs and CPFs are being strengthened, such as through the development of analytical tool kits and national- level resilience indicators. The IEG report stipulates a more in-depth analysis of political economy and governance challenges in SCDs. Similarly, IEG finds that CPFs reviewed propose measures that are not always commensurate with shortcomings related to political economy and governance identified in SCDs. In management’s view, many SCDs have identified governance-related challenges as being among the most critical constraints to inclusive growth, in particular in fragility, conflict, and violence– affected countries. It is intended to produce additional technical resources to help SCD teams in analyzing governance and institutions in the SCD Independent Evaluation Group | World Bank Group xix process as part of plans to operationalize the World Development Report 2017, Governance and the Law. The complexity of issues related to governance and political economy factors calls for a differentiated, country-specific approach to analyzing such factors rather than subjecting SCDs to a checklist of topics to be covered. The degree to which CPFs translate these issues into concrete and relevant options for World Bank Group support to client countries depends strongly on the country context. Enhanced Involvement of IFC and MIGA Management appreciates the report’s finding that IFC and MIGA involvement in the country engagement process has been enhanced. Management’s decision to make all SCDs and CPFs joint World Bank Group institutional products in line with the One World Bank Group approach has resulted in more integrated documents among IFC, MIGA, and the World Bank. There is a stronger emphasis in SCDs and CPFs on private sector–related challenges and opportunities. The report finds that the private sector’s role in economic growth and poverty reduction has become more prominent than in previous CAS/CPS documents. To further strengthen the private sector assessments underpinning SCDs and CPFs, the World Bank Group, through IFC and the Trade and Competitiveness Global Practice, is piloting a new approach on country private sector diagnostics. The diagnostic aims to assess the private sector’s contributions to development, through economy-wide impacts (for exampls, spillovers, job creation, access to products and services, opportunities for the poor) and through opening and creating markets. It will identify national-level and sector-specific market failures and policy impediments to private sector investment. It will include deep dives of selected sectors to assess both constraints and opportunities. It will also connect the dots across industries and between private finance and public actions to prioritize policy reforms and identify investment opportunities for catalyzing private sector activities. Consultations and Dissemination Management concurs that the experience with SCD and CPF consultations has been generally positive in relation to reaching out to a broad range of key stakeholders while there may be a need to more effectively sequence and coordinate consultations held for CPFs and SCDs. Management also shares IEG’s view on the strong potential of SCDs to contribute to and influence the national dialogue on development priorities. Management believes that it is critical to have country teams decide on the extent and nature of SCD dissemination, as they are in the best position to judge what is most effective, including by considering potential trade-offs between broad dissemination and the candor of analysis. xx World Bank Group Country Engagement | Management Response Use of Country Engagement Notes In the case of CENs, IEG’s assessment reveals misalignment between the guidance and use of CENs, in practice, in relation to the justification for choosing CENs over CPFs. In management’s view, clear criteria are in place for using CENs. As stated in the IEG report, a fragility, conflict, and violence context per se is not a sufficient argument for choosing the CEN option for a country, as demonstrated by cases in Afghanistan, Haiti, and Myanmar, for example, for which SCDs and CPFs were prepared. There are country-specific reasons for the choice in the six IEG-examined CEN cases. As recognized by IEG, the country engagement guidance states clearly that a CEN is to be used only when country conditions are ill-suited for a medium- term engagement by World Bank Group due to conflict or political transition or due to World Bank Group lacking sufficient knowledge when reengaging in a country after a prolonged hiatus. Continued efforts will be made to provide direct support to teams through training and other means to ensure that the CEN format will only be chosen in exceptional cases and duly explained in the CEN document. Treatment of Risks in CPFs IEG found that the Systematic Operational Risk Tool, as designed for operations, would need to be adapted for its use in CPFs. The treatment of risks in CPFs differs from that in operations. Nonetheless, management believes that there is substantial added value in using the same basic risk management tool across all financing instruments, Advisory Services and Analytics, and strategy products. Supporting teams in the adequate use of Systematic Operational Risk Tool in CPFs is part of ongoing training programs and corporate quality control. 1 In this document, “management of World Bank Group institutions” refers to the International Finance  Corporation, the Multilateral Investment Guarantee Agency, and the World Bank. 2 he Systematic Country Diagnostic (SCD) Guidance was introduced after the Independent Evaluation Group  report was finalized. It replaces the interim SCD Guidance that was in place for preparation of the first batch of SCDs, including those examined and reviewed in the evaluation. Independent Evaluation Group | World Bank Group xxi management action record Sharpen Guidance for Priority Setting IEG FINDINGS AND CONCLUSIONS The general methodology developed or applied to derive Systematic Country Diagnostic (SCD) priorities has been set out in the draft guidance and been conveyed and supplemented through training and other support to SCD teams. However, the application of this methodology has been mixed; the number and nature of priorities identified varied significantly among country teams. Often, these priorities were at a high level of generality—areas to be addressed rather than specific actions to be taken. For some countries, priorities were specific and limited in number; in others, numerous priorities were identified. The SCD identification of key development constraints is important so that the Country Partnership Framework (CPF) can pick up on them to focus the World Bank Group program, and there was evidence in a number of CPFs that the identification of key focus areas in the SCD resulted in a significant shift of emphasis in the World Bank Group program. The guidance is clear that one-to-one alignment is not expected between SCD and CPF priorities. One major factor driving nonalignment was the government’s view of where the World Bank should focus its support. Other important factors were the weight of the ongoing program and the quality of the SCD analysis. However, while most CPFs had a table or a paragraph comparing the SCD and CPF priorities, in general they did not do a good job of explaining the reasons for the differences between the two. The guidance on principles and possible criteria for priority setting in SCDs needs to be sharpened, while recognizing that CPFs may draw selectively on SCD priorities in the dialogue with the government on strategic CPF priorities. Either a more detailed text discussion or a brief annex providing a more systematic explanation regarding all the identified SCD priorities would be a useful addition to the current CPF. IEG RECOMMENDATIONS Recommendation 1: The World Bank Group should enhance its guidance and ensure more consistent and focused priority setting in SCDs and more explicit discussion of strategic priorities in CPFs. CPFs could be strengthened by a systematic discussion of the basis for the strategic priorities in the country program and indicating its alignment with or the reasons for divergence from each of the priorities identified in the SCD. ACCEPTANCE BY MANAGEMENT Partially agreed. MANAGEMENT RESPONSE Management agrees on the value of enhancing the SCD guidance to strengthen the principles for setting priorities. The revised SCD guidance Note, which was adopted in December 2016, has already done this. However, there is a need for flexibility in this regard and for taking into account country- specific context. guidance. This should be provided to SCD at a more general level, such as on principles and possible criteria for prioritization, examples of good practices in prioritizing, and the minimum level of granularity at which priorities need to be identified. To complement the principles outlined in the revised SCD guidance, additional technical resources on prioritization will be developed for SCD teams and made available at the online resource center. With regard to CPFs, management does not agree with the call for a “more explicit discussion of strategic priorities.” The existing country engagement guidance already explains the main principles for exercising selectivity of strategic priorities in World Bank Group engagement. CPF priorities are derived from SCD priority constraints. Hence, it is the SCD that should— through its objective, technical assessment— prioritize key constraints and opportunities that a country faces in advancing toward the twin goals. As the guidance states, the CPF exercises selectivity in the World Bank Group engagement, by intersecting SCD priorities with client demand and World Bank Group comparative advantage in the country-specific context. xxii World Bank Group Country Engagement | Management Action Record Improve Dissemination of SCDs IEG FINDINGS AND CONCLUSIONS The SCD document provides a new perspective on country development issues, and in some cases, it breaks new analytical ground for a country. These reports thus have a significant public good aspect. All SCDs have been publicly disclosed by the World Bank Group, but some were not disseminated in the country. A number of development partners noted the usefulness of the SCDs for the formulation of their own country programs. However, IEG’s consultations with stakeholders also found significant weaknesses in the extent to which the SCD is shared and discussed, with a considerable number of stakeholders not having seen or read the reports, even when they were posted on the World Bank website. This resulted in a lack of awareness among government officials, civil society, and development partners about the document’s availability and its content, which diminished its possible impact. The SCD, beyond its function as an input to the CPF, provides an important opportunity to help define an agenda for dialogue on key development issues in a country, and thus to enhance broader country ownership. It is therefore important for the World Bank Group to ensure the dissemination of this report, beyond meeting its own disclosure requirements, to enhance broader country ownership around key development priorities, with specific steps to be determined by each country team. IEG RECOMMENDATIONS Recommendation 2: The World Bank Group should disseminate the SCD to further enhance its potential use by other stakeholders in the country, with specific steps to be determined by each country team. With appropriate dissemination, the SCDs have the potential to underpin the development dialogue both with and in borrower countries. ACCEPTANCE BY MANAGEMENT Agreed. MANAGEMENT RESPONSE Management agrees that the important value of SCDs in contributing and influencing national dialogues on development priorities has been widely recognized and should be promoted where possible. The extent and nature of SCD dissemination should be left to country teams, who are the best judges of what is most effective, including by considering the potential trade-off between broad dissemination and candor of analysis. SCDs generally did a reasonable job of identifying knowledge and data limitations, although knowledge gaps were not always identified in the SCDs evaluated, and even when they were, the analytic work proposed in the CPFs was not always consistent with the SCD findings. Most CPFs provided little indication of how the gaps identified in the SCDs would be filled, by whom, and in what time frame. Thus, on the whole, the SCDs do a better job of identifying gaps in data and areas where additional work is required than the CPFs do in addressing these gaps, so that follow-up actions may be left unclear. While the World Bank Group cannot take responsibility for addressing all the knowledge and data gaps identified in the SCDs, it could take responsibility for paying attention to and tracking all the identified gaps, especially those that have a bearing on its strategic priorities. For instance, consideration could be given to including an annex in the CPF document that would detail the knowledge and data gaps and the expected respective responsibilities for addressing them, including the government and other development partners. heavily on available analytical material and data. The guidance would benefit from providing adequate discussion in the CPF showing its expected follow-up to the identified knowledge and data gaps. Independent Evaluation Group | World Bank Group xxiii Address Knowledge and Data Gaps IEG FINDINGS AND CONCLUSIONS SCDs generally did a reasonable job of identifying knowledge and data limitations, although knowledge gaps were not always identified in the SCDs evaluated, and even when they were, the analytic work proposed in the CPFs was not always consistent with the SCD findings. Most CPFs provided little indication of how the gaps identified in the SCDs would be filled, by whom, and in what time frame. Thus, on the whole, the SCDs do a better job of identifying gaps in data and areas where additional work is required than the CPFs do in addressing these gaps, so that follow-up actions may be left unclear. While the World Bank Group cannot take responsibility for addressing all the knowledge and data gaps identified in the SCDs, it could take responsibility for paying attention to and tracking all the identified gaps, especially those that have a bearing on its strategic priorities. For instance, consideration could be given to including an annex in the CPF document that would detail the knowledge and data gaps and the expected respective responsibilities for addressing them, including the government and other development partners. IEG RECOMMENDATIONS Recommendation 3: The CPFs should explicitly indicate how the SCD-identified knowledge and data gaps, which are most relevant to CPF objectives, will be addressed. SCDs depend heavily on available analytical material and data. The guidance would benefit from providing adequate discussion in the CPF showing its expected follow-up to the identified knowledge and data gaps. ACCEPTANCE BY MANAGEMENT Agreed. MANAGEMENT RESPONSE Management concurs that CPFs would benefit from a more systematic presentation of knowledge and data gaps, identifying which of these gaps could be addressed as part of the World Bank Group program. This point could be strengthened in the next revision of the CPF guidance. However, expectations would need to be managed as to the coverage of the World Bank Group program in this regard. The World Bank Group program can only address some of the knowledge gaps identified by the SCD. The CPF will therefore focus on those knowledge and data gaps that are directly related to the selected CPF areas and objectives. xxiv World Bank Group Country Engagement | Management Action Record Strengthen Results Frameworks in CPFs IEG FINDINGS AND CONCLUSIONS There can be tension between the World Bank Group’s ongoing program for a country and the priorities defined in the SCD. This comes to a head in the definition of the results framework. The requirement for a measurable and monitorable results framework means that indicators tend to be drawn mainly from the existing World Bank Group support program. With a five-year program, the planned results for the first three years are generally accounted for almost entirely by the existing program. It is virtually impossible, at least for investment lending, for planned new activities to be reflected significantly in the results framework in that time frame. In practice, a number of results frameworks confine themselves to indicators for the first two to three years that thus do not reflect potential new activities. This gives the wrong set of signals. Although the guidance note recognizes this dilemma and discusses ways of dealing with it, a sharpening of this guidance is needed. The results framework needs to make a clearer distinction between the ongoing operations and the new program to be developed in the course of the coming years and to establish their respective links to the objectives. Overall, the contribution of the World Bank Group’s nonlending activities to the country program was handled unevenly in the CPFs under review, and one-third of the CPFs were found deficient in this regard. Both World Bank Advisory Services and Analytics and International Finance Corporation (IFC) Advisory Services were often poorly integrated into the results frameworks, and longer-term institutional development and capacity-building programs were often ignored. Their contribution to outcomes was sometimes not even shown in the results frameworks. While most CPFs now discuss the strategic direction and content of the IFC/ Multilateral Investment Guarantee Agency (MIGA) programs and how these contribute to the overall World Bank Group program, in many cases this discussion was only weakly reflected in the results frameworks. While the ongoing programs of IFC and MIGA are generally listed, there is usually very little on their planned programs. The IFC and MIGA business models, with their dependence on private sector demand, can often make it difficult to identify their future operations. However, IFC and MIGA can be more explicit in the CPF results frameworks about their expectations and planned focus areas. IEG RECOMMENDATIONS Recommendation 4: The World Bank Group should strengthen the results frameworks in CPFs to ensure that they more clearly reflect the ongoing and new work programs and results of all types of interventions. This could be addressed in four ways: first, by requiring CPFs to distinguish clearly between the ongoing programs and the new programs that reflect the strategic directions in the country, whose results will begin to materialize primarily in the outer years of the CPF; second, by more explicit inclusion in the results framework of nonlending advisory services and activities, as well as institutional development and capacity-building programs; third, by more explicit inclusion in the results frameworks of the current and future areas where IFC and MIGA intend to focus their support, considering constraints from their business models; and fourth, by providing corresponding training on results frameworks to country teams. ACCEPTANCE BY MANAGEMENT Agreed. MANAGEMENT RESPONSE Management broadly agrees on the need to further strengthen the CPF results framework and will continue to refine it as more experience and lessons are accumulated through implementation. (Continued on the following page.) Independent Evaluation Group | World Bank Group xxv Strengthen Results Frameworks in CPFs (continued) The next CPF guidance will draw more attention to the points raised in the report, and management will continue to provide training on results frameworks to country teams as suggested in the report. However, in some cases it may remain difficult to ensure that CPF results frameworks more clearly reflect the ongoing and new work of programs and results of all types of interventions. Notably, there are issues of attribution related to knowledge-related interventions, which limits the ability of including advisory services and analytics in results frameworks. As regards the proposal for a more explicit inclusion in the results framework of the current and future areas of IFC and MIGA support, management believes that the IFC and MIGA business model does not lend itself to developing a detailed list of future projects and expected outcome in the CPF results matrix. As demand-driven organizations focused on the private sector, IFC and MIGA are not in a position to know in advance the specific clients that they will be working with or projects that they will be financing over the medium term. However, the main body of the CPF and the Intervention Logic section of the results matrix would identify the main strategic areas that IFC and MIGA will focus on when implementing the CPF. IFC will also enhance the articulation of the expected main results of its Advisory Services work in the CPF results framework. These aspects will be further clarified in the forthcoming revision of the country engagement guidance. xxvi World Bank Group Country Engagement | Management Action Record Integrate Gender into Country Engagement Guidance IEG FINDINGS AND CONCLUSIONS Several SCDs offer good examples of gender integration, identifying gender gaps related to labor markets, skills development, and school-to-work transitions. Nineteen of the 22 SCDs reviewed discussed gender issues, although the depth and quality of gender analysis in SCDs varied, and 8 SCDs highlighted gender issues among the main challenges. However, when priorities were identified in the SCDs, their gender linkages were often not clearly spelled out. Gender issues that are highlighted in SCDs tend to be echoed in CPFs, reflecting an evolution in focus from human development toward economic empowerment, especially related to labor force participation, skills development, and jobs. Compared with previous country assistance strategies, CPFs are generally better at articulating World Bank Group actions to address specific gender inequalities, and included many more gender-disaggregated indicators. But the alignment between the actions proposed and indicators in the results frameworks is still often weak. The recent gender strategy, approved after the current SCD and CPF guidance was promulgated, provides more specific directions on how gender dimensions can be incorporated in the SCDs and CPFs. IEG RECOMMENDATIONS Recommendation 5: The World Bank Group should modify the country engagement guidance to ensure consistency with the new gender strategy. The guidance on the country engagement model needs to be reconciled with the directions provided in the strategy to ensure its consistent application in SCDs and CPFs. ACCEPTANCE BY MANAGEMENT Agreed. MANAGEMENT RESPONSE The revised SCD guidance and forthcoming revisions to CPF Guidance will clarify how to address the new Gender Strategy and provisions of OP/BP 4.20, stressing in particular the importance of an adequate analysis of gender that goes beyond a brief review in SCDs. This includes the option of preparing a country gender assessment in cases where no such recent or relevant assessment exists, with a view to informing the SCD and CPFs. Where this is not possible, it is proposed that a short note be prepared that highlights data or knowledge gaps on gender equality. Independent Evaluation Group | World Bank Group xxvii Improve Resource and Budget Management IEG FINDINGS AND CONCLUSIONS The evaluation found deficiencies in budget recording within the World Bank, and the absence of a system to allocate and track costs dedicated to SCD/CPF preparation in IFC and MIGA. World Bank budget codes do not capture the full cost of all the staff listed on the SCD and CPF teams, and time recording can be inaccurate, with some country teams charging time heavily to SCDs and others to the CPF charge codes. IFC and MIGA have made a greater contribution to the new model than was the case for the country assistance strategy /country partnership strategy, but MIGA’s staff size does not allow it to contribute equally to all SCDs and CPFs. Further, IFC and MIGA do not have a corporate approach or systems for allocating resources and tracking the costs related to the SCD/CPF preparation—in the case of MIGA the system tracks preparation costs in the aggregate but not at the country level. These gaps occasionally affect the amount of attention that IFC or MIGA staff are able to give to the preparation of the two World Bank Group instruments. IEG RECOMMENDATIONS Recommendation 6: The World Bank Group should improve resource management and budget transparency to strengthen One World Bank Group participation in the new country engagement model. Budget allocation and resource management systems could be strengthened at the country level by improving budget recording practices in the World Bank for greater consistency and to capture the full cost of preparing the SCDs and CPFs; and in IFC and MIGA, more clearly allocating, tracking, and managing resources used on the new country engagement products. ACCEPTANCE BY MANAGEMENT Agreed. MANAGEMENT RESPONSE Management agrees on the importance of better capturing costs of preparing country engagement products. However, it feels that the report’s finding on the link between better budget recording and the strength of One World Bank Group’participation is tenuous. World Bank budget systems adequately provide for allocating costs of SCDs and CPFs to specific codes at country level. IFC and MIGA will develop a framework to further refine the tracking of the costs associated with SCDs and CPFs in an effective and efficient manner. While budget recording practices could be improved, accurate costing of SCDs presents a specific challenge, as teams often draw inputs from other analytical products prepared recently or in parallel. To the extent that they need to be addressed, resource management practices are a much broader topic that go beyond the scope of the country engagement model. xxviii World Bank Group Country Engagement | Management Action Record chairperson’s summary: committee on development effectiveness The Committee on Development Effectiveness (CODE) met to discuss the Independent Evaluation Group (IEG) report World Bank Group Country Engagement: An Early-Stage Assessment of the Systematic Country Diagnostic and Country Partnership Framework and the Draft Management Response. The committee welcomed the report, which was prepared by IEG in response to a request from the Board of Executive Directors for an early assessment of how well the Systematic Country Diagnostic (SCD) and Country Partnership Framework (CPF) processes are working. Members agreed that it is still too early to draw conclusions on the overall effectiveness of the new process, as no results are yet available from implementation. However, members found that despite its limitations, the early look was valuable to draw lessons to date and enable timely course corrections. They noted management’s concurrence with most of IEG’s recommendations to improve the approach to country engagement, with the exception of the call for a more explicit discussion of strategic priorities in CPFs. Members were pleased that the first full SCD guidance, issued by management in December 2016, has strengthened, amongst other issues, the principles for prioritization, cross-cutting issues, and dissemination of SCDs. They agreed with the need to stress core principles but to leave flexibility to country teams to allow them to adapt based on the country context. They appreciated Management’s undertaking of these two complementary analytical exercises on SCD and CPF to take stock and reflect on the early lessons of implementation of the country engagement model. As more experience is gained with the country engagement model, members encouraged Management to enhance guidance and apply agile principles and simplification vis-à-vis time and costs incurred for SCDs and CPFs for both the World Bank Group and its clients. Members generally agreed with IEG’s recommendation for the World Bank Group to enhance its guidance and ensure more consistent and focused priority setting in SCDs. However, they shared Management’s reservations on the recommendation for more explicit discussion of strategic priorities in CPFs. Members concurred that CPFs exercise selectivity in the proposed World Bank Group program on specific areas that intersect with priorities identified by SCD to achieve the twin goals; clients’ priorities and demand for the World Bank Group services; and the World Bank Group’s comparative advantage. They underscored the need to improve the CPF process, including the assessment of risks and identification of the World Bank Group’s comparative advantage in terms of governments and other development partners. Members advised to use selectivity filters to avoid the risk of “false prioritization” arising from trying to accommodate too many priorities from SCD and World Bank Group programs. Members agreed with the importance of strengthening the SCD and Country Engagement Guidance for the World Bank Group to consider country characteristics and identify priorities to address Independent Evaluation Group | World Bank Group xxix themes identified, for example, gender, climate change, fragility and governance, and job creation. They welcomed IEG’s recommendations on gender and management’s commitment to further address gender issues in the SCD and Country Engagement Guidance and through Country Gender Assessments. Members also underscored the importance of Fragility Assessments in the preparation of SCD and CPFs in fragile situations. Members concurred with the recommendation on dissemination of the SCD to further enhance its potential use by other stakeholders in the country, including other development actors and public policymakers. They highlighted the need to fortify coordination and harmonization with development partners and donors. Members also agreed that CPFs should address SCD-identified knowledge and data gaps, and that the World Bank Group should strengthen the results framework in CPFs. Members noted IEG’s recommendation on the need to improve resource management and budget transparency to strengthen One World Bank Group participation in the country engagement model. They were pleased with progress on the integration of the International Finance Corporation (IFC) and Multilateral Investment Guarantee Agency (MIGA) into the CPF process and results framework; members commented on the importance of enhancing strategic prioritization and incentives for IFC and MIGA staff. They noted the piloting of Private Sector Diagnostics by the new Country Economic and Engagement Department in IFC in addition to the IFC 3.0 initiative. They highlighted the importance of IFC’s contribution to policy analysis and private sector development, including the use of the Cascade. xxx World Bank Group Country Engagement | Chairperson’s Summary 1 Background and the Objectives of the New Model T H E N E W WO R L D B A N K G R O U P S T R AT EGY (World Bank Group 2013) aims to focus country programs Introduction on accelerating progress toward sustainably reducing poverty and building shared prosperity (the “twin goals”) in a sustainable manner. The strategy sets two ambitious goals: „„End  extreme poverty by reducing the percentage of people living on less than $1.90 a day (in 2011 purchasing power parity terms) to 3 percent by 2030. Promote ƒƒ shared prosperity by fostering income growth of the bottom 40 percent of the population in a sustainable manner in every country. To accomplish this, the World Bank Group would align all its activities and resources to the twin goals; use its strengths as a global institution by supporting clients in delivering development solutions; and work as “One World Bank Group” by leveraging the strength of all constituent agencies (World Bank, International Finance Corporation [IFC], and Multilateral Investment Guarantee Agency [MIGA]). The World Bank Group would strengthen the focus of its country programs by developing a more evidence-based and selective country engagement model in the context of country ownership and national priorities, and in coordination with other development partners. During the past 50 years, since country strategies were first introduced at the World Bank Group by then–President Robert McNamara, they have evolved from internal management tools to public documents that strive to be country-owned, results-driven, and flexible. This evolution is described in appendix A. Briefly, in the initial stage, individual lending operations were put in the context of the World Bank’s program of country activities. The subsequent early country assistance strategies (CASs) were World Bank-centric notes appended to loan documents. The preparation of free- standing CAS documents for all active World Bank borrowers, 1 International Bank for Reconstruction and Development (IBRD) as well as International Development Association (IDA), began in 1992. With the advent of public disclosure in the late 1990s, the CAS process began to include consultations with key stakeholders, and ultimately became the center of World Bank-government interaction. The early CASs focused mostly on World Bank products and activities. As they evolved, they came to reflect more strategic engagement based on country development goals, but in the process they became longer and more complex. Over time, their focus also shifted from inputs to development results. Results-based CASs were piloted between 2003 and 2005, and were mainstreamed after that. The central innovation was the results framework, which specified links between the World Bank’s interventions and long-term development goals (World Bank Group 2005). In addition, in fiscal 2005 management established a four-year CAS cycle with a midpoint progress report. It also introduced the Interim Strategy Note for use when a full CAS could not be prepared, as typically happened in fragile or conflict-affected situations, and it initiated the use of the CAS Completion Report, with review and validation by the Independent Evaluation Group (IEG). Around this time, the World Bank also adopted a new title for the documents—country partnership strategy (CPS)—thought to better reflect a relationship with the country client based on mutual learning and responsibility. As it evolved, the CAS/CPS became more flexible, allowing World Bank teams to customize World Bank services to the country’s own needs and priorities, as well as to changing circumstances. The guidelines now also called for joint World Bank-IFC-MIGA documents in all countries where there was substantial scope for collaboration and synergies in private sector development activities. CASs/CPSs were often implemented in uncertain environments, and thus were subject to risks, and the documents would therefore have a section discussing the main risks and what the World Bank intended to do in case a given risk materialized. However, the 2009 retrospective of CASs (World Bank Group 2009) noted that even when the risks were well and comprehensively articulated, it was difficult to distinguish how the perceived risk had been addressed in the development of the strategy, the instruments, and the choice of interventions. The retrospective also pointed out that a CAS was expected to reflect the existing state of country knowledge; the country program; dialogue with government, donors, and others; and relationships with stakeholders. The 2009 retrospective identified four shortcomings of the CASs/CPSs: „„There  had often been tension between the World Bank’s sector and corporate objectives (such as for infrastructure, finance, and social security) and the country’s vision and priorities. „„ Despite the emphasis on results, some CASs had still taken a transaction-focused approach instead of concentrating on more strategic objectives and expected development outcomes. „„The World Bank and IFC had not operated in unison in countries. „„The  discussion of risks had been weak in many CASs, without explicit links between identified risks and the CAS program or without sufficient discussion of the ways in which risks would be managed. 2 World Bank Group Country Engagement | Chapter 1 Against this background, the new World Bank Group strategy recognized that while the country programs were broadly aligned with the World Bank Group’s mission, they needed to provide countries with more coordinated and focused programs, and activities and interventions needed to be better prioritized according to their expected impacts on the twin goals. To this end, the World Bank would build country strategies based on a systematic diagnosis that identified paths and priorities for a country to achieve the twin goals. The country strategies would build on the alignment with a country’s own priorities and the World Bank Group’s comparative advantages, working as the One World Bank Group, with each agency adapting its specific business model to ensure maximum development impact. The New Country Engagement Model Accordingly, the World Bank Group would establish a more evidence-based and selective country engagement model incorporating two instruments to enhance country engagement, replacing the CAS/ CSP. A new, analytical Systematic Country Diagnostic (SCD) would use data and analytic methods to support country clients and World Bank Group teams in identifying the most critical constraints to, and opportunities for, reducing poverty and building shared prosperity sustainably, while considering the voices of the poor and the views of the private sector. A separate Country Partnership Framework (CPF), developed from the CAS/CSP, would discuss focus areas for World Bank Group support, aligned with the country’s own development agenda, and these areas were expected to be selected primarily to address the key constraints and opportunities identified in the SCD. The SCD would be prepared upstream from the CPF and would identify priority areas for the country to accelerate progress toward achieving the twin goals in ways that consider the sustainability of the program. The SCD would build on available analysis and diagnostic work, with a focus on the twin goals and on identifying the most critical, pressing actions, not on a comprehensive list of policy recommendations. It would be prepared in consultation with national authorities, the private sector, and other stakeholders, but would reflect the evidence and analysis of the World Bank Group. It would become the reference point for client consultations and help focus the country, the World Bank Group, and other development partners on the goals with the greatest potential impact on achieving the twin goals. The CPF has replaced the CAS/CSP as the main tool guiding the World Bank Group’s support for member countries’ development programs. It maintains the World Bank Group’s country-driven model and starts from the country’s vision of its development goals as set out in its own strategy. The World Bank Group and the country then further draw on the SCD, and together they develop the CPF objectives, which will be derived from the country development goals that reflect the World Bank Group’s comparative advantage and align with the twin goals, as identified in the SCD. The new approach is expected to emphasize learning at all stages. During implementation, the country and the World Bank Group are to engage in a continuous process of monitoring and learning. Every two years, or at the midpoint of the CPF period, a Performance and Learning Review (PLR) will be used to adjust the program and to update the Board on progress. The PLR will be used to flesh out details on Independent Evaluation Group | World Bank Group 3 development objectives and the activities necessary to achieve those objectives that were not clear when the CPF was originally prepared or last updated. At the end of the CPF period, a PLR may be used to extend the CPF by up to two years, but no CPF may run for more than six years. At the end of the CPF period, the World Bank Group will engage with the country to complete a self-assessment and issue a Completion and Learning Review, which is intended to serve as both a means to enhance the World Bank Group’s knowledge and to increase development effectiveness and as a tool for accountability. As part of the Completion and Learning Review, the country team will complete a self-assessment of the program’s performance, World Bank Group performance, and how well aligned the strategy was with the twin goals. The self-assessment will be based on the results framework of the most recent PLR, and IEG will validate the World Bank Group’s self- assessments and verify its findings. Working as One World Bank Group—with intra-agency cooperation—is a central part of the new strategy, and a new Regional Coordinating Mechanism has been established under which World Bank Regional vice presidents, IFC Regional directors, and the MIGA vice president will meet quarterly. In addition, there has been regular co-chairing by the World Bank and IFC of reviews of SCDs and CPFs. The World Bank Group is also expected to increase the number of joint projects and to review its portfolio of products and services to improve synergies and eliminate overlap. The CPF is also intended to improve the identification and managing of risks to the World Bank Group program. To this end, the risk section takes a systematic approach to risk, based on the Systematic Operations Risk-Taking Tool (SORT; World Bank Group 2014), to ensure that teams consider all major risks to achieving CPF objectives, as well as any unintended adverse consequences that may be associated with the program. Teams are expected to identify risks and propose measures to manage or mitigate them, and may also discuss the risks of not engaging in the proposed program, thereby highlighting the program’s risk-reward trade-offs. The IEG Evaluation This evaluation is being undertaken at the request of the Board of Executive Directors to provide information about how the SCD/CPF process is working and to identify areas that can be strengthened to enhance country engagement during the production of SCDs and CPFs. It is a real-time process evaluation, designed to provide early evaluative input into the operationalization and rollout of the new country engagement model. Given that the new CPFs are only now being approved, IEG clearly does not have country-level implementation results from the new country engagement model. Instead, it has relied on evidence on steps taken to produce SCDs and CPFs, both from desk reviews and field visits. The evaluation has three objectives: „„ To explore whether the guidelines, directives, and other resource materials related to the new country engagement process have thus far helped World Bank Group teams produce SCDs and CPFs that are focused on helping countries to tackle key constraints to poverty reduction 4 World Bank Group Country Engagement | Chapter 1 and shared prosperity in accordance with the World Bank Group’s comparative advantage, and whether they have fostered coordination among the World Bank Group institutions. „„ Toprovide the Board and management with early information about the functioning of the SCD/ CPF process and the extent to which the shortcomings related to the previous processes have been addressed, so that changes can be made to improve the process. ƒƒTo identify good practice examples and lessons learned thus far. To this end, the evaluation has focused on four dimensions for each SCD and CPF (see appendix B for a detailed discussion of the evaluation methodology). For the SCD, these include the evidence basis and framing of the issues, growth, poverty, shared prosperity, and sustainability; identification of priorities for achieving the twin goals; and process and World Bank Group integration. For the CPF, the dimensions include alignment with country development priorities and the twin goals, and sustainability; country ownership, selectivity, realism, and systematic accounting of risks; quality of results frameworks; and World Bank Group integration. The evaluation questions are designed to explore the extent to which the new country engagement model enhances selectivity and World Bank Group coordination in establishing evidence-based country priorities that are aligned with the twin goals. The core evaluation questions are: „„ Has the SCD identified priorities in a convincing analytical way that helped design a program of selective interventions under the CPF? „„Has  the CPF designed a program of lending and knowledge activities that aligns (not necessarily one-to-one) with country priorities and opportunities identified in the SCD and that makes the best use of World Bank Group resources and comparative advantage, including joint World Bank-IFC- MIGA projects where appropriate? Has the new country engagement model fostered country ownership, realism, and selectivity? Did ƒƒ the SCD and CPF teams have the necessary and appropriate products, systems, and resources, and was this new model helpful in integrating the work of the three institutions, in coordinating with global practices and cross-cutting solutions areas (CCSAs), and with interactions with government authorities and other key stakeholders? The evaluation included desk reviews—using a systematic, detailed template—of all the first 22 CPFs (cutoff, end of April 2016) discussed at the Board and their associated SCDs, and thus with completed initial engagement cycles.1 These are shown in appendix C. (That appendix also shows information for the SCDs and CPFs completed by end of November 2016, outside of the evaluation time frame.) All Regions were covered, with reasonable representation of both IBRD and IDA borrowers, but with only one country for the Middle East and North Africa Region and two each for South Asia and East Asia and Pacific. This analysis was complemented and validated by meetings with task team leaders, World Bank managers, and Regional chief economists, and by in-depth visits to 10 countries—2 per Region, except for the Middle East and North Africa Region and South Asia (1 each), due to their low number of completed initial engagement cycles. The missions had interviews with governments; other development partners; other in-country stakeholders, including representatives of civil society; and World Bank Group managers and staff. In doing these Independent Evaluation Group | World Bank Group 5 assessments, the evaluation team took appropriate note of World Bank guidance documents and any modifications over time in the SCD and CPF processes, and also drew on other appropriate IEG validation, evaluation, and learning products. Evaluation limitations. The evaluation covers the first 22 SCDs and CPFs (based on timing of the CPFs). This is the full universe of products up to the cutoff date of end of April 2016, and a reasonable number of documents to review, with 22 careful desk reviews for each document and 10 field visits. The most important limitations of the exercise are that as an early, real-time formative evaluation, it could not establish the extent to which the new approach is delivering higher-impact development results in pursuit of the twin goals. Thus, it could not assess the realism of forecasts, plans, and expectations against actual developments and could not assess the quality and contributions of applications of PLRs and Completion and Learning Review. Organization of the report. The rest of the report is organized into four chapters. Chapter 2 provides a response to the first evaluation question: Has the SCD identified priorities in a convincing analytical way that helped design a program of selective interventions under the CPF? Chapter 3 summarizes the findings on CPFs. Chapter 4 provides a response to the third question: Did the SCD and CPF teams have the necessary and appropriate products, systems, and resources, and was this new model helpful in integrating the work of the three institutions, in coordinating with global practices and CCSAs, and with interactions with government authorities and other key stakeholders? The concluding chapter summarizes the key findings and provides recommendations for the strengthening of the new country engagement model. 1 T  he evaluation was somewhat delayed from the schedule anticipated in the approach paper because the engagement cycle for a number of countries was completed later than expected. R eferences World Bank Group. 2005. Results Focus in Country Assistance Strategies: A Stocktaking of Results-Based CASs. Washington, DC: World Bank Group. ———. 2009. Country Assistance Strategies: Retrospective and Future Directions. Washington, DC: World Bank Group. ———. 2013. World Bank Group strategy. Washington, DC: World Bank Group. ———. 2014. Framework for Management of Risk in Operations: Proactive Risk Management for Better Results. Washington, DC: World Bank Group. 6 World Bank Group Country Engagement | Chapter 1 2 Background T H E S C D WA S I N T R O D U C E D under the 2013 World Bank Group strategy to enhance (together with the CPF) the World Bank Group’s country engagement model. The The Systematic purpose of the new SCD instrument, according to a corporate Country directive (World Bank Group 2014c), is to inform the strategic Diagnostic dialogue between the World Bank Group and its clients about priority areas for engagement. It is a diagnostic exercise conducted by the World Bank Group in close consultation with national authorities, the private sector, civil society, and other stakeholders. To this end, the SCD is expected to „„ present a systematic assessment of the constraints the country has to address and the opportunities it can embrace to accelerate progress toward achieving the twin goals, not limited to areas or sectors where the World Bank Group is currently active or where it expects immediate country demand; „„ discuss thoroughly the drivers of poverty reduction and shared prosperity, taking into account environmental, social, and economic sustainability; „„ identifya set of priority areas in which follow-up actions could most effectively and sustainably achieve the twin goals; and serve ƒƒ as the analytical foundation and the reference point for consultations when developing the CPF (World Bank Group 2015).1 Further guidelines (World Bank Group 2014a) and guidance (World Bank Group 2014b) have indicated that the Country Management Unit (CMU) will select the World Bank task team leader(s), who will have integrative and analytical skills. Task teams are expected to draw on expertise across global practices and across the World Bank Group. All SCDs are to be joint World Bank Group products, with IFC participating 7 more extensively in countries where it has a deeper presence and country knowledge, and MIGA represented by the relevant Regional economist. The guidelines foresee a robust and contestable review process for the SCDs, chaired by IFC and the World Bank. All SCDs are to be approved by World Bank, IFC, and MIGA managements. The final SCDs are disclosed in accordance with the access to information policies of the three institutions. Evidence Base and Framing of the Issues IEG reviewed 22 SCDs.2 For a few of the very early documents there were modest variations in approach or treatment that could be ascribed to their being first, but there were no systematic differences between these and the subsequent bulk of reviewed SCDs. A majority of SCDs reviewed (13 of the 22) provide a strong basis for identifying the main issues, opportunities, and challenges the countries face in achieving the twin goals. Seven other SCDs were judged acceptable, and only two were assessed as having significant weaknesses in the evidence base or framing of the issues, in one case due to a lack of systematic identification of knowledge gaps and an excessive number of identified priorities, in the other because some potentially important issues were not discussed. Most SCDs do a good job of synthesizing the main issues and constraints facing their respective countries, with a variety of evidence-based analyses. For example, Uruguay’s SCD analyzed the challenges facing its growth model based on global market expansion, wage increases aimed at lowering income inequality, and progressive expansion of a welfare system. Costa Rica’s SCD benchmarked the country against Latin America in terms of poverty reduction and shared prosperity, analyzing trends in growth, poverty, and rising inequality. Panama’s SCD provided a detailed analysis of poverty and factors contributing to growth, and its inclusiveness and sustainability. The reports generally benefited from analytical work by the World Bank and other organizations, in some cases providing for useful cross-country comparisons, particularly for larger countries. In a few cases, special analytical frameworks were used or developed to examine key variables concerning economic growth and poverty. Thus, the Azerbaijan SCD used an asset-based approach to frame the issues of growth and distribution. For the El Salvador SCD, the country team commissioned a computable general equilibrium model to shed light on the mechanisms through which competitiveness, international trade, and prices are affected by remittances. And the Chad SCD used a computable general equilibrium model to analyze a set of policy options, focusing on growth and job creation in specific sectors. SCDs depend heavily on available analytical material and data (World Bank Group 2014a), and most of the analytical underpinnings come from the World Bank’s own work, although some of the SCDs also used external work by donors or governments (Albania and Mali are two examples). Going forward, such examples could be used to encourage broader use of outside analytical work. The guidance asks that teams identify gaps in data required for analysis of the progress achieved and the projected requirements for meeting the twin goals, that they be discussed in the SCD, and that the CPF indicate the support that the World Bank Group will offer to help fill these gaps. 8 World Bank Group Country Engagement | Chapter 2 Many SCDs identified knowledge gaps that could be addressed to improve the evidence base for future policy making; this was a useful input for the analytical agendas in the CPFs. The most frequently reported knowledge gaps concern labor markets, growth and productivity, and access and quality of basic services. The gaps in data (most often regarding household surveys, firm-level surveys, and labor force data) inevitably meant that some SCDs suffered from weaknesses in the presentation of current realities and analysis of the future needs for achieving the twin goals. The evaluation identified two SCDs as good practice examples of the treatment of data gaps. The SCD for the Arab Republic of Egypt (see box 2.1) identified steps needed to fill existing knowledge gaps, such as the need for improved monitoring systems and experimentation with new antipoverty programs. The Myanmar SCD clearly outlined data concerns and knowledge gaps, with plans for addressing them. (The World Bank was not engaged in Myanmar during the period leading up to the SCD, and thus there had been no opportunity to improve the country’s data situation.) Growth, Poverty and Shared Prosperity, and Sustainability Most SCDs satisfactorily analyzed the drivers of economic growth and their impact on poverty and shared prosperity over time, capitalizing on analytic work at the World Bank and elsewhere. Only one Box 2.1 | Egypt’s Need for Stronger Poverty Data and Analysis Some of the evidence base for the critical elements of the analysis—concerning poverty and shared prosperity, macroeconomic analysis, and the private sector’s role in economic growth and job creation—is not sufficiently strong or current. The SCD attempts to present up-to-date analyses of macroeconomic conditions, poverty, and poverty distribution, but it only partially succeeds due to the lack of recent, good-quality poverty data, mainly related to some methodological differences between staff of the World Bank and their country counterparts on the use and interpretation of household survey data since the revolution of 2010–11. In part because of data issues, some important knowledge gaps have emerged. The SCD argues the need for establishing reliable systems of data collection, both quantitative and qualitative, across key social sectors (and the report questions the results of the government’s household income and expenditure surveys). However, in addition to the knowledge gaps identified in the SCD, there are other important analytic gaps. For instance, given the country’s current macroeconomic difficulties and the World Bank Group’s sizable program in support of the country’s ambitious reform program, there is an urgent need to prepare an in-depth macroeconomic and fiscal analysis or a Public Expenditure Review, and to prepare a much-needed poverty assessment (the first in seven years). IEG has been informed that these two products have now been programmed. Independent Evaluation Group | World Bank Group 9 country (Côte d’Ivoire) was assessed as marginally unsatisfactory in this dimension, with limited data availability undermining the comprehensiveness of the growth, poverty, and sustainability analysis. In the other SCDs, patterns of growth were discussed, and factors contributing to poverty reduction were analyzed, such as agricultural growth in Uganda, hydrocarbons in Azerbaijan and Bolivia, diamond revenues in Botswana, and tourism in Maldives—together with appropriate macroeconomic policies. The identification of opportunities for economic growth was the strongest among the dimensions and subdimensions explored. A number of SCDs argued for shifts in prevailing growth models or identified growth potential. For example, the SCD for Albania calls for a shift from the consumption- driven growth model fueled by transfers to a more sustainable, equitable, and export-driven model based on the more effective use of domestic assets. Similarly, growth analyses for Azerbaijan, Bolivia, and Colombia argue that recent growth models, based on commodity exports that depend for their success on an exceptional external environment that is unlikely to be sustained in the longer term, require transition. In Bolivia, transition is needed to a model based on stronger levels of private investment and advances in productivity; in Azerbaijan, to one based on private sector development supported by effective public sector governance and institutions and greater connectivity; and in Colombia, to one addressing institutional constraints, while improving infrastructure and productivity. In general, emerging challenges identified in the countries reviewed relate to a less supportive external environment and the need to address domestic bottlenecks. The analysis of poverty and shared prosperity was the second-strongest among all dimensions and subdimensions. The impact of economic growth on the poor has been analyzed extensively, with focus on the drivers of poverty reduction, such as job creation and labor income, social assistance programs, and equality of opportunity in education and health. Changing trends in poverty incidence have been identified, both in monetary and nonmonetary dimensions, with an emphasis on the vulnerability of the poor across gender, ethnic, and minority groups and rural/urban dichotomies. The poverty analysis is frequently accompanied by regional and international comparisons. However, a number of countries suffered from outdated data and poverty analyses (of the five countries reviewed in the Africa Region, only Uganda had updated data and analyses). The role of social assistance programs has been analyzed in a number of reports as a factor in addressing poverty. For example, in Haiti, private transfers and international aid have contributed to decreasing poverty; in Panama, government transfers have had the biggest role in poverty reduction in rural areas; and in Bosnia and Herzegovina, Botswana, and Honduras, social assistance programs have been important. Appropriate focus was given to the analysis of shared prosperity and the poorest 40 percent of the population, but generally without in-depth quantitative analysis. In some countries (Bolivia and Botswana), growth was found to be associated with reduced inequality, while in others (Indonesia, the Maldives) it was associated with increased inequality (see box 2.2). Reports usually analyzed the factors contributing to changes in income inequality, with emphasis on vulnerable groups in society. For very poor countries, there may be little point in analyzing separately the poorest 40 percent, since poverty rates may exceed that number. For example, 60 percent of Haiti’s population remains poor, in Bangladesh the poverty rates are still high enough for the twin goals to largely overlap, in Mali the 10 World Bank Group Country Engagement | Chapter 2 incidence of dollar-a-day poverty exceeds 40 percent of the population, and in Uganda 30 percent of the population is in extreme poverty. In such cases, SCD analysis of overall poverty reduction supersedes the focus on the poorest 40 percent. In most countries, however, focusing also on the poorest 40 percent remains relevant. Factors contributing to changes in inequality were identified. For example, in Uruguay, reductions in inequality were associated with reductions in school fees, progressive fiscal policies, and wage increases. In Indonesia, inequality of opportunity and education were key factors in increases in inequality. In the Maldives, an increase in inequality was attributed to limited job opportunities. In Costa Rica, increases in inequality (which was low by regional standards) underscore a large deficit in human capital. Fiscal, environmental, and social sustainability aspects were adequately addressed in all countries except Mali and Côte d’Ivoire. In these cases, important issues were not always given equal attention. For example, in Mali the focus was on social and political stability, but much less so on climate change. Social sustainability was generally addressed well (for example, in El Salvador it was linked to high levels of crime and violence). Environmental sustainability and/or climate change were addressed well or adequately in 19 of the 22 reviewed SCDs, and such issues were listed among Box 2.2 | Impact of Economic Growth on the Poor in Indonesia The SCD finds that commodity-fueled economic growth led to massive job creation in 2001–12, especially in the services sector, and particularly in urban areas. While job creation helped reduce poverty (more than halving it to 11.3 percent by 2014), it did not reduce vulnerability, because most of the new jobs were in low-productivity sectors where real wages remained stagnant for the bottom 40 percent of the wage distribution during 2001–14. Widening wage disparities have added to the increase in overall inequality. Limited workers’ protection (nearly 80 percent of total employment is potentially vulnerable, and women tend to be more vulnerable than men due to differences in job status) leave many Indonesians vulnerable to shocks and add to rising inequality, undermining recent welfare gains and shared prosperity. The bottom 40 percent has benefited far less from economic growth than the rest, and the Gini coefficient increased by almost 12 percentage points during 2000–13, from 30 to 42 (the latter is likely higher). Indonesia’s shared prosperity record is poor compared with other countries in the Region. The SCD identifies education as a key factor in explaining inequality in Indonesia, and the most important factor enabling upward mobility. Other factors identified as contributing to inequality are the distribution of capital- asset ownership and the decline of labor’s share of national income. Overall, a third of inequality is due to inequality of opportunity (circumstances that children are born into). Independent Evaluation Group | World Bank Group 11 Box 2.3 | Indonesia’s Prioritization Process The SCD identified numerous reforms under three broad priority areas: economic growth and jobs, access to key services and opportunities for all, and natural resource management and environmental sustainability. These three priority areas are supported by cross-cutting prerequisites: domestic revenue mobilization and governance reforms. Reforms have been prioritized based on impact (evidence-based), complementarity/synergy, sequencing, and feasibility. While the process methodology is elaborate, the large number of high-priority and transformational policies identified, particularly when added to the cross-cutting prerequisites, however appropriate, weakens the argument of addressing the most critical issues in prioritizing transformative policies. The great number of priorities identified (24) dilutes their potential impact. No effort has been made to discuss systematically the synergies among the many priorities. the identified priorities in 18 SCDs. Vulnerabilities were identified, and special factors in the economy were analyzed (Bolivia’s ethnic diversity, for example). Threats to social and political stability were well analyzed for Chad, where despite overall reductions in poverty, the average income of the bottom 35 percent has fallen, as well as for Uruguay, where issues of an aging population combined with a policy focus on equity and a large middle class. However, not all SCDs—across the risk spectrum—provided a sufficiently robust analysis and identification of relevant social or political risks. The Indonesia SCD, for example, appeared to underplay the significance of rapid urbanization, increased inequality, and the impact of slower economic growth on social stability. Identification of Priorities for Achieving the Twin Goals All SCDs identified priorities for achieving the twin goals. Yet identification of priorities was the weakest of the four dimensions assessed. There was significant variability among country teams in the methodology developed or applied to derive priorities and on the number and nature of priorities identified. These were often at a high level of generality—areas to be addressed rather than specific actions to be taken. For some countries, priorities were specific and limited in number (six priority areas in Chad, five in Panama, two broad priorities with eight actions in Uruguay). In others, numerous priorities were identified (24 in Indonesia, see box 2.3; and 8 priority areas and 36 actions in Uganda). Identifying a large number of priorities goes against the expectation that an SCD would identify the most critical, pressing actions, and can easily undermine attention and impact. The focus on priority areas is reasonable if there is follow-up to define more specific policy actions through analytic work and as part of the policy dialogue. 12 World Bank Group Country Engagement | Chapter 2 Some SCD task team leaders (as in Bolivia and Uganda) indicated that they received conflicting instructions on the degree of specificity of the selection process and did not get clear guidance from management on how to prioritize among competing inputs (including those from various global practices and CCSAs). Also, some SCDs discussed policies and policy alternatives (as in Indonesia), whereas others focused largely on broader structural issues. As mentioned earlier, different methodologies were used to identify priorities, some quite sophisticated or complex (in Azerbaijan, Chad [box 2.4], and Indonesia). Consultative processes often were used to help select priorities. While recognizing that one size does not fit all, clearer guidance is needed on how to identify key priorities in the SCDs. In general, documents did not provide any ordering of the identified priorities on the basis of their impact on the twin goals (except for Botswana and Chad)—this is not a problem if the list of priorities is clearly selective, focused, and well argued. The high-level nature of priorities identified in most SCDs meant that there often was no clear discussion of the paths that could lead from interventions to the desired result, although such paths were discussed for Azerbaijan, Bolivia, Egypt, Indonesia, Panama, and Uruguay with clear constraints to achieving the twin goals and the corresponding set of policy priorities. Most reports did not quantify their aspirations for economic growth, poverty reduction, or shared prosperity. Governance was highlighted in most SCDs as a cross-cutting theme or prerequisite for achieving the twin goals. It was candidly discussed in the SCDs for Myanmar (box 2.5) and Indonesia. In the Box 2.4 | Use of Analytic Approaches in the Selection of Priorities in the Chad SCD The SCD uses a computable general equilibrium model to identify policy options focused on growth and job creation in “poor-intensive sectors.” One set of options relates to raising agricultural productivity. A second set is based on a decline in oil receipts and the need to develop new export markets to sustain economic growth. A third set looks at improving the efficiency of public service delivery through governance reforms. These reforms could reduce poverty by about 3 percentage points by 2030—from 29 percent to 26 percent. (This is one of the few examples of quantified projections of possible impacts in the SCDs.) The SCD then looks at the potential impact of cash transfers to protect against shocks and speed the pace of poverty reduction. It discusses various types of social safety nets, including labor-intensive public works. The SCD uses a tailored version of the Rodrik-Haussmann growth diagnostic framework to identify the binding constraints to achieving the two core poverty reduction approaches—growth in the agriculture sector and income transfers to the poor. This results in the identification of 15 constraints. The SCD then uses a decision tree with a set of filters to identify the constraints that meet key criteria: short-term impact, political feasibility, implementation capacity, affordability, evidence base, and complementarities. Independent Evaluation Group | World Bank Group 13 Box 2.5 | Treatment of Governance in Myanmar The SCD presents a candid picture of governance issues, including a history of cronyism and military control; lack of transparency, particularly with natural resource extraction and associated revenues; armed conflict in certain areas; limited institutional capacity; a weak Public Finance Management Project; and lack of Environmental and Social Impact Assessment requirements and environmental/social regulations pertaining to natural resource extraction. It also mentions public sector capacity constraints in certain areas, but points out the need for more in-depth understanding of capacity constraints in each sector and for a better understanding of the impact of politics on decision making within the bureaucracy. latter case, governance was seen as a factor leading to the overexploitation of natural resources, land rights, and allocation issues, and the SCD prioritized reforms to improve policy recommendations, project implementation, and transparency, as well as to increase user participation and sustainability of investments. However, a clear approach to governance was missing in some SCDs. A number of the SCDs could also have benefited from more systematic cross-country governance comparisons. Process and One World Bank Group IEG assessed the internal World Bank Group processes for SCD preparation and review and implementation of the One World Bank Group concept as less robust than the more substantive areas evaluated, though still satisfactory. Internal processes typically started with country team brainstorming meetings, and continued through the concept note to the Regional Operation Committee (ROC) decision review meeting, which usually resulted in numerous constructive, high- quality comments from different parts of the World Bank Group. The corporate review process was broadly satisfactory, enhancing the quality of reports. The global practices were heavily engaged in the processes, both through participation in the large country teams and through formal review meetings. however, task team leaders interviewed by IEG indicated that the engagements with the global practices could be complex. in some cases, the global practices went beyond reasonable and expected advocacy to enlist senior management to press their case. Some task teams also mentioned pressure from the CMUs to align their diagnoses with aspects of ongoing or planned programs. Occasionally, peer reviewers were engaged in the discussion of SCDs (Haiti, Uganda). Task team leaders indicated that they had good access to the requisite corporate guidance when they required clarification of the guidelines and procedures. The One World Bank Group approach systematically brought together the World Bank, IFC, and MIGA during concept review, document preparation, and the review process, with the intention to 14 World Bank Group Country Engagement | Chapter 2 improve strategic directions through the SCDs and bring the full range of World Bank Group expertise to bear on country analysis. The approach has fostered multisectoral consultations between the World Bank and IFC, with the latter providing analyses of private sector issues and prospects in some cases. IFC’s role in promoting corporate governance and improved labor and working conditions has also provided useful additional perspectives to the analysis in some SCDs. Also, its short industry notes for several SCDs facilitated the prioritization process there, and also in the subsequent CPFs (Bolivia, Colombia, El Salvador, Haiti, and Panama). In the Albania and Indonesia SCDs and CPFs, IFC led and moderated consultations with the private sector. SCDs are written by World Bank staff, with inputs and comments from IFC. World Bank task team leaders generally found considerable IFC engagement in the SCD processes, despite institutional challenges and different operational modalities. Nevertheless, IFC’s contributions in the final document may appear modest because of a decision to focus on high-priority activities, it has a small presence, or where there is limited private sector interest. MIGA’s small size, lack of country presence, operational model, or strategic considerations often did not allow for its extensive participation in the process. A related review of the treatment of private sector development (PSD) issues in SCDs found that the assessment of PSD constraints and opportunities was World Bank-centric, analyzed mostly from the policy perspective, and lacked business-oriented analysis of the private sector and what it needs to contribute to growth and the twin goals (see appendix G). The heavy public sector tilt in the assessment of PSD constraints and opportunities also reflects the World Bank’s lead role in drafting SCDs. IFC team leaders for the SCDs considered their contributions as “inputs” rather than as the basis for fresh thinking about the needs and opportunities for private sector growth. Perhaps as a result, the IFC programs comprise primarily the standard package of finance for small and medium-size enterprises (SMEs) and skill training. Private sector analyses in the SCDs need to move toward a more business-oriented and granular analysis of a country’s private sector, which could selectively provide background on the structure and composition of a business sector, differentiating the industry-specific constraints and opportunities according to the types and sizes of business enterprises. In consultations with stakeholders, at the stage of early dialogue, teams often took time to explain the purpose, nature, and desired outcomes of the process and the product. Box 2.6 describes a good example (Albania) of an extensive consultation process. Some stakeholders expressed their appreciation to IEG for the participatory process, while in other countries some stakeholders (aside from government authorities) would have liked to see greater interaction and feedback. SCDs normally benefited from but did not report on comments received from stakeholders. In Haiti, the World Bank sponsored a competition of academic papers to foster research and debate, as well as to identify issues that might have been overlooked. In some cases, government interference with the consultation processes caused some delays and may have compromised their quality (Azerbaijan and Bolivia), while probably affecting how the World Bank presented its identified priorities only at the margin. Independent Evaluation Group | World Bank Group 15 Box 2.6 | Albania’s Consultation Process A notable feature of the process in Albania was the team’s careful attention to ownership through consultation with a wide range of national stakeholders at all stages of SCD development and a systematic effort to provide participants with feedback. A second round of consultations with national stakeholders and external partners was conducted in late 2014 following completion of the draft report, around the time of the ROC discussion. The consultations appear to have enhanced the broad ownership base of the document by informing and guiding discussions and engaging representatives of different population groups on priorities for World Bank Group support. The two- round consultation design also allowed the second round of consultations (with a large overlap of participation in the two rounds) to serve as a feedback mechanism regarding comments and suggestions from the first round. A number of the reports discussed elements of the government’s strategic priorities, but usually without in-depth analysis. In some cases, the SCDs were developed in parallel with preparation of a new government strategy or medium-term development plan. In Albania and Azerbaijan, country teams viewed this as a decided plus, enabling a meeting of minds about development priorities and enabling the World Bank to play a role at the formative stage of the government’s thinking. In other cases, however, the government felt strongly that it wanted an opportunity to develop its strategy independently of the World Bank’s analysis, and then to engage in a dialogue comparing the two approaches. The Panama SCD, for example, was prepared a few months before a new government was able to complete its own strategy and the government felt that, as a result, it was not in a position to engage sufficiently with the World Bank in this process. In El Salvador, the SCD was prepared in parallel with the preparation of the government’s five-year plan, with substantial interaction with the World Bank. On the positive side, there was cross-fertilization between the World Bank and the government in the development of priorities, but there was not enough time for the government to absorb the World Bank’s findings or integrate them into its own priorities. In addition, some World Bank task team leaders expressed concern that a parallel process could lead to questioning whether the SCD reflects the World Bank’s own views on development priorities. Clearly the optimal timing of the SCD relative to the development of the government’s own strategy needs to be determined for each country context and through close consultation with the national authorities. The SCD documents often break new analytical ground for a country, which gives them a significant public good aspect. All SCDs have been made publicly available by the World Bank Group, in some cases with a time lag. A number of development partners noted the usefulness of the SCDs for the 16 World Bank Group Country Engagement | Chapter 2 formulation of their own country programs. Stakeholders in the Azerbaijan government noted the importance of the SCD in introducing the concept of shared prosperity into domestic policymaking and, in Uruguay, the SCD was seen as a public good, leading to its broader dissemination. However, IEG’s consultations with stakeholders also indicate significant weaknesses in the extent to which the SCD is shared and discussed, with a considerable number of stakeholders not having seen or read the reports, even when they were made publicly available on the World Bank website and/or had been translated from English. The SCD, beyond its function as an input to the CPF, provides an important opportunity to help engender debate and to define an agenda for strategic dialogue on key development issues in a country, and thus also to enhance broader country ownership. It is therefore important for the World Bank Group to engage systematically in the dissemination of this report beyond meeting its own disclosure requirements—and much of that dissemination can be done at moderate or little cost. Plans for dissemination beyond disclosure need to be part of the SCD process from the start, in support of the World Bank Group strategic dialogue and to enhance broader country ownership around key development priorities. Overall, the analysis did not find any systematic differences between SCDs for IBRD and IDA countries (listed in appendix C, table C.3), although the average ratings tended to be slightly lower for the IDA countries for most categories. Data availability was more of a constraint for the SCDs in IDA countries, but teams typically found ways around this issue, including looking for outside information sources. One notable difference is that the focus for many IDA countries is on poverty rather than shared prosperity, due to the large percentages of the poor in many of these countries. Emerging Messages Overall, the new SCD instrument has been a useful addition. For the most part, this first round of SCDs has synthesized the main issues and constraints for a country well, with a variety of evidence- based analyses. This instrument clearly reflects the World Bank Group’s comparative advantage, given the depth of its country experience, broad engagement, and analytic depth relative to other stakeholders and development partners. The SCDs have been well received both inside and outside of the World Bank Group, including by governments and other donors. Internally, the preparation of the first batch of SCDs has been a learning experience. The internal processes have been broadly satisfactory, with numerous consultations within the World Bank at various stages of SCD preparation and review. The corporate reviews have helped enhance the quality of reports, while underlining and supporting the One World Bank Group approach. Integrating the work of multiple global practices with that of CMUs in the preparation of the SCDs has sometimes caused (manageable) friction. The One World Bank Group approach has created the impetus for greater integration of the World Bank, IFC, and (more selectively) MIGA, with positive initial results. However, this integration needs to be broadened beyond countries where IFC and (in a few cases) MIGA currently have major Independent Evaluation Group | World Bank Group 17 engagements. The treatment of the private sector in economic growth and poverty reduction has become more prominent than in previous CAS/CPS documents. SCDs depend heavily on the available analytical material and data. It is therefore important that future SCDs explicitly address knowledge gaps and data limitations, and that these issues be picked up in the CPFs. All SCDs have identified priorities for achieving the twin goals, but with significant variability between country teams on methodology, number of priorities, and degree of specificity, although mostly at a high level of generality. The World Bank Group guidance for priority setting could be improved to encourage more consistent approaches around a limited number of priorities. An important conclusion of this assessment is that the SCD should be seen as transcending its purpose as an input to the CPF, assisting strategic dialogue with key stakeholders and with an important public good dimension. This underlines the need for proactive dissemination. 1  lignment with the priorities of the SCD is one of the criteria for identification of World Bank Group program priorities A in the CPF. The Operations Policy and Country Services note on procedure (World Bank Group 2015) indicates that the SCD is conducted prior to the CPF process and is finalized before the CPF concept review meeting takes place, in order to provide the analytical foundation for the CPF. 2 All 22 SCDs are listed as a group in the bibliography. R eferences World Bank Group. 2014a. Interim Guidelines for Systematic Country Diagnostics. Washington, DC: World Bank Group. ———. 2014b. Systematic Country Diagnostics: An Overview of Current Thinking. Advisory Group for SCD. Wash- ington, DC: World Bank Group. ———. 2014c. World Bank Group Directive, Country Engagement. Washington, DC: World Bank Group. ———. 2015. World Bank Group Procedure: Country Engagement. Washington, DC: World Bank Group. ` 18 World Bank Group Country Engagement | Chapter 2 3 Background U N L I K E T H E S C D, the CPF is a well-established World Bank product. Nevertheless, the new engagement model requires some modifications to the traditional CAS/CPS. The The Country model builds on the SCD’s analysis of the country priorities Partnership for the achievement of the twin goals: accelerating progress Framework toward sustainably reducing poverty and boosting shared prosperity. The CPF identifies areas within the SCD’s framework of constraints and priority actions where the World Bank program can most effectively support the government’s own priorities. This explicit link between the World Bank program and the achievement of the twin goals is a departure from the more generalized support to growth and poverty reduction that framed the CAS/CPS programs. Another innovation in the new model is the emphasis on integrating the World Bank, IFC, and MIGA as One World Bank Group in the results chain linking operations to the achievement of the defined objectives. In addition, the SORT was adopted for use in CPFs.1 The World Bank, IFC, and MIGA issued a guidance note (World Bank Group 2014) on developing CPF products for World Bank Group engagement with member countries. An important focus of this evaluation was to assess whether the guidance provided was adequate, well understood, and implemented. Identification of Priority Objectives for World Bank Group Support The guidance note identifies three criteria for the selection of World Bank Group priorities: alignment with the priorities identified in the SCD; alignment with the government’s 19 FIGURE 3.1 | CPF Selectivity Filters  CPF Objectives Country Twin Development Goals Goals Bank Group Comparative Advantage Source: World Bank Group. 2014, 10. Note: CPF = Country Partnership Framework. own priorities as defined in key strategy documents; and the World Bank Group’s comparative advantages. These are shown in a graphic in the guidance note, suggesting that the Group’s objectives are a subset of those that meet all three criteria (figure 3.1). Most CPFs referred to these criteria, and some provided a discussion of each of them. In practice, there was clearly also a fourth criterion—the priorities reflected in the World Bank Group’s ongoing operations program—whether or not these were consistent with the other criteria. Alignment with the SCD. Aligning the program priorities of the CPF to the constraints and priority areas identified in the SCD is challenging. The SCD and CPF have very different purposes. The SCD’s purpose is to assess what steps the country needs to take to achieve the twin goals by 2030. The purpose of the CPF, in contrast, is the World Bank Group’s own program of support over a five-year period, focusing on the areas where the World Bank Group can maximize its value added to the government’s program. The benefits of thinking ahead about the constraints to achieving poverty reduction and shared benefits in the SCD are shown by the good quality of the summary discussions of constraints in the CPF text. It is also evident from the reviews that the focus on the achievement of the twin goals in the SCDs has driven the CPFs to zero in—much more explicitly than in the past—on what programs are required to support this. However, the different time frames of the two documents create a tension. An important new element in the engagement model is guidance provided to SCD teams to identify the gaps in knowledge and data needed to assess both the incidence of poverty and the constraints to achieving the twin goals. The CPF guidance asks for an indication of how the gaps will be filled and 20 World Bank Group Country Engagement | Chapter 3 in what time frame and for the arrangements and capacity for enhanced poverty monitoring. While, as discussed, the SCDs generally provided a clear identification of the key knowledge and data gaps, the CPFs were not as effective in following up on those gaps. The credibility of the SCD’s analysis depends to a considerable extent on the reliability of the data on which it is based. The CPFs need to do a much better job of looking at each of the knowledge and data gaps and determining how they should be addressed, reflecting the respective roles of the corporate units, CMUs, and global practices in the World Bank Group and the role that external partners might be expected to play, and in what time frame. This is not just a matter of collecting baseline data. It includes the need to put in place the capacity for enhanced poverty monitoring (see box 3.1). Data issues will be discussed in two upcoming IEG evaluations. The SCD identification of development constraints is important in enabling the CPF to pick up on those constraints and focus the World Bank program, but the guidance is clear that one-to-one alignment is not expected. There was evidence in a number of CPFs (such as Uganda) that the identification of key focus areas in the SCD had resulted in a significant shift of emphasis in the World Bank program. However, one major factor driving nonalignment of the SCD and CPF was the government’s view of where the World Bank should focus its support (such as Côte d’Ivoire and Panama, box 3.2). Other important factors were the weight of the ongoing program (Chad) and the quality of the SCD analysis. As discussed in chapter 2, some SCDs did not do a good job of identifying clear priorities and analyzing the trade-offs. In those cases, the CPF teams tended to focus on alignment with the government’s program. While most CPFs had a table or a paragraph comparing the SCD and CPF priorities, in general they did not do a good job of explaining the reasons for the differences between the two. There would be a difference between a CPF not including some SCD priorities, or adding priorities not in the SCD, but in either case it would be useful for the CPF to explain the differences and to acknowledge explicitly any remaining SCD priorities and briefly explain the reasons for leaving them out. This could be done either through more detailed text discussion or a brief annex that more systematically explained all the identified SCD priorities. Alignment with the government’s own priorities. The second criterion for selection of World Bank Group program objectives is alignment with the government’s own program. The CPF is a unique opportunity to engage with the differences in emphasis or approach between the government’s strategy and World Bank Group’s view of strategic priorities and to discuss these differences. The discussion of this alignment, or lack of it, was a weakness in the CPFs reviewed. In many cases the priority areas of the government plan were listed, and followed by a pro forma statement that the World Bank Group objectives were aligned (Haiti, Honduras, Maldives, and Mali). It is also clear that even where there is superficial alignment reflected in the identification of pillars in the government strategy, the weight that governments give to growth in the near term relative to poverty reduction and shared benefits may be different from that of the World Bank Group. In some countries, this is indicated by the prominence of national-level infrastructure in the government plan relative, for example, to the World Bank Group’s emphasis on growth of SMEs, smallholder agriculture, and human capital development (Egypt, Uganda). Sustainable poverty reduction requires both growth and shared prosperity, and there should be discussion of the appropriate balance during Independent Evaluation Group | World Bank Group 21 Box 3.1 | Addressing the Need for Enhanced Poverty Monitoring The guidance (World Bank Group 2014) asks that the CPF discuss the capacity to monitor poverty indicators, but this was rarely done. Nearly a third of all the CPFs reviewed rated low in this regard. A number of the SCDs provided a good discussion of the gaps in development data and of the country capacity to undertake household surveys, carry out poverty analysis based on survey data, and make the data and results publicly available so that they can be used by others. In general, however, the CPFs did not follow up. In Bosnia and Herzegovina, for example, the SCD’s excellent discussion of data issues and identification of key areas for improvement is not followed up in the CPF, where there is no indication that the program would seek to support poverty data collection and analysis. Since this is so central to the new engagement model, it seems that more explicit guidance is needed through the inclusion of an annex to the CPF identifying the knowledge and data gaps and how they will be filled and whether capacity building is required to sustain poverty monitoring. Box 3.2 | Divergences Between the SCD and CPF Priorities In Panama, the CPF priorities diverge substantially from those identified in the SCD. While education, housing and health, and land management and environment are priorities in the SCD, they are not in the CPF. At the same time, the SCD’s focus on the implementation of the National Integrated Development Plan of the Indigenous Peoples of Panama is not a priority in the CPF. While it is understandable if the government’s priorities do not match those of the SCD, the CPF’s focus on areas that were not identified in the SCD as priorities—such as urban connectivity and financial inclusion— challenges the notion that the CPF objectives concentrate resources where they will have the most impact on the achievement of the twin goals. In Côte d’Ivoire, the team preparing the CPF concluded that the SCD had failed to take sufficient account of some priority areas that were critical for achieving the twin goals. The CPF therefore included some areas that were not part of the priorities identified in the SCD. 22 World Bank Group Country Engagement | Chapter 3 the program period and its implications. In the best cases, there was a healthy dialogue to clarify differences of view (Albania, Azerbaijan). Thus, in Azerbaijan the CPF recognizes that government ownership is more critical to the success of the program than perfect alignment between the SCD and CPF. An important element in achieving alignment was the timing of the SCD in relation to the government’s own plan. In the traditional CAS/CPS model, the presumption was that for consistency, it was preferable to wait for the completion of the government’s plan before preparing the World Bank’s strategy. However, evidence is emerging that in the new model, there can be benefits from the preparation of the SCD before or during plan preparation. In some countries, this enabled dialogue when there was sufficient flexibility to develop a well-coordinated approach. In Bosnia and Herzegovina, for example, the key areas of emphasis in the government’s reform agenda were substantially informed by the SCD process. Similarly, in Albania both World Bank-prepared Policy Notes and the SCD were important inputs into the government’s National Strategy for Development and Integration, resulting in close alignment with the CPF priorities. The World Bank Group’s comparative advantage. All CPFs indicate that the World Bank’s comparative advantage was a criterion for the selection of priority objectives. In practice, however, they refer to absolute advantages, such as the World Bank’s convening power or its ability to combine analytic work with operational support. The World Bank’s comparative advantage depends not just on its own capacities, but also on those of the government and development partners. The treatment of development partner programs was uneven. It tended to be better in the Latin America and the Caribbean countries where the World Bank is often a smaller lender than the Inter-American Development Bank and Development Bank of Latin America (Corporacion Andina de Fomento), so that it needs to be selective in identifying areas where it can add value. In El Salvador, for example, the preparation of the strategy took into consideration extensive areas of engagement by development partners. As a result, some objectives identified in the SCD, such as support for the judiciary, reducing crime and violence, and disaster risk management, were not included in the program. In Serbia, a careful examination was undertaken of the World Bank Group’s comparative advantage relative to other partners, especially the European Union, European Bank for Reconstruction and Development, and European Investment Bank, both in terms of areas where each institution may have leverage and the record in program implementation. In some cases, the CPFs indicated that particular areas that were both SCD priorities and consistent with the World Bank Group’s comparative advantage were not selected because the government was not interested in World Bank participation in those areas, such as education, health, and social protection in Bolivia. In this and other cases, the World Bank has reshaped its support in areas that are critical for the eventual achievement of the twin goals to focus on advisory services and analytics. There are also examples where the World Bank felt that it did not have a comparative advantage in a particular sector, but decided to remain engaged because of the interest of the government in continued World Bank support, such as in the water sector in Uganda.2 Independent Evaluation Group | World Bank Group 23 The ongoing program. Among the most difficult issues for country teams in preparing the CPFs was the relation between the SCD priorities and the composition of the ongoing lending program, including the pipeline of projects on which significant preparatory work had been done. In some cases, where the priorities had not shifted much since the previous strategy document, as in Côte d’Ivoire, these were already well reflected in the ongoing program. But many SCDs identified some new priority areas that were not prominent in the World Bank’s ongoing program. In some cases, the objectives implicit in one or two of the projects in the “legacy” portfolio were not covered by the SCD priorities. This presented country teams with a dilemma and, according to many participants, much of the internal discussion on the CPF related to this question. Some country teams dealt with this by papering over the cracks—either the SCD had a long list of priorities that were consistent with almost any set of World Bank–supported programs (Myanmar—albeit a country in a new and special situation), or CPF objectives were categorized in such general terms that they could contain both the ongoing program and the SCD priorities. In the Botswana CPF, for example, only three objectives are defined, but they are: private sector–led, job-intensive growth; effective resource management; and strengthened human and physical assets. This reflects the approach of the previous CAS/CPF of defining broad pillars, rather than to focus on the more specific objectives implied by the new engagement model. There is little evidence in the CPF documents of a serious rethinking of the ongoing program to assess whether it remains relevant. For the most part, it is appropriate that the ongoing program continues to be implemented with a view to achieving development results, although there may be opportunities for canceling or scaling back low-priority or poorly performing operations. But if the priorities are bent to conform to the ongoing program, then the World Bank runs the risk of a sector composition not responsive to addressing the twin goals. The Bangladesh CPF was among the few to distinguish clearly between the “foundational” programs and the “transformational” priorities that were to be supported in the outer years of the CPF. This allowed for a discussion of how the program of World Bank–supported advisory services and analytics in the initial period would set the stage for subsequently addressing the transformational priorities. The increasing interest emerging from the SCDs in the development of social protection systems in some of the poorest countries— those unlikely to meet the twin goals by 2030—is an important instance of the potential benefits of upstream thinking about downstream priorities (Chad, Mali, Uganda). (See box 3.3.) Selectivity Past IEG reviews of CAS/CPS documents have identified the need for more selectivity in World Bank objectives. Selectivity in the CPF can be tracked at three points in the strategic programming cycle. The first is in the number of objectives and the specificity with which these objectives are defined. The second is in the number of objective indicators put forward as proxies for the achievement of the objectives. The third is in the number of World Bank Group operations designed to support the achievement of the objectives. A comparison of the current CPS and the most recent CAS/CPS for the country was used to inform this review. 24 World Bank Group Country Engagement | Chapter 3 Box 3.3 | Matching the CPF Program with the Long-Term Priorities of the SCD An important finding of the SCD preparation is that for most fragile states and low- income countries, even pro-poor growth will not be sufficient to achieve the twin goals by 2030. This will require income redistribution programs through social transfers. The problem is that in the CPF time frame, many of these countries lack the capacity to identify the poor, to finance transfers, and to put in place the governance arrangements that ensure that the transfers reach those who need them. In general, the CPFs do not do an adequate job of indicating what analytic work and technical assistance needs to be carried out in the CPF period to build capacity to implement a meaningful social security system 10 or 15 years from now (Chad, Uganda). There was a modest decline in the number of objectives in the nine countries where comparisons to the past were made (from 122 to 106). However, the number of objectives in itself is not a particularly useful standard of comparison. As the Botswana example shows, some CPFs simply broaden the scope of the objectives so that they encompass a wide area of the program. The objective indicators are a better reflection of selectivity, since these need to be defined in specific, quantifiable terms and their achievement will be used to assess the overall effectiveness of Bank support. Here the progress is striking. For the nine countries, the number fell from 304 indicators in the previous CAS/CPS to 218 in the CPFs. The trend for the number of World Bank Group operations is more difficult to analyze. A program needs to encompass the operations required to support the achievement of the objective. In some cases, however, long lists of operations are provided in support of an objective, but many of these are clearly subsidiary for the achievement of the objective. In Côte d’Ivoire and Uganda, there are objectives linked to more than 20 separate World Bank Group activities. To give another example, whether or not the SCD includes road transport as a priority, the ongoing projects in the subsector are slotted into objectives such as “supporting PSD” or “increasing access of the public to health and education services.” At the same time, the CPF format does not give enough attention to institutional development and capacity-building programs that are worth supporting, even if they cannot be linked to the definition of quantifiable outcomes in the CPF time frame. Examples include long-term engagements in building statistical capacity or promoting the development of think tanks that can support evidence-based policy making. The Results Framework The key elements of a good results framework include the definition of a clear results chain linking the objectives to monitorable and measurable outcome indicators. IEG’s past reviews of CAS Completion Independent Evaluation Group | World Bank Group 25 Reports had indicated that many CAS/CPFs were deficient in setting out a clear results chain and in developing a set of quantified indicators that are good proxies for achievement of the objectives. The review of the CPFs suggests that the definition of the results chain remains a work in progress, and that their results chains can best be described as truncated, with several items missing, as discussed in appendix E. The framing of the guidance note (World Bank Group 2014), which asks for the “intervention logic” to be specified in the results framework, has encouraged more attention to the links between the objectives (normally defined in broad directions) and the particular objectives indicators selected as proxies for their achievement. In general, the reviews found the upstream part of the results chain to be well handled, although some gaps between achievement of objectives and achievement of indicators must be expected. However, the role of the World Bank Group’s program in contributing to the achievement of these outcomes was often not well thought through and explained. It is worth noting the commendable effort made by some team leaders to recognize that the contributions of other donors were also important for the achievement of the results specified in the World Bank Group’s results framework, but such contributions are rarely reflected in the CPF results chains. The definition of monitorable and measurable results for the indicators was a strong point of many of the CPFs reviewed, with more than half of the objectives indicators classified by IEG as outcomes. The reviewers rated this area highly. Another key element has been the distinction between objective indicators, which reflect outcomes or outputs, and supplementary indicators, which are often inputs or process steps on the critical path toward achieving the outcomes. While these process steps are often necessary conditions for the achievement of subsequent outcomes, too often CPFs are content to leave it at that, without thinking through what can be done to assess whether they are actually contributing to progress in the field. To give an example, the Chad CPF identifies inputs such as training of officials in the Ministry of Petroleum or delivering subsidies to community teachers. These indicators are of limited value, however, without complementary evidence on whether those trained were still working in the same field by the end of the program and survey data on whether the trainees found the training useful. Similarly, the impact of subsidies on teacher absenteeism could be tracked. In most programs, there is still a certain amount of cleaning up required, and some indicators are too ambiguous in their definition. See box 3.4 for some examples. The tension, discussed above, between the World Bank Group’s ongoing program and some of the new priority areas defined in the SCD comes to a head in the definition of the results framework. The requirement for a measurable and monitorable results framework means that indicators tend to be drawn mainly from the existing World Bank Group support program. With a five-year program, the planned results for the first three years are generally accounted for almost entirely by the existing program. It is virtually impossible, at least for investment lending, for planned new activities to be reflected significantly in the results framework in that time frame. In practice, a number of results frameworks confine themselves to indicators for the first two to three years, and thus do not reflect potential new activities. This gives the wrong set of signals. Although the guidance note recognizes 26 World Bank Group Country Engagement | Chapter 3 Box 3.4 | Examples of Ambiguous Indicators Despite the progress that has been made in defining measurable objectives, there are still too many instances where CPFs include indicators where the quantification has little or no meaning, for example: In the Maldives: “50 percent of targeted island representatives demonstrate ƒƒ improved capacities to plan and manage multisectoral measures” (World Bank Group 2015c, 38). In Indonesia: “Number of households receiving improved access to reliable ƒƒ energy” (World Bank Group 2015b, 27). In Colombia: “Additional population benefiting from improved transport services in ƒƒ medium-size cities” (World Bank Group 2015a, 71). this dilemma and discusses ways of dealing with it, a sharpening of this guidance is needed. The results framework needs to make a clearer distinction between the ongoing operations and any new program directions that are proposed in response to the analysis carried out in the SCD, which will yield measurable results in the outer years of the program. This has been done, for example, in the Azerbaijan CPF, where the results framework and the text clearly distinguish the ongoing World Bank Group activities—both lending and nonlending—from the new activities and establish their link to the objectives. At the same time, as pointed out above, the World Bank Group often has long-term commitments to build capacity and institutions that are poorly reflected in the results framework. The current formulation of the results framework, for example, does not reflect the World Bank’s continuing support to the National Statistical Bureau in Uganda or for Extractive Industries Transparency Initiative in Chad, which, to be effective, needs to be sustained over a number of CAS/CPF periods. These support programs need to be separately delineated with the identification of markers of progress that allow for monitoring to determine if these programs are on track. As discussed in the context of selectivity, the articulation of the links between the World Bank Group program and the objectives indicators remains relatively weak. It would be useful in the results frameworks to designate the core activities (both lending and nonlending) that will drive the achievement of the indicators and for the text on the results chain to deal with these links more explicitly. Listing the plethora of other activities that contribute contextual support to the objective over time adds confusion rather than providing clarity to the results frameworks. The World Bank Group’s nonlending activities—both World Bank Advisory Services and Analytics and IFC Advisory Services—are often poorly integrated into the results frameworks. Given the importance Independent Evaluation Group | World Bank Group 27 of these activities in the World Bank Group’s program, this is an issue in need of stronger guidance. The emphasis on quantification works against the integration of these activities, as does the failure to incorporate longer-term engagement of the World Bank Group in supporting institutional development and capacity building. The contribution to outcomes of World Bank Advisory Services and Analytics and Advisory Services was sometimes not shown at all in the results framework (Indonesia). Overall, the contribution of analysis and advice to the World Bank Group program was handled unevenly in the CPFs under review. One-third of the CPFs were found deficient in this regard. By contrast, the Bangladesh and Panama CPFs were viewed as good practice. One remaining weakness of some CPFs in this area is the failure to discuss the capacity of the country to monitor the elements of the results framework. This is not the same as the issue of overall data development and monitoring capacity, since the data in the results framework is often generated at the project level and may be area-specific. A particular issue is the weakness of the baseline data, as evidenced by the large number of indicators that have zero as their baseline, indicating that only the change can be monitored, but without an indication of how the World Bank Group will contribute to the achievement of the twin goals. The inadequate quality and frequency of household surveys make it difficult in many countries to assess the impact of the program, and the World Bank Group is rightly making progress in this area a priority. One World Bank Group The integration of IFC and MIGA into the CPF process has been significantly improved under the new country engagement approach. All CPFs reviewed discussed the strategic direction and content of IFC (and in some cases MIGA) operations and their contributions to the overall World Bank Group program. However, in many cases this discussion was only weakly reflected in the CPF results framework. While ongoing IFC and MIGA operations are generally listed, there is usually very little on their planned programs. IFC and MIGA ascribe this to their respective business models, which depend heavily on private sector demand, making it difficult to identify future operations. In Uganda, for example, agreement was reached during the CPF preparation that IFC would attempt to identify new operations in the agriculture sector, with potential support for value chains and agricultural finance, including leasing, which would complement a number of World Bank–supported activities. The CPF discusses this, but the results framework, while specifying the future World Bank program, has only a general reference to possible IFC engagement in the sector. IFC could go further in identifying the kinds of operations it might support, based on its regional, country, or industry strategies and the role of its advisory services in promoting operations outcomes in these areas, while recognizing its dependence on private sector demand. IEG’s review of the World Bank Group’s diagnostics of issues for PSD in the SCDs (see chapter 2) also looked at how these were reflected in the CPF results frameworks. The review found that program priorities identified in the CPF were heavily focused on providing credit lines to SMEs and skills training for job creation and growth, perhaps because of the policy focus of the SCD diagnostics. This 28 World Bank Group Country Engagement | Chapter 3 could raise the question of whether IFC team leaders had sufficient time and/or resources (staff and dedicated budgets) to contribute fully to the SCD/CPF processes. The guidance asks that the CPF identify programs that will be undertaken jointly by the World Bank Group entities. However, a number of the IFC participants in the CPF process interviewed by IEG noted that the difference between the World Bank and IFC operating models makes it difficult to develop joint programs. One exception seems to be the use of public-private partnerships in the energy sector, which characterizes a number of programs and where Bank activities to develop the policy, institutional, and regulatory frameworks as well as Bank lending have meshed with IFC support for investments and MIGA guarantees. In most cases, however, Bank, IFC, and MIGA involvement proceeds in parallel rather than through direct cofinancing by the institutions. Ten CPFs envisioned joint projects or programs, which is quite an ambitious number compared with past CASs. Generally, these were listed as “potential.” A few CPF results matrixes included ongoing projects/programs cofinanced by at least two of the World Bank Group institutions. But the realization of such cofinanced projects depends ultimately on public and private sector client demand. This is a topic that goes beyond the treatment in the CPF and requires some fundamental thinking in the World Bank Group about when and where joint programs should be prioritized. On balance, IEG assessed World Bank Group collaboration in CPF preparation positively. Some good practice examples are cited in box 3.5. Box 3.5 | World Bank Group Collaboration in the CPFs In Mali, the CPF is emphatic that IDA and IFC will work together to unlock the ƒƒ potential for transformation, with planned interventions for both soft and hard investments. Five areas have been listed for special attention. A joint IFC–World Bank business plan will be built around competitive value chain development operations for specific areas. New IDA and IFC knowledge and technical assistance interventions are planned for financial sector and capacity building in the agricultural sector. Mali is also a priority country for MIGA. ƒƒIn Indonesia, there is considerable evidence of significant roles for the IBRD, IFC, and MIGA for the implementation of the CPF. Particularly for large and complex infrastructure investments such as energy, maritime, and housing, the government has requested that the World Bank Group bring the balance sheets of all three institutions to bear. Although the brunt of World Bank Group support remains with the IBRD, multiple IFC operations are already ongoing and new ones are planned under the CPF. MIGA has an ongoing project, but envisions new operations in the future, depending on demand. (Box continues on the following page.) Independent Evaluation Group | World Bank Group 29 Box 3.5 | World Bank Group Collaboration in the CPFs (continued) In the Egypt CPF, collaboration and appropriate division of labor among the World ƒƒ Bank, IFC, and MIGA is both meaningful and effective. The World Bank and IFC have identified and worked together on three key areas of collaboration: energy, youth skills development, and investment climate/competitiveness. Both the World Bank and IFC acknowledge that collaboration and coordination make business sense, given the large size of the Egypt portfolio. Since the 2011 revolution, MIGA has provided political risk insurance coverage to several foreign direct investors, complementing Bank and IFC support. Bosnia and Herzegovina and Bangladesh were also cases of a well-coordinated ƒƒ operational approach. The Côte d’Ivoire CPF gave special prominence to MIGA’s contribution to the program. In the Egypt CPF, collaboration and appropriate division of labor among the World Bank, IFC, and MIGA is both meaningful and effective. The World Bank and IFC have identified and worked together on three key areas of collaboration: energy, youth skills development, and investment climate/competitiveness. Both the World Bank and IFC acknowledge that collaboration and coordination make business sense, given the large size of the Egypt portfolio. Since the 2011 revolution, MIGA has provided political risk insurance coverage to several foreign direct investors, complementing Bank and IFC support. Risk Analysis All CPFs used the SORT, which was introduced in 2014 to provide a more structured method of rating risks that would be broadly comparable across countries. It identifies nine risk categories: political and governance, macroeconomic, sector strategies and policies, technical design of project and program, institutional capacity for implementation and sustainability, fiduciary, environment and social, stakeholders, and other. Each risk category is rated as high, substantial, moderate, or low. The evidence suggests that for the CPFs, the country teams have taken considerable care in deciding the ratings and have, if anything, been conservative (on the high side) in their risk assessment. The descriptions of the risks are often very generic; however, possibly in some cases out of concern that governments would react adversely to a more precise formulation. In addition, risk mitigation is not well handled in the text, possibly because of the broad way in which the risks themselves have been framed. There are many risks for which mitigation is not possible, at least in the short term. In these cases, it is a matter of how the World Bank Group should adapt to these risks. 30 World Bank Group Country Engagement | Chapter 3 A key part of the discussion of adaptation should be the question of the size of the World Bank’s portfolio in relation to the various risks. Very few CPFs discussed this, even when the CPF identifies important questions of the country’s absorptive capacity (El Salvador, Myanmar). The presentation and discussion of risks also needs to draw a clearer distinction between World Bank and IFC programs, since these have very different risk profiles. It is also possible that MIGA’s long experience in country risk assessments could usefully be drawn on for the CPF risk analyses. It is evident that some elements of the SORT were designed for use in projects, and have little relevance for the country-level risk analysis. While stakeholder risk, for example, has a very clear meaning at the project level, it is difficult to see what it means when aggregated to the country level, since different parts of the program are likely to have very different levels of stakeholder risk. Similarly, the category of sector strategies and policies was intended to deal with the tension between, say, an agricultural project and the policies and institutions in the agriculture sector. It is not clear how such specific sector risks would be aggregated to the country level. Technical design is another category that seems inappropriate for a country-wide assessment—at the country level, this would be picked up by the rating for institutional capacity. It is evident from table 3.1 that these three risk categories were consistently rated as less risky than others for most countries, with very little discussion in the text. It is probable that they were simply not seen as relevant to the country-wide risk assessment. The SORT combines higher-order political, social, and environmental risks with more specific operational risks. It needs to be rethought in the context of the CPF, with an increased focus on how the World Bank is addressing systemic operational risks. In Uganda, for example, the delay of the legislature in approving IDA credits has emerged as a serious constraint to achieving the results indicators. The World Bank has decided, therefore, to carry out briefings and consultations with TABLE 3.1 | Comparison of Risk Ratings by Category for 22 Countries Rating Low/Moderate Substantial/High Political and governance 4 18 Institutional capacity 2 20 Environmental and social 10 12 Fiduciary 8 14 Macroeconomic 9 13 Sector strategies and policies 13 9 Stakeholder 19 3 Technical design 19 2* Note: *In one country, technical design was not rated. Independent Evaluation Group | World Bank Group 31 parliamentarians at a much earlier stage of project preparation and to continue this through the preparatory process to mitigate this risk. Similarly, systemic issues of procurement and safeguards, and how the World Bank is addressing them, could be taken up in this section, also including the interplay between program design and operational risk that should be more extensively discussed in the CPFs. The Use of Country Engagement Notes World Bank Group guidance advises teams to use a Country Engagement Note (CEN) only when country conditions are in crisis brought about by conflict or political transition, or when the World Bank Group lacks sufficient knowledge when reengaging in a country after a prolonged hiatus, and is unable to define detailed objectives or develop a program for the medium-term (World Bank Group 2014). Fragility or predictable instability is not in itself an argument for a CEN, and a number of SCD/CPFs were prepared for fragile states. Because CENs are often prepared in situations where the World Bank Group has knowledge gaps, it is not required that a CEN be preceded by an SCD, and it is not expected to contain as much documentation and analysis as a CPF. However, the 2014 guidance note states that “it must contain sufficient analysis to support the preparation and monitoring of the proposed engagement . . . [and] when possible, it is expected to have solid underpinnings, including assessments on poverty, gender, fragility, and other important aspects of development” (World Bank Group 2014, 17). This evaluation reviewed the six CENs prepared from January 2015 to April 2016, two of which were done for countries classified as fragile and conflict-affected situations—the Central African Republic and Guinea-Bissau. The others were for Ecuador, Fiji, Guyana, and Turkmenistan (see appendix F). At the same time, some fragile and conflict-affected situation countries, such as Afghanistan and Myanmar, which under the previous system may have prepared Interim Strategy Notes, adopted the SCD/CPF process. Lack of confidence about country ownership for a medium- term program seems to have been the primary motivation in Ecuador, Guyana, and Turkmenistan. For some countries, the judgment was that the World Bank Group lacked the knowledge base to develop a full SCD. Most of the CENs were prepared for relatively small countries where the instrument provides the possibility of delivering the strategy product at lower cost. However, in IEG’s view, this is better handled by managing the costs of the SCD/CPF directly—for example, by producing shorter documents with a small team, and by allowing for an update, rather than a full SCD, in the next round. IEG assessed adherence of the CENs to the guidance along multiple dimensions: timing, content, evaluability, and preparations for subsequent regular country programming documents. This is presented in appendix F. In summary, the review found a number of apparent deviations from the expectations in the guidance, leading to the conclusion that either the guidance or the current practice needs to be modified: 32 World Bank Group Country Engagement | Chapter 3 „„Justification: Most country teams justify CENs on the basis of a political crisis. However, this justification seems weak in Ecuador, where elections are not scheduled until 2017 and there is no immediate crisis. „„ frame: CENs are expected to have a 12- to 24-month cycle, with a maximum limit of 2 years. Time However, Fiji exceeds this limit in its delivery schedule and its ability to complete intended activities. „„Dataavailability: Ecuador and Guyana had data that, if country ownership were not an issue, could have permitted the preparation of an SCD. „„Trackingperformance: A CEN should describe benchmarks and indicators and analyze performance and lessons at the end of the CEN period. However, the absence of even a rudimentary results framework in three of the six CENs will make it difficult to accomplish this at the end of the CEN period. „„Risk analysis: All the CENs use SORT to assess risks, but only half of them acknowledge high risks, underestimating institutional and governance risks. „„Document length: All 6 CENs exceed the prescribed 15-page limit. Emerging Messages There is evidence of a modest improvement in the quality and focus of the CPFs in comparison with the previous CAS/CPSs. The focus on the achievement of the twin goals in the SCDs has caused the CPFs to focus much more explicitly on what programs are required to support this, and there are clear indications of stronger selectivity. Most CPFs now also include discussions of the strategic direction and content of the IFC/MIGA programs and how these contribute to the World Bank Group program. Most CPFs have clearly benefited from having the SCDs and from the widespread consultations on that document. However, the CPFs do not yet bring forward systematically and completely the priorities identified in the SCD—priority constraints, identified knowledge gaps, or identified data weaknesses and gaps. Also, the results frameworks would have benefited if more clearly set in the context of longer-term expectations for the World Bank Group’s work in the country, building on the SCD analysis. The CPF results frameworks show improvements but are still a work in progress. The requirement in the guidance note (World Bank Group 2014) for the intervention logic to be specified has helped to direct more attention to the links between objectives and their indicators, and the definition of monitorable and measurable results for the indicators has been a strong point of many of the CPFs reviewed. The intervention logic is often explained well, and most indicators are now measurable and monitorable. But the broader results chains are truncated, many or most indicators may be from ongoing rather than new activities, and there is too little effort to link nonlending activities to results. There is more systematic attention to risks, but the risk frameworks may need rethinking. The CPFs now apply—with significant care—the same analytical tool (SORT) as lending operations, but some Independent Evaluation Group | World Bank Group 33 elements of this tool have little relevance to country-level analysis. As a result, current discussions are quite generic, and with no real separation between country risks that the World Bank cannot influence and more operational risks, or between risks for the World Bank versus IFC/MIGA. The World Bank Group needs to clarify the justification for preparing CENs and their role in preparing the ground for a subsequent SCD/CPF cycle. An economic crisis or a change of government is not sufficient justification to use CENs. All CENs have proposed to fill knowledge gaps, but only Guinea- Bissau used the CEN period effectively to prepare an SCD. It is not clear whether the remaining countries will be able to follow suit or will propose another CEN for the next cycle. 1 T  he SORT has broader application than the CPF. It applies both to World Bank–supported projects and country programs. 2  his may be a case where the World Bank felt that it did not have an absolute advantage in supporting the sector, T given the need to engage with a large number of subnational units. The government, however, felt that the World Bank had a comparative advantage in supporting this area, relative to other potential development partners. R eferences World Bank Group. 2014. World Bank Group Guidance for Country Partnership Framework Products. Washing- ton, DC: World Bank Group. ———. 2015a. Country Partnership Framework for the Republic of Colombia for the Period FY2016–21. Washing- ton, DC: World Bank Group. ———. 2015b. Country Partnership Framework for the Republic of Indonesia for the Period FY2016–20. Washing- ton, DC: World Bank Group. ———. 2015c. Country Partnership Framework for the Republic of Maldives for the Period FY2016–19. Washing- ton, DC: World Bank Group. 34 World Bank Group Country Engagement | Chapter 3 4 Background T H I S C H A P T E R examines various aspects of the fourth evaluation question: Did the SCD and CPF teams have the necessary and appropriate products, systems, and resources, Institutional and and was this new model helpful in integrating the work of Cross-Cutting the three institutions, in coordinating with global practices Issues in the and cross-cutting solutions areas, and with interactions with New Country government authorities and other key stakeholders? Engagement Model Internal Processes The key to the new country engagement model is its two-step approach. It consists of two distinct documents with different World Bank Group task managers: an analytical document (SCD) setting out the binding constraints on the country’s development and the priorities as seen by the World Bank Group, followed by a shorter programming document (CPF) that describes the World Bank Group’s assistance strategy derived from the intersections of the identified priorities, the government priorities, and the comparative advantages of the World Bank Group. In general, the timing and processing of SCDs and CPFs have been robust and helpful for most of the 22 SCDs and CPFs examined. Internal SCD processes typically have started with country team brainstorming meetings, and continued through the concept note to the ROC decision review meeting, which usually gave rise to constructive comments from various parts of the World Bank Group. Processing problems generally arose from country and timing issues. In some cases, the documents were prepared while a government was still working on its own development plans. In Bolivia, both the SCD and the CPF were completed before the country published its Economic and Social Development Plan, which led to restrictions on the publication and dissemination of the SCD. It may have been preferable in this case to delay the CPF to provide the opportunity to incorporate the 35 government’s completed strategy. In Chad, the government held up publication of the SCD and the processing of the CPF because of concerns about apparent inconsistencies between the SCD and the recent Heavily Indebted Poor Countries Completion Point Paper prepared by the World Bank and the International Monetary Fund. The CPF consultation process for Egypt was much more extensive and inclusive than that for the SCD, which took place only at the concept paper stage. IEG found the corporate review process for the SCD and CPF comprehensive and satisfactory. In many cases the reviewers noted considerable scrutiny or support from Regional management, and (except for some of the very early SCDs) useful support from central units. The formal review processes brought together different parts of the World Bank Group and helped improve the quality of the reports. The review meetings were co-chaired by the World Bank and IFC, which can involve scheduling difficulties. It is not clear, however, that a ROC is really needed for an SCD at the concept review stage, or whether review can be delegated to the chief economist for the Region involved. (IEG found that all chief economists played active and constructive roles in support of the SCD preparations, but with different approaches based on regional differences and personal preferences.) The constructive role of peer reviewers has also been noted (box 4.1). SCD teams can be very large. For example, the Albania SCD had as many as 80 staff listed as team members, many from the global practices, making it difficult to differentiate those more substantively involved from those who simply commented on drafts of the document. On the whole, the relations between the CMU and the global practices seemed to work relatively well. In a few cases, there were some bumps in the road, such as contradictory advice from managers and program leaders, or Box 4.1 | The Corporate Review Process for the Serbia SCD ƒƒThe SCD concept note review meeting was co-chaired by the Europe and Central Asia Region and IFC vice presidents and received several sets of comments, including from three peer reviewers and the International Monetary Fund mission chief. Main issues discussed were the storyline on growth and shared prosperity and the ƒƒ analytical framework. The team was advised to be selective and to narrow down the areas to be ƒƒ analyzed. E xtensive and high-quality comments were shared for the ROC meeting from three ƒƒ peer reviewers, several global practices, the Europe and Central Asia Development Effectiveness Unit, IFC, International Monetary Fund staff, and others. ƒƒMajor issues discussed were the strengthening of interlinkages, the analysis of risks to sustainability, prioritization, and sequencing. 36 World Bank Group Country Engagement | Chapter 4 global practices pushing excessively for “their” issues or projects. (The engagement in the SCD and CPF processes of IFC and MIGA is discussed separately in this chapter.) The Internal Audit Department has assisted IEG with data on processing times and costs discussed in the following paragraphs, which compare data for the SCD and CPF with data from the previous CASs in the same 22 countries. The new processes take longer. The evaluation compared the time elapsed from the concept review meeting to the Board delivery date for all three documents. The average time for the CPF alone went down to 140 days, but the average for SCD delivery alone (from concept review to Board) was 285 days, and in practice, there has been some overlap between the SCD and CPF processes in most cases. Thus the average elapsed time has increased from 231 days for the last CAS to 388 days for the new country engagement model as a whole (figure 4.1 and table 4.1). However, the average numbers mask significant differences, and the CAS numbers include some outliers (Botswana and Maldives), where preparation of the previous CAS took more than two years. In the future, processing times for the SCD/CPF may be reduced somewhat as the World Bank Group moves up the learning curve for the new documents. FIGURE 4.1 | Elapsed Time for SCD-CPF Process 1,000 900 800 700 600 Days 500 400 300 200 100 0 er ia ng ijan sh d ia ts z. a o d ia yp d ca ab ire lv . or on aiti do s M esia s ya li na r Se a U ia U da ay Sa ep Pa ma M Ma In ura ve an m C ha Az an an liv Bo Her C mb rb ad gu de i Ar o n Eg ôt ta R H R Ba ba di w t, ’Iv C ga o n n b d ru la B al Al ol os e H El ia C sn Bo CAS timing SCD concept note to CPF board discussion timing TABLE 4.1 | Comparison of Elapsed Time for the CAS, SCD, and CPF CAS Elapsed Time SCD Elapsed Time CPF Elapsed Time Time (no. days) (no. days) (no. days) Average for 22 countries 231 285 140 Minimum 36 97 60 Maximum 877 520 336 Note: Elapsed time refers to the interval between concept review meeting and the Board of Executive Directors discussion date in the same set of 22 countries. CAS = country assistance strategy; CPF = Country Partnership Framework; SCD = Systematic Country Diagnostic. Independent Evaluation Group | World Bank Group 37 The new process clearly requires additional resources since the SCD is a new product. The evaluation compared budgets (World Bank only) charged to each SCD and CPF with the expenses on the last CAS in the same set of 22 countries. Despite the shorter time spent on the CPF compared with the CAS, actual expenditures charged to the CPF code were, on average, marginally higher than those spent on the previous CAS. The $261,000 average expense on the SCD was additional (table 4.2). Budget records indicate several anomalies in charges posted to SCD and CPF budgets. The budget recording system underestimates the real cost of SCDs and CPFs. During the country missions, sector staff indicated that they did not charge all the time spent they spent on the SCD or CPF to the respective charge codes, while IFC staff were mostly financed by IFC’s own budget, which is not reflected in the costs. Preparation of the SCDs has substantial budget implications. Four of the 22 SCDs cost more than $450,000, while the average cost was $261,000. No discernable pattern was detected by Region. In smaller countries, the cost of SCDs was just as high as in larger countries, but CPF costs were slightly lower. Over time this additional cost of the SCD might be offset by the longer life cycle of CPFs compared with the previous CAS. It is also possible that in the next round more SCDs could be prepared as updates to lower costs. One World Bank Group—The Involvement of IFC and MIGA The new country engagement approach places strong emphasis on integration of the World Bank, IFC, and MIGA as One World Bank Group. In the past, IFC and occasionally MIGA increasingly participated in the CAS in countries where they had a substantial presence, but their role in the CAS process remained weak because of the demand-driven nature of their portfolio. Even in joint CASs, most country strategies lacked specific proposals on collaborative activities, and joint programs were often not reflected in the results matrixes. IFC’s country engagement consisted of listing its ongoing investment and advisory services projects in the CAS. Monitoring and evaluation of IFC activities was also difficult, since their expected outcomes or indicators were rarely included in CAS results frameworks. The guidance notes make clear that under the new approach “All CPF components and the CEN are joint documents approved by the World Bank, IFC, and MIGA” (World Bank Group 2014a, 5). Sign-off TABLE 4.2 | Comparison of Budget Used by the CAS, SCD, and CPF CAS Cost SCD Cost CPF Cost Cost (US$, thousands) (US$, thousands) (US$, thousands) Average for 22 countries 172 261 189 Minimum 10 0 73 Maximum 420 469 593 Note: CAS = country assistance strategy; CPF = Country Partnership Framework; SCD = Systematic Country Diagnostic. 38 World Bank Group Country Engagement | Chapter 4 by World Bank, IFC, and MIGA team leaders and managers is now required prior to formal reviews, and their senior managements are expected to be involved in the SCD and CPF processes. These arrangements are reinforced by the formalization of a Regional Coordination Mechanism, consisting of senior regional management teams of the World Bank, IFC, and MIGA in each Region. The Regional Coordination Mechanism is reported to have played less of a role in SCD/CPF preparation than in monitoring implementation. IFC’s participation in the SCDs and CPFs is now stronger and more evident than under the previous CAS approach. IFC was an integral part of the team in all 22 SCDs and CPFs reviewed by IEG, even in countries where it had limited operations. There is now an IFC structure and established work flow for the new approach, with internal ownership resting with the IFC Regional directors and country managers. Normally, both the IFC team leader and strategy officers participate in internal deliberations, including in-country consultations, and lead private sector engagements for SCDs and CPFs. As a result, the integration of the IFC program in the SCD/CPF process is much more substantive than in the previous CAS process. The new approach has brought positive changes, including improved coordination and collaboration between World Bank Group entities and, for IFC, greater involvement in shaping the overall country strategy and more knowledge and learning gained from the stakeholder consultations. The new approach has also provided MIGA with better engagement opportunities within the World Bank Group and, in the countries where they have been strategically engaged, coordination is evident. However, the expectations of an integrated, One World Bank Group approach have not been supported by dedicated resources for IFC and MIGA staff. While World Bank staff have specified budget codes for SCD and CPF, IFC and MIGA staff do not have a separate budget for these activities at the country level. IFC does not yet have a mechanism for tracking the cost of its SCD and CPF engagement in individual countries,1 and there is no allocated budget at the country level for SCD and CPF preparation. All these issues, plus MIGA’s limited staff, have affected the extent of engagement by several country units (Côte d’Ivoire, Costa Rica, and Uganda are examples). MIGA uses a time recording system code to track staff time and cost of its new country engagement, which is also funded through its operating budget. Hence, the level and depth of IFC participation in an SCD or CPF process depends on the resources (budget and staff) of Regional and country teams. Like IFC, MIGA does not have a country-specific budget allocation for its SCD and CPF engagement. As discussed in chapters 2 and 3, the participation of IFC (and MIGA) was more limited in the SCDs than in the CPFs. The SCDs still generally reflect a standardized approach to private sector issues. With a few exceptions, private sector interventions are viewed primarily as a response to a government policy of improving the business climate, with evidence largely from the Doing Business reports and Enterprise Surveys on the binding constraints for the private sector. There is relatively little discussion of private sector strategic priorities to address the twin goals. IFC’s ongoing and new engagements were integrated within the 22 CPFs reviewed. IFC programs mostly clustered around inclusive growth objectives, often with IFC supporting two or more objectives, and with the level of its involvement reflecting the relative scale of its operations. However, IFC programs were not adequately reflected in results frameworks, where concern about potential accountability based on a portfolio dependent on Independent Evaluation Group | World Bank Group 39 private sector demand seems to have outweighed the potential of the results framework in providing strategic focus to the programs for support through IFC investments and advisory services. The volume of MIGA’s existing exposure and business prospects in a country drive the extent of its participation in the SCD and CPF processes. MIGA’s engagements are limited by staffing constraints. MIGA could benefit from proactive and inclusive World Bank team leadership and simple steps to bring MIGA into the teams early and include them throughout the country engagement process. For IFC and MIGA, the CPFs reflect mostly ongoing programs, highlighting the difficulty of planning ahead under their business models. Sixty-three percent of IFC projects listed in the 22 CPF results matrixes reviewed are ongoing projects (including commitments dating back to fiscal 2007). Nearly 40 percent of IFC’s ongoing and new projects support access to finance and financial inclusion objectives in the CPF, and 25 percent support infrastructure development objectives. In the future, it would be useful for IFC and MIGA to identify areas of concentration and lines of promotional activities in the CPF results matrixes with process and intermediate indicators that could be updated and replaced in the PLR. Most CPFs discussed collaborative initiatives undertaken by the World Bank Group. Ten CPFs envisioned joint projects or programs that were usually listed as “potential,” while other CPF results matrixes included a few ongoing cofinanced projects by at least two of the World Bank Group institutions. However, expectations of increased World Bank Group cofinanced projects need to be tempered by the reality in the field and lessons from joint projects.2 Converting these collaborative activities into projects that the World Bank Group could cofinance depends ultimately on public and private sector client demand. Joint projects have been useful and highly relevant in high-risk contexts, in supporting pioneering investments and in complex projects. However, joint work also entailed costs and risks. Partnerships Both country instruments are based on collaboration with governments, although in different ways. The SCD reflects the evidence and analysis of the World Bank Group and is prepared in consultation with national authorities, the private sector, and other stakeholders. The CPF is developed in partnership with the government, usually through several ministries or agencies and at various levels, although wider consultations are also expected. This document is expected to take as its starting point the country’s own vision of its development goals and its strategy for achieving them. The guidance states that “It is important to make the case [in the CPF] that there is a clear, country- owned development agenda on which to anchor the CPF program” (World Bank Group 2014a, 8). Stakeholder consultations have been effective in outreach to key stakeholders to obtain inputs and help clarify priorities for the SCDs, and subsequently the CPFs. Consultations were held with governments, development partners, think tanks, civil society organizations, and the private sector. IFC played an important role in outreach to the private sector. In Bangladesh, Bolivia, Egypt, and Myanmar, consultations for the SCD and CPF were held in several cities. In Bolivia, a local donor coordination forum sustained meaningful coordination with development partners and facilitated the development of a framework document with the government, which outlined potential common elements for all 40 World Bank Group Country Engagement | Chapter 4 development partners in Bolivia, ensuring that their strategies reflect the engagement of other partners. A number of stakeholders also expressed appreciation for the participatory processes, but other nongovernmental entities would have liked to see greater interaction and feedback on their comments. However, to many of the stakeholders, the difference between the SCD and CPF consultation processes was somewhat blurred, and some participants found them lengthy and duplicative. In many cases, there were several rounds of consultations. The first was typically on a summary SCD concept note to get inputs into the analysis and focus of the SCD; the second, often when the SCD was complete, aimed to disseminate the SCD and lead into the CPF discussions. This also provided an opportunity for the government and civil society to influence the CPF program. Following such processes, IEG found good ownership of the CPF priorities, but with limitations (box 4.2). However, as mentioned in chapter 2, the SCD was not always shared with stakeholders, which prevented others from giving feedback on it for the subsequent CPF or from being able to use it elsewhere, weakening its value as a public good. Box 4.2 | Country Partnership of Proposed CPF Priorities Has Been Good but Is Not Assured In Albania, the World Bank sees a window of opportunity to push forward needed ƒƒ changes and structural reforms in key areas (fiscal, financial sector, energy, transport, agriculture, and social protection) to make fast progress toward the twin goals. In Bangladesh, the strategic approach of the government is similar to the World Bank’s ƒƒ approach and differs only in the pace of the reforms being undertaken. In Bolivia, the CPF is aligned with the government’s intent to sustain poverty reduction ƒƒ and continue growth. While poverty eradication goals are aligned, the government differs with the World Bank on how to achieve that goal, particularly on the role of the public and private sectors. Differences in government and World Bank priorities are limiting the areas of World Bank engagement, such as support for human development. In Egypt, the CPF is broadly consistent with the government’s program, but with some ƒƒ important differences between government and CPF strategies regarding a number of mega infrastructure projects proposed in the government’s development plan over the medium term. In Azerbaijan, a 5 percent increase in the lending envelope is designed to encourage ƒƒ much-needed structural reforms for PSD; effective public sector, human capital, and skills; infrastructure connectivity; and social and environmental sustainability. But the government did not want the World Bank involved in the social sectors, although these were identified as priorities in the SCD. Independent Evaluation Group | World Bank Group 41 Cross-Cutting Themes and Issues in SCDs and CPFs The guidance on SCDs and CPFs recommends attention to four cross-cutting themes: gender, climate change, governance, and fragility. With one exception, the review found that the 22 SCDs reviewed did a reasonable job (from a processing point of view) of addressing these cross-cutting themes. There were, however, some variations between SCDs in how well the various themes were addressed. In addition to these themes, this section briefly discusses treatment of knowledge and governance issues. Gender The depth and quality of gender analysis in SCDs varies. (See appendix H.) In all, 19 of the 22 SCDs reviewed discussed gender issues (16 on labor and 8 on education and health), while 8 SCDs highlighted gender issues among the main challenges. SCDs referred to gender assessments in the 6 countries where they were current and available during preparation. The review found several good examples of gender integration, such as Albania and Chad. The Albania SCD discussed gender constraints on labor and access to financial markets. The Chad SCD discussed constraints to women’s agricultural productivity, as well as the impacts of high fertility and early childbearing on women’s economic activity and health. However, when priorities were identified in the SCDs, their gender linkages were often not clearly spelled out. Gender inequality may not be recognized among the most binding constraints to the twin goals in many countries, even if education, skills, and access to jobs might be important dimensions of the priorities identified by SCDs. Gender issues highlighted in SCDs tend to be echoed in CPFs. In SCDs, gender gaps identified related to labor markets, followed by skills and school-to-work transitions. These issues were often also picked up in the CPFs, where there seems to be an evolution beyond human development in the previous generation of CASs toward economic empowerment, especially related to labor issues, skills development, and jobs. Poverty among female-headed households is also discussed. “More and better jobs” is the centerpiece of the Bangladesh SCD, which also provides a very thorough analysis of female employment and gender equality. While gender issues may not be limited to women’s issues, the discussions on gender in SCDs and CPFs are associated only with women’s issues. Risky behavior among men and boys (crime, drugs) in Central American countries is not addressed. Compared with previous CASs, CPFs are better at articulating World Bank Group actions to address specific gender inequalities, and they include many more sex-disaggregated indicators. But the alignment between the actions proposed and indicators in the results frameworks is still relatively weak. The Costa Rica CPF, for instance, refers to gender disparities and stereotypes in education and labor markets, as well as IFC work on access to finance for women, but the results framework only included indicators on female beneficiaries in two health and education projects. CPFs often include indicators on the percentage of female beneficiaries in existing projects but provide little new information beyond what is already being monitored by individual projects. Treatment of gender under the country engagement model has not been fully consistent with the World Bank’s commitments on gender. The SCD guidance note (World Bank Group 2014b) mentions gender in passing, while the 2014 CPF guidance note (World Bank Group 2014a) did not 42 World Bank Group Country Engagement | Chapter 4 fully adopt the gender requirements under Operations Policy 4.20, the IDA replenishments, or in the corporate scorecard. The World Bank Group Gender Strategy (World Bank Group 2015), finalized after the guidelines were completed, provides more specific directions on how gender dimensions can be incorporated in the SCDs and CPFs, noting that SCDs should highlight how closing the key gender gaps in endowments, economic opportunities, and voice and agency would boost the attainment of the twin goals. The gender strategy recommends broad consultations on gender with the government and civil society as part of the CPF process. The CPF guidance needs updating to incorporate these expectations. Climate Change and Natural Resource Management Climate change is an emerging agenda that needs analysis in the SCDs to position the World Bank Group to engage more meaningfully in the future. In some countries, such as Bangladesh, it received a great deal of attention in the SCD and was elevated to one of the three focus areas in the CPF (see box 4.3). Similarly, in Egypt, climate change and the environment is one of the main cross-cutting themes discussed throughout the SCD and the CPF. Overall, 18 of the SCDs contained a substantial discussion of climate change or environmental sustainability, and included them directly or indirectly among SCD priorities. However, there was limited attention to climate change in the SCDs in some vulnerable countries (such as Bolivia—although it flagged climate change and natural disasters among knowledge gaps—and Mali). Climate change and environmental sustainability received attention in most of the CPFs, including in countries where there had been little discussion in the SCDs. Among the 22 CPFs reviewed, 18 specified objectives or subobjectives addressing climate change or environmental sustainability. The Uganda SCD is silent on climate change, while the CPF does discuss some of the links between population density and environmental sustainability, without operational implications. In Bolivia, climate change is not a priority in the SCD, but the CPF emphasizes the need to strengthen capacity Box 4.3 | Responding to Government Priorities on Climate Change in Bangladesh In Bangladesh, building on the government’s priority to adaptive delta management,  the SCD has integrated the discussion of climate change with sustainable resource management, especially of land, water, and aquatic resources.  Climate and environmental management is one of the three CPF focus areas, with clear links to the other focus areas—growth and competitiveness and social inclusion—and to the twin goals reflected in the results framework. World Bank Group activities will focus on boosting Bangladesh’s resilience to climate  change and natural disasters, improving the management of water infrastructure, and promoting agricultural productivity with climate-smart farm practices. Independent Evaluation Group | World Bank Group 43 to manage climate risks, reduce vulnerability to natural disasters, and deal with the environmental impacts of agro-industries. Overall, in several such cases there was often a discussion in the text, but this was not reflected in the results framework. Possible reasons include the timing mismatch— climate change is a long-term proposition, and actions taken now are unlikely to have outcomes in the time frame of a CPF—and the fact that much of the World Bank Group’s work on climate change is funded by trust funds. In other countries, environmental sustainability is reflected in the CPF in relation to specific sector operations, such as energy or land management. Fragility The evaluation reviewed the contribution of fragility assessments to SCD/CPFs and CENs in the six countries where they were available. Fragility assessments were undertaken recently in the 4 fragile countries (Chad, Colombia, Côte d’Ivoire, and Mali) of the 22 that prepared SCDs and CPFs during the evaluation period, and 2 countries with CENs (the Central African Republic and Guinea-Bissau). Uganda, though not classified as fragile, had a risk and resilience assessment prepared in June 2016 (appendix J). Bosnia and Herzegovina and Haiti did not have recent fragility assessments to inform the process, and this may have impeded the quality of the diagnostics, and hence the subsequent strategy. In its recent evaluation of fragile situations (World Bank 2016), IEG recommended that fragility assessments be applied systematically as part of the SCD/CPF process in countries where fragility, violence, and conflict are a significant impediment to attaining the strategic objectives of the World Bank Group. The Chad, Côte d’Ivoire, and Mali SCDs explicitly referred to their respective fragility assessments and discussed the historical nature of fragility in the country, as well as current drivers of fragility. Fragility issues were also discussed in their CPFs, though not in as much detail as in the SCDs. The fragility assessment is featured quite prominently in the Mali CPF. Overall, the SCDs and CPFs in this sample made good use of available fragility assessments. Their availability in countries with fragile situations may have made it easier for SCDs, CPFs, or CENs to integrate fragility within the World Bank Group strategy for those countries. Knowledge As discussed in chapters 2 and 3, knowledge gaps are not always identified, and even when they are, the analytic work proposed is not always consistent with SCD findings and CPF priorities. On the whole, the 22 SCDs did a better job of identifying knowledge gaps (most often regarding labor markets, growth and productivity, and access and quality of basic services) and areas where additional work is required than the CPFs did in addressing these gaps. The analytical work proposed in the CPFs was not always relevant and consistent with SCD findings and recommendations. While only one SCD (Indonesia) was found to be deficient and one moderately unsatisfactory (Egypt) in identifying some of the most important knowledge gaps (such as macroeconomic and fiscal assessments), 7 of the 22 CPFs reviewed were less than satisfactory in their analytical response. Deeper analysis of the 10 countries visited by the evaluation team reinforced the finding of a mismatch between knowledge gaps identified by SCDs and the analytical response in the CPFs. 44 World Bank Group Country Engagement | Chapter 4 While most of the SCDs identified data gaps (most often regarding household surveys, firm-level surveys, and labor force data), especially on poverty, some were less specific in the areas where focused attention was needed. CPFs often went beyond the recommendations of the SCD, and in some instances, such as Azerbaijan and Côte d’Ivoire, the CPF proposed analytical work in areas other than those recommended by the SCD, while in Bolivia the CPF was more selective. In Indonesia and Uganda, where a large body of analytical work already exists, the value of this exercise was not evident, at least as far as identifying knowledge gaps was concerned. Governance The evaluation reviewed the governance and political economy aspects of SCDs and CPFs in a purposive sample of seven countries: Azerbaijan, Bangladesh, Chad, Colombia, Egypt, Indonesia, and Uganda. These countries were selected for Regional spread (two countries for Africa and one country each in the other Regions) and because they are known to have significant governance challenges. One factor heightening these challenges was fragility. Guidance for SCDs points out that political economy factors are a crucial part of country context, to be used in identifying constraints, solutions, and opportunities. Guidance for CPFs suggests three key aspects of governance that may be considered: the credibility and legitimacy of the government, incentives for pursuing public interest policies, and capacity of the public administration to implement the policies. There is considerable evidence from IEG’s evaluation that World Bank teams are following the guidance, with governance and political issues analyzed and addressed in SCDs and CPFs, but at different levels of intensity (appendix I). However, some countries try to influence the SCDs to exclude issues they consider sensitive and, in some instances, not explicitly addressing key political economy issues makes the SCD report hard to follow. The issue of weak incentives for reform is featured in numerous ways, with proposals for incentivizing professionals to take hard-to-fill positions and to improve results through performance-based management. Weak capacity is also widely discussed, pointing to causes such as limited human capital and weak educational and health systems. However, such analysis misses the point that some governments may take on the appearance of reform, without improving governance outcomes—for example, by supporting new laws, regulations, and computerized systems, but not their functional implementation. This is not reflected in the guidance, and it is not apparent in the SCDs and CPFs reviewed. Governance features prominently as a key constraint in the SCDs for five of the seven countries, as an overarching priority, a main cross-cutting issue, or an intrinsic part of sustainability. Governance is also featured in all seven CPFs, but at different levels of priority. The Egypt CPF, for example, supports a transformative program to renew the social contract to support private sector job creation, social inclusion, and enhanced governance. In contrast, the Bangladesh CPF views governance reform as a long-term agenda rather than an immediate priority, even though governance is identified as a risk factor. Overall, most SCDs and CPFs take governance into account in both analysis and program recommendations, but the measures proposed to address governance and capacity constraints are not always commensurate with the shortcomings identified. Consideration should be given Independent Evaluation Group | World Bank Group 45 to updating the SCD/CPF guidance so that it incorporates some of the latest thinking now being presented in the 2017 World Development Report (World Bank 2017). Emerging Messages On the whole, consultation processes have been effective in reaching out to key stakeholders within the countries to obtain inputs and clarify priorities for the SCDs and the CPFs. Consultations were held with governments, development partners, think tanks, civil society organizations, and the private sector. IFC played an important role in outreach to private sector stakeholders. The new programming approach has been widely supported by both CMUs and global practices, and in general the new approach has been working well and with appropriate internal procedures. The new process requires more time to complete, and requires additional resources. Average processing times may decline somewhat as the World Bank Group moves up the learning curve for the new instruments. Under the new country engagement approach, IFC’s (and to a lesser extent MIGA’s) participation in the SCDs and CPFs is now broader than under the previous CAS approach, and the integration of the IFC program in the SCD and the CPF is substantive compared with the previous CAS approach, especially for the CPF. However, for IFC and MIGA staff, the expectations of an integrated, One World Bank Group approach have not been matched by the available resources. The new analytical approach in the SCDs still reflects mostly a one-size-fits-all treatment of private sector issues, with limited new information coming from the diagnostics of private sector constraints. Private sector analysis in the SCDs needs to go beyond general policy constraints to include a more business-oriented and granular analysis of a country’s private sector. The new approach does a reasonable job of addressing cross-cutting themes, although the treatment of gender has not been fully consistent with World Bank commitments. The programming implications of gender discussed in SCD priorities are only partially addressed in CPFs. CPFs are much more likely to refer to what is being done in the existing portfolio to address gender issues than to propose future actions (except with regard to analytical work). The availability of fragility assessments in countries with fragile situations may have made it easier for SCDs, CPFs, or CENs to integrate fragility within the World Bank Group strategy for those countries, but the use of CENs needs to be minimized, limiting its use only to countries facing conflict or political crises, or where the World Bank Group is reengaging after a prolonged hiatus. Knowledge gaps are not always identified, and even when they are, the analytical work proposed is not always consistent with SCD findings and CPF priorities. On the whole, the SCDs do a better job of identifying gaps in evidence and data where additional work is required than the CPFs do in addressing them. 46 World Bank Group Country Engagement | Chapter 4 Most SCDs and CPFs take governance into account in both analysis and program recommendations. However, consideration should be given to updating the SCD/CPF guidance so that it incorporates some of the latest thinking arising from the 2017 World Development Report. For example, analysis of weak capacity and recommendations to address it could be strengthened with deeper analysis of the bargaining among powerful interests needed for such improvements to succeed. 1  s part of IFC’s organizational alignment (2016), a new IFC vice presidency, Economics and Private Sector A Development, will oversee the Regional strategy teams, which typically lead the SCD and CPF process. Staffing and other matters related to the new country engagement are expected to be examined, including budgeting systems. 2  he October 2013 World Bank Group strategy classifies World Bank Group projects into two types: (a) cofinancing of T a single project using a combination of World Bank, IFC, and/or MIGA products and instruments (Type 1 joint project); and (b) sequenced, separate projects with shared objective or strategy using a range of World Bank Group products/ services (Type 2 joint project). R eferences World Bank. 2016. World Bank Group Engagement in Situations of Fragility, Conflict, and Violence. Washington, DC: World Bank Group. World Bank Group. 2014a. World Bank Group Guidance for Country Partnership Framework Products. Washing- ton, DC: World Bank Group. ———. 2014b. Interim Guidelines for Systematic Country Diagnostics. Washington, DC: World Bank Group. ———. 2015. World Bank Group Gender Strategy (FY16-23): Gender Equality, Poverty Reduction and Inclusive Growth. Washington, DC: World Bank Group. ———. 2017. World Development Report 2017: Governance and the Law. Washington, DC: World Bank. Independent Evaluation Group | World Bank Group 47 5 A New Approach U N D E R I T S N E W S T R AT EGY, the World Bank Group committed to align its activities and resources to the twin goals of poverty reduction and shared prosperity. It did so by leveraging its Findings and strengths as a global institution to support its clients and work as Recommendations One World Bank Group, drawing on all of its constituent agencies. To this end, it replaced the CAS/CPS with a more evidence-based and selective country engagement model, consisting of two instruments: an analytical SCD and a separate CPF. The SCD is prepared upstream of the CPF and uses evidence- based analysis to identify binding constraints and priority areas with the greatest potential to help achieve the twin goals. It draws on consultations with national authorities, the private sector, and other stakeholders. The SCD is the reference point for the World Bank Group’s dialogue with in-country stakeholders and development partners on the country strategy, and it is an input to the broader development dialogue in the country. The CPF maps out the strategy for the World Bank Group’s support to a member country. It starts from the country’s own vision and development plan. The World Bank Group and the country then draw on the SCD and agree on the CPF objectives, which are derived from the country development goals that reflect the World Bank Group’s comparative advantage and the SCD priorities for achieving the twin goals. Under the new country engagement model, the initial SCD and CPF will be followed by PLRs for the CPF program, and at the end of every CPF period there will be a Completion and Learning Review as a self-evaluation instrument, subject to IEG validation.1 Main Findings The SCD has provided a useful synthesis of the country’s main issues and development constraints. The instrument draws on the comparative advantages of the World Bank Group—no other development partner has a comparable program range 48 and depth of analytic work. The SCD has been well received both inside and outside the World Bank Group, including by governments and other donors. With some variations, the SCDs reviewed identified the main constraints and challenges facing the countries and the priorities to achieve the twin goals. A limited number of SCDs had weaknesses in the use of evidence-based analysis to select priorities. SCDs depend heavily on available analytical material and data. Many SCDs identified knowledge gaps to improve the evidence base for future policy making; this was a useful input for the analytical agendas in the CPFs. Data gaps also inevitably meant that some SCDs suffered from weaknesses in their analysis of current circumstances and future needs for achieving the twin goals. It is therefore important that SCDs identify knowledge gaps and data limitations, and that CPFs aim to close gaps and improve data quality. The majority of SCDs analyzed satisfactorily the drivers of economic growth and their impact on poverty and shared prosperity over time, capitalizing on analytic work done at the World Bank and elsewhere. The analysis of poverty and opportunities for economic growth was generally strong, although some reports suffered from outdated data sets. The poverty analysis was frequently accompanied by useful regional and international comparisons. Analysis of the trends in income inequality was normally undertaken for countries where poverty rates are below 40 percent of the population. Fiscal, environmental, and social sustainability aspects were addressed by a number of SCDs, but other important issues did not always receive attention. With IFC contributions, the role of the private sector in economic growth and poverty reduction was more prominent in the analysis. However, there is much room for improvement in the treatment of the private sector in the SCDs if it is to become the anchor of economic growth and poverty reduction. Threats to social and political stability were well analyzed in several cases, particularly when the analyses were able to draw on a fragility assessment, but not all SCDs provided a sufficiently robust analysis and identification of relevant social or political risks. A number of SCDs fell short in the identification and definition of priority areas for action. There was significant variability in the methodology used to derive priorities and in the number and nature of priorities identified. Often these were at a relatively high level of generality—areas to be addressed rather than specific actions to be taken. Some SCDs identified a great number of priorities. In a few good practice examples, after defining broad areas, the country team developed a set of prioritization criteria and identified core program priorities based on these criteria. The guidance for priority setting could be improved to encourage more consistent approaches to selecting a limited number of priorities. Governance was highlighted in most SCDs as a cross-cutting theme or prerequisite for achieving the twin goals. However, a clear approach to governance was missing in some SCDs, which could have benefited from more systematic cross-country comparisons on governance indicators and more explicit discussion of capacity constraints and plans to build institutional capacity in line with country- specific governance challenges. Independent Evaluation Group | World Bank Group 49 The preparation of the first batch of SCDs has been a learning experience for the World Bank Group and its stakeholders. The processes were broadly satisfactory, with numerous consultations within the World Bank Group at various stages of SCD preparation and review. Formal reviews enhanced the quality of reports. Corporate guidance sometimes helped to clarify the guidelines and procedures. The global Practices were engaged through participation in the task teams and the formal review meetings. However, integrating the work of multiple global practices in the preparation of the SCDs sometimes caused friction, although this was manageable. On the whole, the consultation processes were effective in reaching out to key stakeholders within countries to obtain inputs for the SCDs, and subsequently the CPFs. Consultations were held with governments and stakeholders, but to many stakeholders the difference between the SCD and CPF consultation processes was somewhat blurred. Some of the SCDs discussed elements of the government’s strategic priorities, but usually without in-depth analysis. The government’s own development strategy documents were not always available prior to SCD preparation. In some cases, there were benefits from the preparation of the SCD before or during preparation of the national plan. This enabled a dialogue at a stage where there was sufficient flexibility to develop a well-coordinated approach. Sharing of the SCD findings with country stakeholders has fallen short of serving the potential public good nature of the instrument. In some countries, the SCD document broke new analytical ground. In Albania, the SCD had an important public good dimension—assisting strategic dialogue with key stakeholders and helping to define an agenda for dialogue on key development issues in the country, enhancing broader country ownership not only of the World Bank Group program but also of the country’s own development agenda. All SCDs have been made publicly available, in some cases with a time lag, but there were significant weaknesses in the extent to which the document became part of the ongoing stakeholder dialogue, which reduced its value as a public good. Going forward, it is important for the World Bank Group to engage systematically and proactively to ensure wider distribution of the SCDs beyond meeting its own disclosure requirements. There is evidence of a modest improvement in the quality and focus of the CPFs in comparison with the previous CAS/CPSs. Most CPFs included discussions of the strategic direction and content of IFC and MIGA programs and how these contribute to the overall World Bank Group program. The focus on the twin goals in the SCDs helped program selection in the CPFs, and their selection of objectives has clearly improved. The requirement in the guidance note (World Bank Group 2014) for the intervention logic to be specified increased attention to the links between objectives and their indicators, and the definition of monitorable and measurable results for the indicators was a strong point of many of the CPFs reviewed. Most CPFs clearly benefited from the SCDs and from consultation processes. However, many did not systematically address the priorities identified in the SCD—strategic priorities, knowledge gaps, or data gaps. Most CPFs referred to the three criteria for selection of program priorities but were influenced heavily by the ongoing program. The World Bank Group’s guidance note for the CPF (World Bank Group 50 World Bank Group Country Engagement | Chapter 5 2014) identifies three criteria for the selection of program priorities: alignment with the priorities identified in the SCD, alignment with the government’s own priorities, and the World Bank Group’s comparative advantages. In practice, it was evident that there was a fourth criterion—the priorities reflected in the World Bank Group’s ongoing operations program, whether or not these were consistent with the other criteria. The SCD identification of development constraints and priorities was an important input to help the CPF focus the World Bank Group’s program. In a number of countries, the identification of key priorities by the SCD implied a significant shift in emphasis in the World Bank Group’s program. Although one-to-one alignment between the CPF and SCD priorities was not expected, the CPFs did not do a good job of explaining the reasons for their differences with the SCD’s priorities. And when there was tension between the new SCD priorities and the existing lending program and pipeline, there was little evidence in the CPFs of an assessment about the relevance of the ongoing program. One useful approach, found in a few CPFs, was to distinguish clearly between the ongoing programs and the new priorities that could be supported by the CPF. The lack of discussion of the alignment between the World Bank Group and government programs was an important shortcoming in the CPFs reviewed. Often the priority areas of the government plan were merely listed, with a statement that the World Bank Group objectives were aligned. In practice, even where there is apparent alignment, the weight that governments give to growth relative to poverty reduction and shared benefits may be different from that of the World Bank Group. All CPFs mentioned the comparative advantage of the World Bank Group as a criterion for the selection of priority objectives. In practice, the CPFs referred to absolute advantages such as the World Bank’s convening power or its ability to combine analytic work with operational support. The comparative advantage of the World Bank Group depends not just on its own capacities, but on those of the government and development partners as well. But the treatment of development partner programs was uneven. Only a handful of CPFs provided a systematic analysis of what other donors were doing and what the implications were for the World Bank Group program. There are indications—fewer objectives and well-defined indicators under each objective—that the programs under the CPFs are becoming more selective, but deficiencies remain in the results frameworks. Many of the operations listed were peripheral to the achievement of that objective. At the same time, the CPF format did not seem to give enough attention to nonlending activities, including institutional development and capacity-building programs. Indicators in the results framework tended to be based largely on the existing program. Results for the first three years were accounted for almost entirely by the existing program. A number of results frameworks were confined to indicators for the first three years and did not reflect potential new activities. The results framework needs to make a clearer distinction between the ongoing operations and the newer program objectives to be developed in the coming years, as has been done in a few of the CPFs. The definition of the CPF results chain remains a work in progress. The guidance note (World Bank Group 2014), which asks for the “intervention logic” to be specified in the results framework, has helped to focus more attention on the links between the objectives (defined broadly and directionally) Independent Evaluation Group | World Bank Group 51 and the particular objectives indicators (with targets) selected as proxies for the achievement of the objectives. The definition of monitorable and measurable indicators for the results was a strong point of many of the CPFs reviewed, and in general the reviews found this part of the results chain to be well handled. However, the role of the World Bank Group’s program in contributing to the achievement of these outcomes was often not clearly explained, and only in a very few cases did the results matrix include information about the role of the programs supported by other development partners in achieving the objectives. All CPFs used SORT, but the instrument may need to be adapted to increase its relevance at the country level. CPF teams have been conservative in their risk assessments. But the description of risks was often generic, and the risk mitigation or risk adaptation strategy was often vague, with a lack of a clear distinction between World Bank and IFC programs that normally have different risk profiles. It was also evident that some SORT elements have little relevance for country-level risk analysis. The SORT may need to be adapted for the CPF, with an increased focus on how the World Bank Group is addressing systemic operational risks at the country level and on the interplay between program design and operational risk. A CEN was used on a limited number of occasions when a full-scale SCD/CPF was seen as premature. The guidelines (World Bank Group 2014) stipulate that a CEN may be used in countries facing conflict or political crisis, or when there is reengagement after a prolonged hiatus, and lack of a knowledge base prevents formulation of detailed objectives or a medium-term program. The review of six country cases found inconsistencies in the application of current CEN guidelines. Half of the CENs reviewed lacked the basic elements that would allow an analysis of performance and lessons, and most of them did not make sufficient provision to ensure that key knowledge gaps would be filled in time for the teams to prepare an SCD and CPF before the end of the CEN cycle. There were also inconsistencies in the CENs on time dimensions, overall justifications, and length of the documents. The new country engagement model required more time to complete and more resources than the previous CAS. Compared with the latest CASs for the reviewed SCDs/CPFs, the time spent on production of the CPF was less than on the previous CAS, but the SCD required additional preparation time. Consequently, overall costs for the new model (for the World Bank alone) nearly doubled. For the two instruments together, the average elapsed time increased from 231 days to 388 days under the new model. The average processing times may decline as the World Bank Group moves up the learning curve for the new instruments. The One World Bank Group approach has promoted multisectoral consultations within the World Bank, and between the World Bank and IFC, with the latter providing analyses on private sector issues, but budget constraints remain. IFC’s participation in the SCDs and CPFs was broader than under the previous CAS approach, which led to improved coordination among World Bank Group entities. IFC was more involved in shaping the World Bank Group country strategy and gained more knowledge from the stakeholder consultations. IFC’s role in promoting corporate governance and improved labor and working conditions was also useful. MIGA’s small size, lack of country presence, and operational model did not allow for extensive participation in this process. IFC does not yet have 52 World Bank Group Country Engagement | Chapter 5 a mechanism for tracking the cost of its SCD and CPF engagement in individual countries and there is no allocated budget at the country level for SCD and CPF preparation. MIGA uses a time recording system code to track staff time and cost of its new country engagement, which is also funded through its operating budget. Like IFC, MIGA does not have a country-specific budget allocation for its SCD and CPF engagement. The new analytical approach to the private sector in the SCDs reflected a one-size-fits-all treatment of private sector issues, with limited new information coming from the diagnosis of private sector constraints. The level of IFC participation and the integration of its contribution varied in the SCD documents. With a few exceptions, the analysis of private sector contributions to the twin goals was largely from a government policy perspective of improving the business climate. The private sector analysis in SCDs provided little information on the profile, structure, and composition of the country’s private sector. Such analyses in future SCDs need to go beyond general policy constraints to include a more business-oriented and granular analysis of a country’s private sector. Coordination of the strategic focus among the World Bank Group constituent agencies has improved in the CPFs, but further improvement is needed. Most CPFs discussed the strategic direction and content of IFC and MIGA programs and how these contribute to the overall World Bank Group program, but this discussion was often not adequately reflected in the results framework. While ongoing programs of IFC and MIGA were generally listed, there was very little on their planned programs. Expectations of increased joint World Bank Group (cofinanced or parallel) projects need to be tempered by the country circumstances and investment climate, as well as by lessons from World Bank Group joint projects. The new approach did a reasonable job of addressing the cross-cutting themes—gender, climate change, and fragility—recommended in the guidance on SCDs and CPFs. Although the treatment of gender was not yet fully consistent with other Bank commitments on gender, compared with previous CASs, CPFs were generally better in articulating World Bank Group actions to address specific gender inequalities, and included many more sex-disaggregated indicators. SCDs indicated that the World Bank Group’s work on gender is evolving toward skills development and jobs. Climate change and environmental sustainability received attention in most countries reviewed, and 18 of the 22 CPFs included objectives to address it. But in a few countries where it was relevant, climate change was not adequately addressed. The availability of fragility assessments in countries with fragile situations appears to have made it easier for SCDs, CPFs, or CENs to integrate fragility within the World Bank Group strategy for those countries. Recommendations Recommendation 1: The World Bank Group should enhance its guidance and ensure more consistent and focused priority setting in SCDs and more explicit discussion of strategic priorities in CPFs. CPFs could be strengthened by a systematic discussion of the basis for the strategic priorities in the country program and indicating its alignment with or the reasons for divergence with each of the priorities identified in the SCD. Independent Evaluation Group | World Bank Group 53 Recommendation 2: The World Bank Group should disseminate the SCD to further enhance its potential use by other stakeholders in the country, with specific steps to be determined by each country team. With appropriate dissemination, the SCDs have the potential to underpin the development dialogue both with and in borrower countries. Recommendation 3: The CPFs should explicitly indicate how the SCD-identified knowledge and data gaps, which are most relevant to CPF objectives, will be addressed. SCDs depend heavily on available analytical material and data. The guidance would benefit from providing adequate discussion in the CPF showing its expected follow-up to the identified knowledge and data gaps. Recommendation 4: The World Bank Group should strengthen the results frameworks in CPFs to ensure that they more clearly reflect the ongoing and new work programs and results of all types of interventions. This could be addressed in four ways: first, by requiring CPFs to distinguish clearly between the ongoing programs and the new programs that reflect the strategic directions in the country, whose results will begin to materialize primarily in the outer years of the CPF; second, by more explicit inclusion in the results framework of nonlending advisory services and activities, as well as institutional development and capacity-building programs; third, by more explicit inclusion in the results frameworks of the current and future areas where IFC and MIGA intend to focus their support, considering constraints from their business models; and fourth, by providing corresponding training on results frameworks to country teams. Recommendation 5: The World Bank Group should modify the country engagement guidance to ensure consistency with the new gender strategy. The guidance on the country engagement model needs to be reconciled with the directions provided in the strategy to ensure its consistent application in SCDs and CPFs. Recommendation 6: The World Bank Group should improve resource management and budget transparency to strengthen One World Bank Group participation in the new country engagement model. Budget allocation and resource management systems could be strengthened at the country level by improving budget recording practices in the World Bank for greater consistency and to capture the full cost of preparing the SCDs and CPFs; and in IFC and MIGA, more clearly allocating, tracking, and managing resources used on the new country engagement products. 1 T  he PLRs are to be prepared every two years, or at the midpoint of the CPF, as short documents that briefly summarize progress in implementing the CPF program and review the continuous relevance of the strategy. In countries where significant changes have taken place, the PLR is expected to realign the strategy and the CPF program. The PLR can also be used to extend the time period of the CPF for up to two years, but no CPF may run for more than six years. In light of the changes to country programming under the new model, some adjustments to the current criteria for the CLRs and their independent validations could be warranted. Such adjustments might consider whether—and if so, how—to take account of (i) the new SCD instrument, (ii) the likely somewhat longer time between CPFs, and (iii) the possibility that programs might be adjusted, perhaps significantly, during the CPF period, including as little as two years before the end of the period. R eference World Bank Group. 2014. World Bank Group Guidance for Country Partnership Framework Products. Washing- ton, DC: World Bank Group. 54 World Bank Group Country Engagement | Chapter 5 bibliography The bibliography is divided into four sections: general documents that elucidate the topic, SCDs, CPFs, and CENs. The later three sections are alphabetized by country rather than by title of the work. I. General Das, Maitreyi. 2016. “Social Inclusion in Macro-Level Diagnostics: Reflecting on the World Bank Group’s Early Systematic Country Diagnostics.” Policy Research Working Paper 7713. Washington, DC: World Bank Group. Mahmud, W., S. Ahmed, and S. Mahajan. 2008. “Economic Reforms, Growth, and Governance: The Political Economy Aspects of Bangladesh’s Development Surprise.” Commission on Growth and Development Working Paper 22. World Bank for the Commission on Growth and Development, Washington, DC. Morra-Imas, Linda G., and Ray C. Rist. 2009. The Road to Results: Designing and Conducting Effective Develop- ment Evaluations. Washington, DC: World Bank. Parnini, S. 2006. “Civil Society and Good Governance in Bangladesh.” Asian Journal of Political Science 14 (2): 189–211. Porter, Douglas, Matt Andrews, Joel Turkewitz, and Clay Wescott. 2012. “Managing Public Finance and Procure- ment in Fragile and Conflict Settings.” International Public Procurement Journal 14 (4): 369–94. Rudra, N., and S. Sardesai. 2009. “Political Analysis of Decentralization: Capturing the Stakeholder Perspective— Lessons from the Bangladesh Study.” Social Development Notes 118. World Bank, Washington, DC. World Bank. 2011. World Bank Progress in Harmonization and Alignment in Low-Income Countries. Washington, DC: World Bank Group. ———. 2014a. The Big Business of Small Enterprises: Evaluation of the World Bank Group Experience with Target- ed Support to SMEs 2006–2012. Washington, DC: World Bank Group. ———. 2014b. Guidelines for Reviewing World Bank Implementation Completion and Results Reports: A Manual for Evaluators. Washington, DC: IEG. ———. 2014c. Results Frameworks in Country Strategies—Lessons from Evaluations. Washington, DC: World Bank Group. ———. 2014d. Results and Performance of the World Bank Group. Washington, DC: World Bank Group. ———. 2014e. Selectivity in Country Strategies: The Evidence. Washington, DC: World Bank Group. ———. 2014f. World Bank Group Assistance to Low-Income Fragile and Conflict-Affected Countries. Washington, DC: World Bank Group. ———. 2015a. “Bolivia: Completion and Learning Review for the Period FY2012–15. IEG Review.” Report 101189. IEG, Washington, DC. ———. 2015b. World Bank Group Engagement in Resource-Rich Developing Countries: The Cases of Bolivia, Ka- zakhstan, Mongolia, and Zambia—Clustered Country Program Evaluation—Synthesis Report and Manage- ment Response. Washington, DC: World Bank Group. ———. 2015c. Financial Inclusion—A Foothold on the Ladder toward Prosperity? Washington, DC: World Bank Group. Independent Evaluation Group | World Bank Group 55 ———. 2015d. Past and Future—Bank-IFC Cooperation at the Country Strategy Level. Washington, DC: World Bank Group ———. 2015e. The Poverty Focus of Country Programs: Lessons from World Bank Experience. Washington, DC: World Bank Group. ———. 2016a. Results and Performance of the World Bank Group. Washington, DC: World Bank Group. ———. 2016b. World Bank Group Engagement in Situations of Fragility, Conflict, and Violence. Washington, DC: World Bank Group. World Bank Group. 1998. Country Assistance Strategies: Retrospective and Outlook. Washington, DC: World Bank Group. ———. 2000. Country Assistance Strategies: Retrospective and Implications. Washington, DC: World Bank Group. ———. 2003. Country Assistance Strategies: Retrospective and Future Directions. Washington, DC: World Bank Group. ———. 2005. Results Focus in Country Assistance Strategies: A Stocktaking of Results-Based CASs. Washington, DC: World Bank Group. ———. 2009. Country Assistance Strategies: Retrospective and Future Directions. Washington, DC: World Bank Group. ———. 2010a. Development for All—A Gender Action Plan for the East Asia and Pacific Region FY11–13. Washing- ton, DC: World Bank Group ———. 2010b. Gender Equity in Latin America and the Caribbean: A Roadmap for Action of FY12–FY14. Washing- ton, DC: World Bank Group ———. 2011. World Development Report 2011: Conflict, Security, and Development. Washington, DC: World Bank Group. ———. 2012a. Africa Region Gender Action Plan. A Blueprint for Gender Informed Activities in the Region, FY2013- 17. Washington, DC: World Bank Group. ———. 2012b. “Good Practice Note: Integrating Gender into Country Assistance Strategies.” World Bank, Wash- ington, DC. ———. 2012c. Regional Gender Action Plan for the Middle East and North Africa Region, FY13–FY16. Washington, DC: World Bank Group. ———. 2012d. South Asia Regional Gender Action Plan FY2013–FY15. Washington, DC: World Bank Group ———. 2013a. ECA Regional Gender Action Plan FY2013–FY15. Washington, DC: World Bank Group. ———. 2013b. World Bank Group Strategy. Washington, DC: World Bank Group. ———. 2014a. Framework for Management of Risk in Operations: Proactive Risk Management for Better Results. Washington, DC: World Bank Group. ———. 2014b. Interim Guidelines for Systematic Country Diagnostics. Washington, DC: World Bank Group. ———. 2014c. Systematic Country Diagnostics: An Overview of Current Thinking. Advisory Group for SCD. Wash- ington, DC: World Bank Group. 56 World Bank Group Country Engagement | Bibliography ———. 2014d. World Bank Group: A New Approach to Country Engagement. Washington, DC: World Bank Group. ———. 2014e. World Bank Group Directive, Country Engagement. Washington, DC: World Bank Group. ———. 2014f. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. ———. 2015a. Systematic Country Diagnostic—An Update. Washington, DC: World Bank Group. ———. 2015b. World Bank Group Gender Strategy (FY16-23): Gender Equality, Poverty Reduction, and Inclusive Growth. Washington, DC: World Bank Group. ———. 2015c. World Bank Group Procedure: Country Engagement. Washington, DC: World Bank Group. ———. 2017. World Development Report 2017: Governance and the Law. Washington, DC: World Bank. II. Systematic Country Diagnostics (SCDs) World Bank Group. 2015. Next Generation Albania. A Systematic Country Diagnostic. Report 95980. Washing- ton, DC: World Bank Group. ———. 2015. Azerbaijan Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Bangladesh. More and Better Jobs to Accelerate Shared Growth and End Extreme Poverty. A Sys- tematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Bolivia Systematic Country Diagnostic. Rebalancing Inclusive Growth to Deepen Gains on Poverty and Inequality Reduction. Washington, DC: World Bank Group. ———. 2015. Rebalancing Bosnia and Herzegovina. A Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Botswana: Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Republic of Chad. Priorities for Ending Poverty and Boosting Shared Prosperity. Washington, DC: World Bank Group. ———. 2015. Colombia Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Costa Rica Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Côte d’Ivoire. From Crisis to Sustained Growth Priorities for Ending Poverty and Boosting Shared Prosperity. Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Egypt. Promoting Poverty Reduction and Shared Prosperity. A Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. El Salvador. Building on Strengths for a New Generation. Systematic Country Diagnostic. Washing- ton, DC: World Bank Group. ———. 2015. Haiti: Toward a New Narrative. Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Honduras. Unlocking Economic Potential for Greater Opportunities. Honduras Systematic Country Diagnostic. Washington, DC: World Bank Group. Independent Evaluation Group | World Bank Group 57 ———. 2015. Indonesia Systematic Country Diagnostic. Connecting the Bottom 40 percent to the Prosperity Gen- eration. Washington, DC: World Bank Group. ———. 2015. Maldives—Identifying Opportunities and Constraints to Ending Poverty and Promoting Shared Pros- perity. Washington, DC: World Bank Group. ———. 2015. Mali—Systematic Country Diagnostic: Priorities for Ending Poverty and Boosting Shared Prosperity. Washington, DC: World Bank Group. ———. 2014. Myanmar—Systematic Country Diagnostic: Ending Poverty and Boosting Shared Prosperity in a Time of Transition. Washington, DC: World Bank Group. ———. 2015. Panama—Locking in Success. A Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Serbia—Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015. Uganda—Systematic Country Diagnostic. Boosting Inclusive Growth and Accelerating Poverty Re- duction. Washington, DC: World Bank Group. ———. 2015. Uruguay—Systematic Country Diagnostic. Washington, DC: World Bank Group. III. Country Partnership Frameworks (CPFs) World Bank Group. 2015. Albania Country Partnership Framework for the Period FY2015–19. Washington, DC: World Bank Group. ———. 2015. Azerbaijan—Country Partnership Framework for the Period, FY2016–20. Washington, DC: World Bank Group. ———. 2015. Bangladesh—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Bolivia—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Bosnia and Herzegovina—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Botswana—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Chad—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Colombia—Country Partnership Framework for the Period FY2016–21. Washington, DC: World Bank Group. ———. 2015. Costa Rica—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Côte d’Ivoire—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. 58 World Bank Group Country Engagement | Bibliography ———. 2015. Egypt—Country Partnership Framework for the Period FY2015–19. Washington, DC: World Bank Group. ———. 2015. El Salvador—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015. Haiti—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015. Honduras—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Indonesia—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Maldives—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015. Mali—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015. Myanmar—Country Partnership Framework for the Period FY2015–17. Washington, DC: World Bank Group. ———. 2015. Panama—Country Partnership Framework for the Period FY2015–21. Washington, DC: World Bank Group. ———. 2015. Serbia—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015. Uganda—Country Partnership Framework for the Period FY2016–21. Washington, DC: World Bank Group. ———. 2015. Uruguay—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. IV. Country Engagement Notes (CENs) World Bank Group. 2015. Central African Republic—Country Engagement Note for the Period FY2016–17. Wash- ington, DC: World Bank Group. ———. 2015. Ecuador—Country Engagement Note for the Period FY2016–17. Washington, DC: World Bank Group. ———. 2015. Fiji—Country Engagement Note for the Period FY2015–17. Washington, DC: World Bank Group. ———. 2015. Guinea-Bissau—Country Engagement Note for the Period FY2015–16. Washington, DC: World Bank Group. ———. 2015. Turkmenistan—Country Engagement Note for the Period FY2016–17. Washington, DC: World Bank Group Independent Evaluation Group | World Bank Group 59 A PPE NDI X E S World Bank Group Country Engagement An Early-Stage Assessment of the Systematic Country Diagnostic and Country Partnership Framework Process and Implementation AN INDEPENDENT EVALUATION Appendix A. Evolution of Country Assistance Strategies Country strategies at the World Bank Group have evolved from internal management tools to public documents that strive to be country-owned, results-driven, and flexible.1 For most of its early history, the World Bank engaged with countries mainly though lending operations, without articulating a broader country strategy. Internal planning of the lending programs at the country level began in the early 1970s with the Country Program Papers. When the Board of Executive Directors began to seek information on the strategy for individual countries in the late 1980s, individual lending operations began to be put in the context of the World Bank’s program of country activities. Then, in 1990, the International Development Association (IDA) deputies asked that strategies be prepared for all IDA countries and discussed with the Board as part of World Bank operations. These early country assistance strategies (CASs) were 10- to 15-page notes appended to loan documents. They were World Bank-centric and rarely involved consultation with country governments or other stakeholders. The Board of Executive Directors extended the preparation of CASs to all active World Bank borrowers, International World Bank for Reconstruction and Development (IBRD) as well as IDA, in 1992. At that point, CASs were prepared in parallel with internal country strategy papers prepared for management. In 1994, the CAS became a free-standing document and the sole strategy document for both management and Board review. The change to a free-standing document effectively delinked CAS discussions from lending operations in 1996. With the advent of public disclosure in the late 1990s, the CAS process began to include consultations with key stakeholders, and ultimately became the center of World Bank-government interaction. The starting point for these CASs was the country’s own development strategy. To ground the CAS program in the country context, a good strategy started with a careful diagnosis of country conditions and a clear description of the government’s own priorities and development strategy. Over time, the coverage of the government’s agenda and development priorities improved, and by late 1997 the government’s development agenda and priorities were covered substantively in a separate section of the document. By then, more than 90 percent of CASs discussed the goals and implementation of the government’s ongoing policy and reform programs, and roughly two-thirds did so in some detail and relatively well. However, in fiscal 1998 only about 40 percent of CASs put ongoing reforms in perspective by examining the effectiveness of past reforms, and only 20 percent did so systematically. The early CASs focused mostly on World Bank products and activities, but as they evolved, they came to reflect more strategic engagement based on country development results. As CASs aimed to address an expanding range of issues, however, they became more complex, less strategic, and longer. They were being overloaded with goals and agendas. Addressing this concern, the first CAS retrospective, in 1998, devoted considerable attention to strategic selectivity, providing quantitative analysis of sectoral selectivity, defining strategic selectivity as a matter of qualitative judgment based on three criteria, and recommending actions to strengthen selectivity and focus.2 As experience grew, the focus of the CASs shifted from inputs—a series of products and activities—to development results. With this evolution, CASs started to be designed to attain 63 certain development outcomes. Results-based CASs were piloted between 2003 and 2005, and were mainstreamed after that. The results-based CAS was meant to help make country programs more strategic, as well as more selective. The central innovation was the results framework, which specified links between the World Bank’s interventions and long-term development goals. Measurable indicators of progress would be tracked through CAS implementation, encouraging active management of CAS programs and allowing both self- and independent evaluation. The new approach was expected to create a better strategy, with greater chances of success and development effectiveness (World Bank Group 2005). In fiscal 2005, management established a four-year CAS cycle with a midpoint progress report. It also introduced the Interim Strategy Note (ISN) for use when a full CAS could not be prepared, as typically was the case in fragile or conflict-affected states or situations (FCS), and it initiated the use of the CAS Completion Report. Around this time, the World Bank also adopted a new title for the documents. The country partnership strategy (CPS), as it was now called, was thought to better reflect a relationship with the country client based on mutual learning and responsibility. As it evolved, the CAS also became more flexible, allowing World Bank teams to customize World Bank services to the country’s own needs and priorities as well as to changing circumstances. Changes in country demand during the CAS period meant that the World Bank needed to remain flexible during the CAS implementation to ensure continued alignment with country priorities. In higher-income countries with access to alternative financing and knowledge services, as well as in fragile and conflict-affected states—where needs can change quickly—the World Bank needed to change and adapt its services during implementation. As the importance of the CAS increased, it became clear that IBRD- and IDA-funded activities would have greater development impact if they were designed and implemented as part of an integrated World Bank Group–wide approach. CAS guidelines called for joint World Bank-IFC CASs in all countries where there was substantial scope for collaboration and synergies in private sector development activities. Synergies ultimately would be a matter of collaboration at the operational level, but World Bank staff believed that joint CASs could set the stage by articulating the private sector role—and thus IFC’s role—in addressing key development challenges. CASs are often implemented in uncertain environments, and are thus subject to risks. All CASs now have a section that discusses the main risks, and what the World Bank intends to do in case a given risk materializes. However, the 2009 retrospective of CASs (World Bank Group 2009) noted that even when the risks were well articulated and comprehensive, it was difficult to distinguish how the perceived risk had been addressed in the development of the strategy, the instruments, and the choice of interventions. The 2009 CAS retrospective also pointed out that a CAS was expected to reflect the existing state of country knowledge; the country program; dialogue with government, donors, and others; and relationships with stakeholders. While the CAS was not meant to be analytic, upstream analysis, 64 World Bank Group Country Engagement | Appendix A including analysis of the sources of growth, macroeconomic sustainability, determinants of poverty, and public expenditure and financial management systems, was expected to be carried out regularly, in advance of CASs, and to explicitly inform the CAS diagnosis and the design of World Bank support. Despite the evolution of CASs, World Bank staff have identified four important shortcomings: ƒƒ The World Bank’s sector and corporate objectives (such as for infrastructure, finance, and social security) have often created tension with the country’s vision and priorities. The government may or may not prioritize the strategic objectives that the World Bank espouses. Moreover, the World Bank has not always appeared to invest resources in a way that would maximize the chances of achieving both country and World Bank objec- tives. Weak results frameworks were partly to blame—as a list of activities spread across many sectors failed to constitute the critical mass of interventions needed for maximum development impact. ƒƒ Despite the emphasis on results, some CASs have still taken a transaction-focused approach instead of concentrating on more strategic objectives and expected develop- ment outcomes. While World Bank interventions have broadly aligned with the mission of poverty reduction, in some cases they have lacked focus and have not concentrated on issues likely to have the greatest impact on countries’ development. A need remained for more selectivity and focus. ƒƒ The World Bank and IFC have not operated in unison in countries. While the formality of joint CASs after fiscal 2009 enhanced World Bank-IFC cooperation during strategy formulation, cooperation during program implementation has remained erratic. ƒƒ The discussion of risks has been weak in many CASs, without explicit links between identified risks and the CAS program, or without sufficient discussion of the ways in which risks would be managed. A more pointed identification of risks—and measures to deal with them— was required in CASs. A New World Bank Group Strategy and Its Instruments Against this background and the weaknesses identified, the new World Bank Group strategy (World Bank Group 2013) aims to focus country programs on accelerating progress toward reducing poverty and building shared prosperity (known as the twin goals) in a sustainable manner. It recognizes that while today’s country strategies are broadly aligned with the World Bank Group’s mission, they need to provide countries with more coordinated and focused programs, and activities and interventions need to be better prioritized according to their expected impact on the twin goals. It also recognizes that the World Bank Group needs to establish an effective mechanism to reconcile country demands with the twin goals and move away from any vestiges of a supply-driven approach. Independent Evaluation Group | World Bank Group 65 The new strategy sets two ambitious goals: end extreme poverty by reducing the percentage of people living on less than $1.90 a day (in 2011 purchasing power parity terms) to 3 percent by 2030 and promote shared prosperity by fostering income growth of the bottom 40 percent of the popu- lation in every country in a sustainable manner. To accomplish these goals, the World Bank Group would: align all its activities and resources to the twin goals, use its strengths as a global institution by supporting clients in delivering development solutions, and work as “One World Bank Group” by le- veraging the strength of all constituent agencies (World Bank, International Finance Corporation [IFC], and Multilateral Investment Guarantee Agency [MIGA]). The World Bank Group would strengthen the focus of its country programs by developing a more evidence-based and selective country engage- ment model in the context of country ownership and national priorities, and in coordination with other development partners. Implementation of the strategy started in 2014 with a reorganization of the World Bank Group into global practices. This change was intended to help accelerate progress toward the twin goals through a new form of problem-solving engagement with countries. In parallel, the World Bank Group’s engagement with country members is being revamped (World Bank Group 2014b), as outlined below, to make country programs more strategic and to strengthen analytical aspects of the engagements, ensuring that the twin goals are at the center of the engagements, while also concen- trating efforts and resources in areas that are key for the country development strategy. Two instruments will underpin the enhanced country engagement model (figure A.1). A new Sys- tematic Country Diagnostic (SCD) will use data and analytic methods to support country clients and World Bank Group teams in identifying the most critical constraints to, and opportunities for, reducing poverty and building shared prosperity sustainably, while considering the voices of the poor and the views of the private sector. The Country Partnership Framework (CPF), evolving from the CAS, will discuss focus areas for World Bank Group support, aligned with the country’s own development agenda and selected primarily to address the key constraints and opportunities identified in the SCD. Figure A.1. The New Approach to Country Engagement Source: World Bank Group 2014b, 6. The SCD will be prepared upstream from the CPF and will identify priority areas for the country to accelerate progress toward achieving the twin goals in ways that take into account the sustainability of the program. The SCD will be prepared in consultation with national authorities, the private sector, and other stakeholders, but it will reflect the evidence and analysis of the World Bank Group. It will become the reference point for client consultations and help focus the country, the World Bank Group, and other development partners on goals with the most potential for impact in achieving the twin goals. 66 World Bank Group Country Engagement | Appendix A The CPF will replace the CAS as the main tool guiding the World Bank Group’s support for member countries’ development programs. The CPF will maintain the World Bank Group’s country-driven model and will start from the country’s vision of its development goals as set out in its own strategy. The World Bank and the country will further draw on the SCD and will develop the CPF objectives to- gether. The objectives will be derived from the country development goals that reflect the World Bank Group’s comparative advantage and align with the twin goals, as identified in the SCD. The keys to selecting CPF objectives require that each objective: (a) focuses on areas identified by the SCD or other analysis as critical for achieving the twin goals in a sustainable manner, (b) is consis- tent with the World Bank Group’s comparative advantage and capacity to deliver based on expe- rience, and (c) is congruent with the country’s own development goals and capacity to deliver (see figure A.2). In the end, while selectivity is multidimensional and complex, the Independent Evaluation Group’s (IEG’s) evidence from country program evaluations and country strategies suggests that selectivity can be enhanced by paying careful attention to client country capacities, grounding all operations in solid analytical work, taking a long-term view and sequencing interventions accordingly, properly factoring in design and implementation risks to results delivery, and taking account of the World Bank’s comparative advantage (World Bank 2014). Figure A.2. CPF Selectivity Filters Source: World Bank Group 2014c, 10. The new approach is expected to emphasize learning at all stages. During implementation, the coun- try and the World Bank Group are to engage in a continuous process of monitoring and learning. Every two years, this will lead to a Performance and Learning Review (PLR, the third block in figure A.1), which will be used to adjust the program and to update the Board on progress. The PLR will be used to flesh out details of development objectives and the activities necessary to achieve those objectives that were not clear when the CPF was originally prepared or last updated. At the end of the CPF period, a PLR may be used to extend the CPF by up to two years. At the end of the CPF period, the World Bank Group will engage with the country to complete a self-assessment and issue a Completion and Learning Review (CLR, the fourth block in figure A.1). Independent Evaluation Group | World Bank Group 67 The CLR will be both a means to enhance the World Bank Group’s knowledge and to increase devel- opment effectiveness and a tool for accountability. As part of the CLR, the country team will complete a self-assessment of the program’s performance, World Bank Group performance, and how well aligned the strategy was with the twin goals. The self-assessment will be based on the results framework of the most recent PLR, and IEG will validate the World Bank Group’s self-assessments and verify its findings. Working as One World Bank Group—intra-agency cooperation—is central to the new strategy. The CPF is expected to mainstream joint business planning as the backbone for strengthening oper- ational collaboration. The World Bank Group is expected to increase the number of joint projects and review its portfolio of products and services to improve synergies and eliminate overlap. These initiatives will be reinforced through stepped-up efforts to align policies and practices and promote changes in the operational culture of each agency. A new Regional Coordination Mechanism is to formalize country- and regional-level coordination among the World Bank, IFC, and MIGA, and help the World Bank Group with its regional engagements. The CPF is designed to improve the identification and managing of risks to the World Bank Group program. The risk section will take a systematic approach to risk, based on the Systematic Opera- tions Risk-Taking Tool (SORT; figure A.3), to ensure that teams consider all major risks to achieving CPF objectives, as well as any unintended adverse consequences that may be associated with the program (World Bank Group 2014a). The risk section is designed to increase transparency by better informing stakeholders of program risks. Teams are expected to identify risks and propose measures to manage or mitigate them, and may also discuss the risks of not engaging in the proposed pro- gram, thereby highlighting the program’s risk-reward trade-offs. Figure A.3. Likelihood of Risk and Impact Source: World Bank Group 2014a, 5. 68 World Bank Group Country Engagement | Appendix A The risk assessment will take into account the likelihood of the risks materializing and the likely se- verity of their impacts on the achievement of the intended results. The assessment will be based on current residual risk—that is, after considering the impact of mitigation measures that have already been implemented, but not presuming any future additional mitigation measures (figure A.3). The risk categories have been designed with the intention of capturing the key risks that could affect programs in CPFs. The categories are: political and governance, macroeconomic, sector strategies and policies, technical design of program, institutional capacity for implementation and sustainability, fiduciary, environment and social, stakeholders, and other. References World Bank. 2014. Selectivity in Country Strategies: The Evidence. Washington, DC: World Bank. World Bank Group. 1998. Country Assistance Strategies: Retrospective and Outlook. Washington, DC: World Bank Group. ———. 2000. Country Assistance Strategies: Retrospective and Implications. Washington, DC: World Bank Group. ———. 2003. Country Assistance Strategies: Retrospective and Future Directions. Washington, DC: World Bank Group. ———. 2005. Results Focus in Country Assistance Strategies: A Stocktaking of Results-Based CASs. Washington, DC: World Bank Group. ———. 2009. Country Assistance Strategies: Retrospective and Future Directions. Washington, DC: World Bank Group. ———. 2013. World Bank Group Strategy. Washington, DC: World Bank Group. ———. 2014a. Framework for Management of Risk in Operations: Proactive Risk Management for Better Results. Washington, DC: World Bank Group. ———. 2014d. World Bank Group: A New Approach to Country Engagement. Washington, DC: World Bank Group. ———. 2014c. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. 1 This section is based on four country assistance strategy retrospectives (World World Bank Group 1998, 2000, 2003, and 2009) and on World World Bank Group 2005. 2 Potential magnitude of impact, likelihood of country action, and additionality of World World Bank contribution. Independent Evaluation Group | World Bank Group 69 Appendix B. Methodology Methodological Framework The evaluation was designed around a comparative assessment framework of Systematic Country Diagnostics (SCDs) and Country Partnership Frameworks (CPFs). The methodological design includ- ed the following components: ƒƒ Assessment templates were developed for the SCDs and CPFs, each comprising 4 dimensions with 22 (for SCD) and 28 (for CPF) indicators/questions, respectively. ƒƒ Twenty-two desk-based assessments of the SCDs and CPFs were conducted following the dimensions and indicators of the templates. These covered all documents for which the CPFs had been discussed by the Board through end of April 2016. ƒƒ Ten selected country visits (out of the 22 countries) were conducted to validate and deepen the desk-based assessments. The selection of countries is elucidated in appen- dix C. ƒƒ As part of the above, interviews were conducted with selected stakeholders at the head- quarters of the World Bank Group and at the country level as part of the country visits. In addition, interviews were conducted with other World Bank Group representatives en- gaged in the SCD and/or CPF processes, including all six World Bank chief economists. The templates for assessing the SCDs and CPFs were initially designed on the basis of the guidelines for SCD and CPF preparation, the World Bank Group Strategy of October 2013, and a note on the new approach to World Bank Group country engagement. Subsequently, they were applied to a few pilot cases, and the final version of templates incorporated the lessons learned from those pilots, as well as lessons learned by IEG and presented in learning notes on results frameworks, selectivity, and World Bank-International Finance Corporation (IFC) cooperation. Systematic Country Diagnostics The following dimensions were assessed in the SCD review—these were determined as the core dimensions in the respective SCD guidelines: ƒƒ Evidence basis and framing of the issues ƒƒ Growth, poverty and shared prosperity, and sustainability ƒƒ Identification of priorities for achieving the twin goals ƒƒ Process and One World Bank Group. 70 The accompanying template provides the questions that were answered for each dimension. The evaluators rated each dimension using a six-point scale based on criteria set out in the attached rating scale table (table B.1). Evidence basis and framing of the issues assessed the quality of the background analytical work and data the SCD used to frame the main issues and challenges facing the country in question. It also as- sessed how the SCD identified and addressed data gaps and weaknesses found during its production. Growth, poverty and shared prosperity, and sustainability assessed whether the SCD convincingly showed the key opportunities and constraints the country faces. It assessed whether sources of growth were properly identified, their impact on poverty and inclusiveness, and prospects of eco- nomic growth (fiscal and debt), social, and environmental sustainability. These issues were analyzed against the background of the twin goals. Identification of priorities for achieving the twin goals assessed whether the SCD identified some key areas (sufficiently narrow in scope to reflect an effective prioritization process) on the critical path to reducing poverty and enhancing shared prosperity, including progress achieved so far. The report assessed wheth- er future priorities are in the context of growth, inclusiveness, and sustainability. This dimension also as- sessed whether the SCD had an adequate discussion of trade-offs between short- and long-term goals. Process and One World Bank Group assessed whether the production of the SCD, and any consulta- tion, reflected a unified approach by the World Bank, IFC, and MIGA to underpin the new country en- gagement model. It also assessed the quality of corporate review, and the adequacy of the World Bank Group’s interactions with the government and civil society stakeholders during SCD preparation. Country Partnership Frameworks The following dimensions were assessed in the CPF review. These were designated as the core dimensions in the respective CPF guidelines: ƒƒ Alignment with country development priorities and the World Bank Group’s opportunities to achieve the twin goals identified in the SCD, and sustainability ƒƒ Country ownership, realism, and selectivity ƒƒ Quality of results framework and systematic risk assessment ƒƒ One World Bank Group. As with the SCD, for each dimension of the assessment the evaluators provided a rating on the six- point scale based on criteria set out in the attached rating scale table. Alignment with country development priorities and the twin goals and sustainability assessed whether the CPF (following its guidance note, World Bank Group 2014) found the proper intersection Independent Evaluation Group | World Bank Group 71 between country development goals and opportunities to achieve the twin goals identified in the SCD when putting together the World Bank Group interventions and the results framework of the program. Clarity of the country’s development strategy and definition of its priorities in the CPF were key in that assessment. The sustainability of policies was also assessed in this section. Country ownership, realism, and selectivity assessed the extent to which the CPF took advantage of consultations with stakeholders—and particularly with the government—to develop a program whose success will be enhanced by an adequate assessment of country capacity and government com- mitment to the policies. This section also assessed selectivity by evaluating whether resources were concentrated in areas representing an adequate intersection of country development goals, the twin goals, and the World Bank Group’s comparative advantage. And it looked at whether proposed an- alytic work was relevant and consistent with CPF priorities and SCD findings. Moreover, it assessed the extent to which the CPF adequately considered major risks to the implementation of the World Bank Group program, and options to manage the most serious ones. Quality of results framework and systematic risk assessment assessed the appropriateness and real- ism of the results framework, focusing on the results chain running from World Bank Group interven- tions to specified outcomes. It also assessed whether objectives were achievable, outcome indica- tors were appropriate to measure progress toward the objectives, and whether they were measurable and systematically tracked. The CPF results frameworks were compared with the results frameworks for the previous CASs for the same countries. One World Bank Group assessed evidence of appropriate collaboration between the World Bank, IFC, and MIGA in working toward CPF objectives, and whether the results framework offered evi- dence of joint interventions by these institutions. Documentation The IEG reviewers had access to relevant documentation for the SCD and CPF review. The main documents were the following: ƒƒ Concept notes for the SCD and CPF ƒƒ SCD decision draft ƒƒ CPF decision draft ƒƒ CPF Board version ƒƒ Minutes of the Operations Committee and Regional Operations Committee (ROC) meet- ings for the SCD and CPF ƒƒ Comments provided in the context of the Operations Committee and ROC meetings 72 World Bank Group Country Engagement | Appendix B ƒƒ Latest International Monetary Fund Article IV Consultation ƒƒ Other relevant recent reports. Interviews with Management, Staff, Borrower, Stakeholders, and Partners After the desk review and initial assessment of the SCD and CPF, the reviewers identified information gaps and determined whether it was necessary to interview specific people, specifying the proposed agenda and questions. The interview took the form of e-mails, phone calls, face-to-face meetings, or short questionnaires. During field visits, the IEG teams met stakeholders in the country (from gov- ernment, disenfranchised groups, academia, the private sector, and nongovernmental organizations, among other groups) and multilateral and bilateral partners, to validate consistency of views reflected in the SCD and CPF documents. The task manager for the evaluation signed off on all the interviews. Quality Assurance of the SCD/CPF Assessments Reviewers received detailed instructions on the application of the templates. To further ensure consistency between assessments, to reduce possible inter-reviewer bias, and to enhance their quality, all assessments were reviewed by the task team leader. Any differences in viewpoints were discussed and resolved through dialogue between the task team leader and the reviewers of the individual assessments. Basic Template for Systematic Country Diagnostic and Country Partnership Process Evaluation Systematic Country Diagnostic Evidence basis and framing of the issues ƒƒ To what extent is the SCD providing a basis for outlining the main issues and challenges for the country? ƒƒ Does the SCD identify gaps in evidence and areas where additional work is required, as an input to the CPF? ƒƒ Are the quality of the data discussion and the steps that have been taken to address data gaps and weaknesses, including the proxies used for missing data, appropriate? ƒƒ Are there synergies on the priorities and opportunities identified across the themes of growth, inclusion, and sustainability? Growth, poverty and shared prosperity, and sustainability ƒƒ To what extent is the SCD identifying key opportunities for overall economic growth? Independent Evaluation Group | World Bank Group 73 ƒƒ To what extent is the SCD analyzing adequate and equal opportunities, especially the distribution of the gains from growth among the poor and the bottom 40 percent, and basic elements of social well-being and shared prosperity? ƒƒ Does the SCD look sufficiently into sustainability issues (fiscal and debt, social, and environmental)? ƒƒ Are significant threats to social and political stability adequately analyzed? Identification of priorities for achieving the twin goals ƒƒ Does the SCD identify limited areas that are on the critical path to achieving the goals? ƒƒ Are the priorities for poverty reduction and shared growth clearly spelled out in quantita- tive terms and are they realistic? ƒƒ Is the results chain needed to achieve the goals clearly specified and consistent with the objectives? ƒƒ Is there a discussion of possible trade-offs between short- and long-term goals? ƒƒ To what extent is governance (institutional arrangements and capacity, transparency, and accountability) addressed in the SCD? ƒƒ Does the SCD distinguish clearly between the most important priorities and other, less critical constraints? ƒƒ Does the SCD effectively address cross-cutting issues (gender, climate change, jobs, and fragility)? Process and One World Bank Group ƒƒ How generally did the processes go, including whether there were any issues from the relative timing of the SCD and the CPF? ƒƒ Was the corporate review process of adequate quality? ƒƒ Was the preparation of the SCD reflective of the One World Bank Group objective, with IFC and MIGA as an integral part of the team? How was the engagement with the global practices? ƒƒ Were the SCD findings and priorities informed by stakeholder consultation? ƒƒ Were stakeholder comments presented in the report? 74 World Bank Group Country Engagement | Appendix B ƒƒ Were those who participated in consultation informed as to how the World Bank has dealt with their comments and suggestions? ƒƒ Were time constraints important for the quality of the report? Country Partnership Framework Alignment with country development priorities and the twin goals and sustainability ƒƒ Does the CPF present an assessment of the government’s program and make clear any divergence of views between the country and the World Bank Group? ƒƒ Does it highlight any country poverty or prosperity goals and discuss how they align with the twin goals? ƒƒ Does the CPF provide a concise description of the country’s poverty and income distri- bution patterns and trends, including at least poverty trends using national and interna- tional poverty lines, inequality measures, and income/consumption growth of the bottom 40 percent of the population? ƒƒ Does the CPF present information on the causes of poverty, including the major obsta- cles to poverty reduction and the set of structural and social elements that are essential to poverty reduction? ƒƒ Is there a discussion of any progress in poverty reduction since the last strategy? ƒƒ Is there an analysis of the linkages between poverty reduction and the level and pattern of growth in the country? ƒƒ Is there an assessment of the country’s capacity to monitor poverty indicators? ƒƒ Are cross-cutting issues (gender, climate change, jobs and the fragility) given sufficient attention in the document and are links to poverty/shared prosperity and sustainability (fiscal, social, environmental) articulated well in the CPF results framework? Country ownership, realism, and selectivity To what extent ƒƒ were there consultations with stakeholders, and were those consultations reflected in broad commitment (particularly from the government) and in the design of the strategy? ƒƒ is the World Bank Group’s country program matched to country capacity and appropri- ately estimated government commitment? Independent Evaluation Group | World Bank Group 75 ƒƒ are the program objectives consistent with the World Bank Group’s comparative advantage? ƒƒ do CPF objectives concentrate resources into critical areas where they will have the most development impact (intersection of country development goals, twin goals, and World Bank Group comparative advantage)? Quality of results framework and systematic risk assessment ƒƒ Does the CPF articulate in its results framework—measured with relevant indicators—the links of World Bank Group activities to poverty, shared prosperity, and sustainability? ƒƒ Does the framework emphasize outcomes over outputs/inputs/processes? ƒƒ Are all outcomes measurable and measured (with a system set up for measuring them), including through indicators? ƒƒ Do indicators have quantified baselines and targets? ƒƒ Do the chosen indicators adequately measure final outcomes (say, quantity or quality) and have the proper scope (say, same geographic coverage as the final outcome)? ƒƒ Are the indicators time-bound and consistent with time-dimension of the CPF’s targeted outcomes? ƒƒ Are results chains from interventions to outcomes explained sufficiently, including any scaling-up of interventions expected to achieve the targeted outcomes? Are obstacles to be overcome, logical causality, assumptions, and risks highlighted? ƒƒ Were the criteria for selecting the ongoing portfolio and the portfolio under preparation discussed explicitly in the CPF, and was the overall program contribution to CPF objec- tives explained? ƒƒ Is proposed analytic work relevant and consistent with CPF priorities and SCD findings? ƒƒ To what extent does the CPF consider major risks (domestic, international, and World Bank-related) and options to manage the most serious ones? One World Bank Group ƒƒ Is there evidence of appropriate collaboration and appropriate division of labor between the World Bank, IFC, and MIGA in working toward CPF objectives? ƒƒ Is there evidence of smooth working of the Regional Coordination Mechanism, with joint implementation work (World Bank, IFC, MIGA) where relevant, and timely inclusion of IFC and MIGA in joint work? 76 World Bank Group Country Engagement | Appendix B ƒƒ How meaningful was donor coordination in the preparation of the CPF? ƒƒ Are there program complementarities (World Bank, IFC, MIGA) in achieving common goals, and are those complementarities expressed in the results matrix? ƒƒ Is IFC working on poverty issues? Are IFC and MIGA interventions included in the results framework with appropriate, measurable indicators? ƒƒ Is there evidence of joint work of the World Bank, IFC, and MIGA in specific interventions? Rating Scale for SCD/CPF Process Questionnaire For internal evaluation purposes, the responses to the questions were rated on a six-point scale using the criteria in the table below. This rating system was a tool for the aggregation and reporting of specific evaluation findings. The ratings on individual questions were aggregated to arrive at an overall rating for each block of issues under the SCD process and the CPF process. The overall rating for each block was the simple average of the ratings for each question under the block, giving equal weight for all the questions. The possibility of using unequal weights was considered during the eval- uation if warranted by special issues, but no such weighting was found necessary. Table B.1. SCD/CPF Process Questionnaire Rating Scale Question or Block Rating Criteria for the Rating Numerical Rating Highly Exceptional attainment on the issue raised by the question or 6 satisfactory dimension. No shortcomings identified. Satisfactory Good attainment on the issue raised or dimension. No major 5 shortcomings identified. Moderately The issue or dimension was addressed but not comprehensively, or 4 satisfactory was addressed in a way not fully helpful for the development of the SCD or CPF. Some shortcomings identified. Moderately The issue or dimension was addressed in a way that was not helpful for 3 unsatisfactory the SCD or CPF. Several shortcomings identified. Unsatisfactory The issue or dimension was addressed perfunctorily, representing 2 a detriment to the quality of the SCD and CPF. Many shortcomings identified. Highly The issue or dimension was not addressed by the SCD or CPF. 1 unsatisfactory Reference World Bank Group. 2014. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. Independent Evaluation Group | World Bank Group 77 Appendix C. List of Country Case Studies Table C.1. List of SCD/CPF Country Case Studies Region Country SCD Date CPF Board Date AFR Côte d’Ivoire July 2015 September 2015 Botswana March 2015 November 2015 Chad September 2015 December 2015 Mali July 2015 December 2015 Uganda December 2015 April 2016 EAP Myanmar December 2014 April 2015 Indonesia October 2015 December 2015 ECA Albania April 2015 May 2015 Serbia June 2015 June 2015 Azerbaijan June 2015 July 2015 Bosnia and Herzegovina December 2015 December 2015 LCR Panama February 2015 April 2015 Costa Rica May 2015 May 2015 El Salvador May 2015 June 2015 Haiti June 2015 September 2015 Bolivia June 2015 November 2015 Honduras October 2015 December 2015 Uruguay June 2015 January 2016 Colombia June 2015 April 2016 MNA Egypt, Arab Rep. September 2015 December 2015 SAR Bangladesh September 2015 April 2016 Maldives November 2015 April 2016 Note: Boldface indicates where country visits have taken place. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LCR = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. 78 Table C.2. Desk Review of Country Engagement Notes CEN Board CEN Period Borrower Fragile Country Date (Fiscal Year) Status Country Classification Situation Fiji March 2015 2015–17 IBRD Upper-middle income Guinea-Bissau March 2015 2015–16 IDA Low income Fragile Central African Republic July 2015 2016–17 IDA Low income Fragile Turkmenistan Nov. 2015 2016–17 IBRD Upper-middle income Ecuador March 2016 2016–17 IBRD Upper-middle income Guyana March 2016 2016–18 Blend Lower-middle income Note: CEN = Country Engagement Note; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. Table C.3. Case Study Countries by Borrower Status  Institution AFR EAP ECA LAC MNA SAR IDA Chad Myanmar Haiti Bangladesh Côte Honduras Maldives d’Ivoire Mali Uganda IBRD Botswana Indonesia Albania Colombia Egypt, Arab Rep. Azerbaijan Costa Rica Bosnia and El Salvador Herzegovina Serbia Panama Uruguay Blend Bolivia Note: Boldface indicates where country visits have taken place. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; IBRD = International World Bank for Reconstruction and Development; IDA = International Development Association; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. Independent Evaluation Group | World Bank Group 79 Table C.4. Case Study Countries by Income Status  Income level AFR EAP ECA LAC MNA SAR High income Uruguay Upper-middle Botswana Albania Colombia income Azerbaijan Costa Rica Bosnia and Panama Maldives Herzegovina (fragile) Serbia Lower-middle Côte Myanmar El Salvador Egypt, Bangladesh income d’Ivoire (fragile) Arab (fragile) Rep. Indonesia Bolivia Honduras Low income Chad Haiti (fragile) (fragile) Mali (fragile) Uganda Note: For fiscal year 2016. Boldface indicates where country visits have taken place. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. Table C.5. List of SCDs Completed Outside the Evaluation Time Frame Region Country Delivery to Board Date AFR Mauritius July 2015 Lesotho July 2015 Madagascar August 2015 South Sudan October 2015 Ethiopia March 2016 Mozambique May 2016 Cameroon June 2016 Guinea-Bissau June 2016 Togo September 2016 EAP Pacific Islands January 2016 Vietnam September 2016 Thailand November 2016 80 World Bank Group Country Engagement | Appendix C Table C.5, continued. ECA Bulgaria June 2015 Montenegro May 2016 Uzbekistan June 2016 Moldova September 2016 LCR Belize January 2016 Brazil February 2016 Guatemala May 2016 MNA Lebanon June 2015 Tunisia June 2015 Jordan February 2016 SAR Afghanistan February 2015 Sri Lanka October 2015 Note: IEG reviewed 22 SCDs that had corresponding CPFs prepared before the evaluation cutoff date of end of April 2016. As of the end of November 2016, the total number of SCDs prepared is 46. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. Table C.6. List of CPFs and CENs Completed Outside the Evaluation Time Frame Region Country Delivery to Board Date CPF AFR Lesotho June 2016 ECA Bulgaria May 2016 Uzbekistan June 2016 Montenegro June 2016 LCR Guatemala November 2016 MNA Tunisia May 2016 Lebanon July 2016 Jordan July 2016 SAR Sri Lanka June 2016 Afghanistan October 2016 CEN EAP Cambodia May 2016 MNA Yemen July 2016 Note: IEG reviewed 22 CPFs that had corresponding SCDs and 6 CENs prepared before the evaluation cutoff date of end of April 2016. As of the end of November 2016, the total number of CPFs prepared is 32 and CENs is 8. CEN = Country Engagement Note. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. Independent Evaluation Group | World Bank Group 81 Appendix D. Poverty Reduction and Shared Prosperity in Systematic Country Diagnostics In 2013, as part of its new corporate strategy (World Bank Group 2013), the World Bank Group ad- opted two ambitious goals: ending extreme poverty globally and boosting shared prosperity in every country in a sustainable way. This appendix assesses progress toward meeting the first goal on the basis of changes in the rate of extreme poverty, using the World Bank’s newly estimated international poverty standard of $1.90 a day in income or consumption expenditure. The target is to reduce the extreme poverty headcount rate from 12.7 percent of the population in developing countries in 2012 to 3 percent by 2030. The second goal is measured by the growth in the average income or con- sumption expenditure of the poorest 40 percent of the population in each country. A major challenge in measuring progress toward achieving the twin goals is the lack of reliable data on a significant number of economies. Monitoring trends in poverty requires conducting household surveys at least every three years. But despite some progress, 21 countries still have no poverty estimates, and 74 countries lack the basic data required to monitor poverty and shared prosperity between two points in time within a 10-year period.1 As part of its evaluation of the Systematic Country Diagnostic (SCD) and Country Partnership Frame- work (CPF), the Independent Evaluation Group reviewed 22 SCDs.2 Part of its review examined the quality of the data and analysis pertaining to the twin goals. It posed the following questions: ƒƒ Are the quality of the data discussion and the steps that have been taken to address data gaps and weaknesses, including the proxies used for missing data, appropriate? ƒƒ To what extent is the SCD analyzing adequate and equal opportunities, especially the distribution of the gains from growth among the poor and the bottom 40 percent and basic elements of social well-being and shared prosperity? ƒƒ Are the priorities for poverty reduction and shared growth clearly spelled out in quantita- tive terms and are they realistic? Following are the main findings from this review. Data Quality ƒƒ Eight of the 22 country cases reviewed (Botswana, Chad, Côte d’Ivoire, Arab Republic of Egypt, Haiti, Mali, Maldives, and Myanmar) had significant data gaps or out-of-date data, and 3 had some data gaps (Azerbaijan, Honduras, and Maldives). The data inadequacies adversely affected the quality and relevance of the analysis of poverty and shared pros- perity, particularly over the three- to five-year period leading to the preparation of their SCDs. Most, but not all, of the SCDs reviewed identified data issues where they existed. ƒƒ Of the eight countries that lacked sufficient data, three (Botswana, Chad, and Mali) were in Sub-Saharan Africa, and two (Haiti and Maldives) were small islands. Of the four Sub-Saharan countries reviewed for this evaluation, only Uganda had adequate data. 82 ƒƒ Almost all of the 12 countries that had sufficient data (Albania, Bangladesh, Bolivia, Bos- nia and Herzegovina, Colombia, Costa Rica, El Salvador, Indonesia, Panama, Serbia, Uganda, and Uruguay), including the data needed to monitor and analyze poverty and shared prosperity in the SCDs, are middle-income countries. Nearly all the middle-in- come countries in the sample had sufficient and up-to-date poverty and shared prosper- ity data of acceptable quality to the World Bank (exceptions were Azerbaijan and Egypt). Analysis of Poverty and Shared Prosperity in SCDs ƒƒ Half of the 22 countries reviewed—most of them middle-income countries—conducted solid analysis of inequality and shared prosperity in their SCDs, though in some, such as Azerbaijan and Egypt, the analysis was based on dated data (five to six years old). ƒƒ For another eight, mostly low-income countries (Bangladesh, Botswana, Chad, Côte d’Ivoire, Haiti, Mali, Myanmar, and Uganda), the twin goals largely or entirely overlapped during the review period because of these countries’ very high poverty levels (between 30 to 50 percent of their respective populations). In these countries, the focus of the analysis was mainly on the inclusiveness of economic growth rather than inequality. ƒƒ In most countries, analysis of shared prosperity was combined with analysis of in- equality; nonincome correlates of shared prosperity; and, in middle-income countries, the challenges facing the middle class. In Europe and Central Asia, the focus was on economic growth and how it affects different segments of the population and ethnic groups (such as the Roma), rather than on inequality or the situation at the bottom of the distribution per se. Priorities for Achieving the Twin Goals in SCDs ƒƒ Nearly all the SCDs reviewed discussed priority areas that require the attention of policy makers in achieving the twin goals. There was considerable variation in terms of their depth, the inclusion of quantitative targets, and realism, however. Only four (Azerbaijan, Bangladesh, Egypt, and Indonesia) included quantitative targets. Other Noteworthy Findings ƒƒ Most of the SCDs cited improving the income or consumption growth of the bottom 40 percent of the distribution. ƒƒ Nearly all the SCDs provided the average annual income or consumption growth of the bottom 40 percent. Independent Evaluation Group | World Bank Group 83 ƒƒ Only 11 SCDs (Albania, Bolivia, Colombia, Costa Rica, El Salvador, Honduras, Indonesia, Panama, Serbia, Uganda, and Uruguay) provided the estimated growth in income or consumption expenditures for the bottom 40 percent between the Great Recession of 2008–09 and 2012–15. ƒƒ Many of the SCDs provided the income growth of other groups, such as the growth of the top 60 percent or top 10 percent or the mean income growth, for comparison, but the comparison indicators selected differed across countries. ƒƒ The SCDs used a range of inequality measures. The Gini index was featured most fre- quently, but the Theil index and the 90/10 ratio were also used. ƒƒ Most of the SCDs implicitly incorporated non-income indicators into the understanding of shared prosperity in most of the SCDs, but they rarely included data on such indica- tors among the bottom 40 percent. 1 These 74 countries have no data points, only one data point each, or two data points that are more than five years apart. 2 The countries are Albania, Azerbaijan, Bangladesh, Bolivia, Bosnia and Herzegovina, Botswana, Chad, Colombia, Costa Rica, Côte d’Ivoire, thEgypt, El Salvador, Haiti, Honduras, Indonesia, Panama, Maldives, Mali, Myanmar, Serbia, Uganda, and Uruguay. For 11 of them (Albania, Azerbaijan, Bangladesh, Bolivia, Côte d’Ivoire, Egypt, Indonesia, Panama, Myanmar, Serbia, and Uganda), the country case study by the Independent Evaluation Group included field visits and interviews with the local counterparts and stakeholders. Reference World Bank Group. 2013. World Bank Group Strategy. Washington, DC: World Bank Group. 84 World Bank Group Country Engagement | Appendix D Appendix E. Results Frameworks in CPFs Background—Results Frameworks in Country Assistance Strategies Early country strategies focused mostly on World Bank products and activities, but as they evolved, they came to reflect more strategic engagement based on development results. As strategies aimed to address an expanding range of issues, however, they became more complex, less strategic, and longer. As experience grew, the focus of the strategies shifted from inputs—a series of products and activities—to development results. With this evolution, the strategies started to be designed to attain certain development outcomes. Results-based country assistance strategies (CASs) were piloted between 2003 and 2005, and were mainstreamed after that. The results-based CAS was meant to help make country programs more strategic as well as more selective. The central innovation was the results framework, which specified links between the World Bank’s interventions and long-term development goals. Measurable indicators of progress would be tracked through implementation of the CAS, encouraging active management of CAS programs and allowing both self- and independent evaluation. The new approach was expected to create a better strategy, with greater chances of success and development effectiveness. A fiscal 2014 learning product reviewed the results frameworks in CASs (World Bank 2014b). The key conclusions were the following: ƒƒ The results-based CAS approach brought several benefits, including the focus on results, better alignment between World Bank Group country engagement and national priorities, and flexibility in instruments to accommodate programming for the increasingly diverse set of World Bank clients. ƒƒ However, common deficiencies included a focus on outputs instead of outcomes, a weak link between designed interventions and outcomes, and the absence of monitoring indicators to track outcomes. ƒƒ Such aspects are important because several recent evaluations and reports had demonstrated that a weak CAS results framework is a key determinant of unsatisfactory outcome performance at the country-program level. The paper noted that the results framework is an integral part of all the components of the Country Partnership Framework (CPF) process, and saw the operational results chain as the most critical and challenging task for the development of the results framework: ƒƒ The results chain begins with inputs, moves through activities and outputs, and culmi- nates in outcomes, impacts, and feedback.1 ƒƒ A proper results chain needs to make explicit the underlying assumptions about the actions by the government, the private sector, and other development partners, including the possible impact of external shocks. 85 ƒƒ To the extent that several instruments are mobilized, it is important for the results chain to first recognize the synergy between instruments. To assess the success of a World Bank Group program, monitorable indicators are needed. This last element of the results framework should be the results matrix, which is a tool for assessing the achievements of the CAS objectives. It summarizes the key country goals, the CAS objectives, and the supporting programs of the World Bank Group, government, and development partners, but most importantly the outcome indicators. The paper notes that, in general, good outcome indicators should be close proxies of the CAS objec- tives, of the same scope as the CAS objectives (may be achieved by a set of indicators rather than a single one), measurable and with a baseline and target, and supported by a data collection system. The paper notes that intermediate indicators are not sufficient evidence for the achievement of CAS objectives. Results Frameworks in the CPF Guidelines The current CPF guidelines discuss the CPF results frameworks (World Bank Group 2014): ƒƒ The CPF program is selective, with a limited number of CPF objectives that concentrate the World Bank Group’s resources in critical areas where they will have the most devel- opment impact. ƒƒ In many cases the World Bank Group is unable to define its program beyond the initial years of the CPF. ƒƒ The World Bank Group program is presented at the strategic level with emphasis on the CPF objectives rather than activities. ƒƒ There must be a clear articulation of a results chain for each CPF objective, including the intervention logic that explains how ongoing and planned activities link to each CPF objective and how the CPF objective relates to a country development goal. ƒƒ It is particularly important to be clear about assumptions regarding contributions from other development partners or the country in regard to achievement of the CPF objec- tive. ƒƒ For each results chain, there should be a brief description of ongoing and planned activi- ties that contribute to the country achieving the CPF objective. ƒƒ For purposes of future evaluation, the CPF text may identify certain CPF objectives as of higher priority than others. 86 World Bank Group Country Engagement | Appendix E The guidelines then discuss the results matrix that is required as a CPF annex: ƒƒ It should provide a concise summary of the results chain articulated in the text, showing the linkages between the country development goals, the development challenges, and the CPF objectives. ƒƒ The matrix presents only the CPF objectives that are expected to be met by the end of the CPF period (the text also should discuss longer-term results). ƒƒ The matrix should include indicators that provide evidence that the objectives have been achieved—they should be both achievable and measurable within the CPF period. They can be either quantitative or qualitative, with the former having baselines and targets. ƒƒ It is best practice for the matrix to include information on data sources. ƒƒ If necessary, the main text is used to explain how selected indicators demonstrate that a CPF objective has been achieved. ƒƒ Teams may include supplementary progress indicators in the matrix if it helps in the as- sessment of progress—and if these are relevant in helping to assess World Bank Group performance. Analysis of the CPF Results Matrixes IEG undertook analysis of the results frameworks in the CPFs for the 10 countries selected for field missions for the evaluation.2 The CPF objectives and corresponding objective indicators were ana- lyzed and coded into one of three categories: process, output, or outcome. Objectives were coded and validated by at least two people. If there was disagreement on a classification, the team dis- cussed it and came to a consensus. IEG consulted relevant literature and used the following practical definitions (World Bank 2014a, 25–28; Morra-Imas and Rist 2009): ƒƒ Outputs: Goods produced, activities, products or services delivered, or actions taken. ƒƒ Outcomes: Behavioral changes, such as increased consumption of goods and services. (For this purpose, outcomes included intermediate outcomes.) ƒƒ Process: Indicators that describe activities that contribute to the achievement of out- comes. The numerical analysis drew from the results matrixes, but the analysis also consulted the discussion in the text of the reports of the results frameworks. In addition to coding the objectives and the objective indicators, the analysis looked for explicit prioritization or ranking of CPF objectives, reliance on actions or activities completed by external actors, and clear inclusion of IFC activities in either objective or Independent Evaluation Group | World Bank Group 87 supplementary process indicators. The team also captured information on whether indicators included a timeline and a baseline/target. The numerical information is listed in table E.1. The main findings include: ƒƒ The average number of CPF objectives is slightly more than 10, with a range from 5 (Bo- livia) to 18 (Indonesia). Of the 106 objectives analyzed (across all 10 CPFs), 48 percent are classified as outputs, followed by outcomes (39 percent) and process (13 percent). ƒƒ Every objective was aspirational in formulation (to enhance, improve, strengthen, and the like), and not a single one included a target, whether numerical or qualitative. ƒƒ On average, there are two objective indicators for each CPF objective (an average of 22 per CPF). In contrast to the CPF objectives, most of the objective indicators across the sample (56 percent) are outcomes, followed by outputs (44 percent). Only 1 of the 218 objectives indicators is a process indicator. ƒƒ There is an average of 32 supplementary process indicators in each CPF, with some variance. The number of supplementary indicators ranges from 12 (Bolivia) to 53 (Arab Republic of Egypt). ƒƒ There is no language in the analyzed results frameworks that indicates that any objec- tives are ranked or prioritized over others. There is language on alignment of CPF objec- tives with SCD priorities, but within the objectives there is no differentiation. ƒƒ None of the results frameworks references reliance on actions or activities of others, though quite a few described partnerships, synergies, and collaboration with govern- ment agencies and other development agencies. ƒƒ Almost all of the objectives and indicators are time-bound, with a time span ranging from two years to six years. There is variation within CPFs—not all the indicators in the same CPF are measured at the same interval. ƒƒ Almost all the objectives indicators included a baseline and target. The exception is Indonesia, where 8 of the 45 objective indicators (17 percent) did not have a baseline or a target (they are designated as “to be determined”). ƒƒ Explicit reflection of the contribution of IFC interventions to achievement of the indicators was included for some indicators, but this was not consistently adopted for objectives where IFC interventions were referenced in the ongoing or indicative program. 88 World Bank Group Country Engagement | Appendix E Conclusions from This Analysis ƒƒ The results frameworks can best be described as constituting truncated results chains, with several items missing. ƒƒ However, the core intervention logic is often well presented. ƒƒ The results frameworks seem to assume (often probably quite reasonably) that the objectives relate to country development goals, but this is sometimes not discussed very clearly. As one illustration, the Uganda CPF explains (in paragraph 70) that the World Bank Group will continue supporting key government (Second National Development Plan) priorities. This seems a rather short way of explaining that the program objectives relate to country development goals. ƒƒ The objectives are formulated in aspirational (or directional) terms and without targets. ƒƒ The objectives indicators will therefore be vital to determining whether objectives have been met. These are often quite narrow, while the objectives are typically general and broad. (An illustration is given in the next section.) It must therefore be expected that, for example, for the ex post self-evaluation or IEG validation, it will sometimes be found that the objective indicator(s) may have been met, but not the objective itself. ƒƒ There are no examples from this sample of any objectives being ranked above others. ƒƒ There are also no examples of the results frameworks referencing reliance on the sup- port or actions by others. Correlation Between Objective and Objective Indicator—An Illustration ƒƒ For the Uganda CPF, objective 1 is “Enhanced economic governance and fiscal man- agement.” ƒƒ The text of the CPF sets out the World Bank’s broad and long-term ambitions and areas of focus, presumably going beyond the CPF period, but this is not stated explicitly, and there is no clear sense from the text about where the ambitions end for the CPF period. (Thus, the ambitions for objective 1, as set out in the text, include economic governance issues, such as addressing tax exemptions, evidence-based resource allocation, effec- tiveness of internal audits, improving performance of procurement systems, implementa- tion of fiscal decentralization, preparing to leverage the benefits of oil, and various steps to strengthen country systems for transparency, participation, and accountability.) Independent Evaluation Group | World Bank Group 89 Table E.1. Results Frameworks in CPFs 90 Objectives by category Indicators by category Any objective or indicator Any de- Objectives Indicators Count of sup- given pri- pendence in CPF Process Output Outcome in CPF Process Output Outcome plementary pro- ority or impor- on actions Country (no.) (%) (%) (%) (no.) (%) (%) (%) cess indicators tance? of others? Albania 12 33 50 17 16 — 44 56 36 No No Azerbaijan 8 50 50 25 — 32 68 39 No No Bangladesh 12 17 25 58 26 4 50 46 26 No No Bolivia 5 60 40 13 — 46 54 12 No No Côte d’Ivoire 10 10 40 50 21 — 33 67 32 No No Egypt, Arab Rep. 16 6 50 44 33 — 45 55 53 No No Indonesia 18 11 56 38 33 — 37 63 45 No No Myanmar 12 17 42 42 22 — 50 50 24 No No World Bank Group Country Engagement | Appendix E Panama 7 14 71 14 12 — 67 33 25 No No Uganda 6 17 50 33 12 — 58 42 31 No No Total 106 — — — 218 — — 323 — — Average — 13 48 39 21.8 0 44 56 32.3 — — Note: — = not available; CPF = Country Partnership Framework. ƒƒ The two objective indicators in the results matrix for objective 1 seem limited relative to the objective, as well as to the ambitions described in the text. One indicator relates to decentralized spending in health and education, and one to non-oil domestic revenue in percent of gross domestic product. These indicators scarcely cover all the areas of ambition and work, and it is therefore easy to imagine circumstances under which the indicators may be met, but not the objective itself. 1 This explanation of the results chain coheres with the literature. In The Road to Results, for example, the results chain is described as a “causal sequence for a development intervention that stipulates the necessary sequence to achieve desired objectives, beginning with inputs, moving through activities and outputs, and culminating in outcomes, impacts and feedback” (Morra-Imas and Rist 2009, 166). That text goes on to say that a results chain does not have to have the same level of detail as a flow chart (described in the same place), but it should be possible to depict it with key assumptions in a chart. 2 Albania, Azerbaijan, Bangladesh, Bolivia, Côte d’Ivoire, the Arab Republic of Egypt, Indonesia, Myanmar, Panama, and Uganda. References Morra-Imas, Linda G., and Ray C. Rist. 2009. The Road to Results: Designing and Conducting Effective Development Evaluations. Washington, DC: World Bank. World Bank. 2014a. Guidelines for Reviewing World Bank Implementation Completion and Results Re- ports: A Manual for Evaluators. Washington, DC: World Bank. ———. 2014b. Results Frameworks in Country Strategies—Lessons from Evaluations. Washington, DC: World Bank Group. World Bank Group. 2014. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. Independent Evaluation Group | World Bank Group 91 Appendix F. Country Engagement Notes Introduction and Scope In April 2014, the World Bank Group introduced a new approach to country engagement to better serve its clients and support the twin goals of eliminating extreme poverty and increasing shared prosperity in a sustainable manner. The central mechanism of the approach, the CPF, is designed to help the World Bank Group to selectively address areas with the most potential to affect countries’ efforts to achieve the twin goals. The approach relies on an SCD, which will present a timely and evidence-based assessment of the constraints a country must confront, with a select set of priority focus areas to address these constraints and accelerate progress. The World Bank Group and the country will draw on the SCD to work together to develop the CPF objectives, deriving them from the country development goals that reflect the World Bank Group’s comparative advantage, as well as alignment with the twin goals. The World Bank Group guidance for CPF products (World Bank Group 2014b) states that on the limited occasions when a government and the World Bank Group are unable to define detailed ob- jectives or develop a program for the medium term, a Country Engagement Note (CEN) will be used instead of the SCD/CPF process. This appendix describes the conditions under which the guidance provides for use of a CEN and ex- amines experience with the six CENs prepared through April 2016.7 It examines the rationale for use of CENs, as well as their adherence to the guidance regarding timing and content, the evidence base and knowledge gaps, the benchmarks and indicators to monitor progress, and the assessment of risks. CEN Background and Guidance The Board paper creating the new country engagement approach acknowledges a lack of satisfac- tion with the option to prepare an Interim Strategy Note (ISN) rather than a CAS (World Bank Group 2014a). While initially the ISNs were intended to be short notes to guide activities in conflict situations until a full CAS could be developed, country teams were allowed significant discretion in when to use them. Consequently, ISNs had been used not only in conflict-affected countries, but also in a wide variety of other situations, including when the World Bank Group was reengaging in a country after a long absence, when there was a high level of political uncertainty, and when there was a need to align the CAS cycle with an upcoming country development plan. By 2014, the World Bank Group had produced 27 ISNs, and as many as 3 in a row in some countries. The Board paper says: “Although ISNs were intended to be short notes that are not required to have a results matrix, some recent ISNs have been long documents, setting out long-term objectives with detailed results matrixes. For all intents and purposes, many ISNs have become two-year CASs” (World Bank 2014a, 4). ISNs were not formally evaluated by the country team, nor were they reviewed by IEG in the same way as CASs. It had become a concern that the World Bank had come to rely on ISNs, even in some countries with a large portfolio of long-term investments, or a large pipeline of future investments. 92 The guidance for CPF products allows the use of CENs only under four specific, limited conditions— for countries in transition from conflict or political crisis, when a country is going through an unusually uncertain period (for example, preelection, social crisis, natural disaster) that prevents the formation of medium-term objectives, when the World Bank Group lacks sufficient knowledge when reengaging in a country after a prolonged hiatus, or in transitory situations when the World Bank Group does not have the knowledge required to develop a full strategy. Under all four possible circumstances “the decision to use a CEN . . . is based on the expected time horizon of the development objectives the [World Bank Group] is supporting” (World Bank 2014b, 4). The persistence of fragility is not defined as a crisis. To the contrary, the guidance stresses that “uncertainty or fragility alone does not justify the use of a CEN; it is expected that a full CPF will be prepared in most fragile states and volatile countries” (4). Thus, under the new country engagement model, Afghanistan and Myanmar prepared SCDs and CPFs although under the previous system as FCS countries they might have prepared ISNs. Even when a high level of uncertainty persists, it advises, a CPF should be prepared if the World Bank Group “can undertake activities with long- or medium-term impact” (4). Furthermore, “the CEN does not substitute for a full CPF, but bridges a gap until longer-term objectives can be developed and a CPF prepared” (World Bank 2014b, 4). While there clearly can be considerable overlaps between the use of ISNs and CENs, the two main circumstances where CENs may be used are when either an immediate crisis or knowledge gaps prevent the preparation of a medium-term strategy. Given the more dynamic and uncertain country circumstances under which CENs are used, they are expected to have a much shorter cycle—12 to 24 months—than CPFs, which may run for up to 6 years. The 2014 guidance further clarifies that in exceptional circumstances, a CEN may be followed by another CEN if a CPF continues to be infeasi- ble. But continual use of CENs in fragile or conflict-affected countries is no longer consistent with the guidance, and “CENs are not used to align the timing of [the] program with the government’s plan” (World Bank Group 2014b, 4). This implies that a CEN may be permissible if there is an immediate crisis that prevents formulation of medium-term objectives or if there are significant knowledge gaps. If neither condition is met, the country team should adopt the SCD/CPF path. Where knowledge gaps exist “one of the main purposes of the CEN is to gather enough information through gender assess- ments, poverty assessments, fragility assessments, and other key economic and sector work as out- puts of the CEN for the [World Bank Group] to develop a full program” (World Bank Group 2014b, 17). Expected Content of CENs Because CENs will often be prepared in situations where the World Bank Group has knowledge gaps, it is not required that a CEN be preceded by an SCD, and it is not expected to contain as much documentation and analysis as a CPF. However, the guidance note states, “it must contain sufficient analysis to support the preparation and monitoring of the proposed engagement. . . . [And] when possible, it is expected to have solid underpinnings, including assessments on poverty, gender, fragil- ity, and other important aspects of development” (World Bank Group 2014b, 17). Independent Evaluation Group | World Bank Group 93 CENs are expected to be no more than 15 pages, and are to be structured around the objectives that the World Bank Group expects to help the country achieve in the short term. A CEN will not have a detailed results matrix, but the document should describe benchmarks and performance monitoring indicators to assess progress, which could be in the form of a matrix listing outputs or milestones of activities. CENs should discuss risks to the World Bank Group program, based on the SORT. The risk section should focus on the risks to the CEN program achieving its goals, including the risk of unintended negative consequences. CENs are not required to have a CLR and are not subject to IEG validation. However, at the end of the CEN period, teams should conduct a thorough analysis of World Bank Group performance and les- sons learned, which is expected to be incorporated into the subsequent CPF or CEN. This guidance places the burden entirely on self-assessment, which led to the proliferation of ISNs in the first place. Rationales for CENs Two of the six reviewed CENs were prepared for countries classified as fragile and conflict-affected situations (FCS)—the Central African Republic and Guinea-Bissau. The Central African Republic has been affected by a recent internal conflict and is also going through a political transition. Although the country reached the Heavily Indebted Poor Countries Initiative completion point for debt relief in 2009, a political security crisis triggered by rebellion in the north against an increasingly exclusive regime resulted in large-scale violence. This deteriorated into violence based on ethno-religious recrimination, displacement of over 25 percent of the population, and a significant economic down- turn. The crisis created by the failed rebellion led to a large-scale international response, including peacekeeping interventions and emergency relief by United Nations agencies and nongovernmental organizations. The Central African Republic is also one of three countries going through a political transition. The transitional government began a national dialogue culminating in the National Forum in Bangui in May 2015, laying the groundwork for national elections scheduled for October 2015. The World Bank supported the crisis response in January 2014 with a $100 million package of emergency operations through regular and restructured IDA lending. The CEN continues the crisis response and maps out the next steps for recovery and development over 18 months (July 2015–end 2016). “Given that much of the country is insecure, access for World Bank staff is still restricted, and the country is undergoing a fragile political transition, a CEN is considered the most appropriate vehi- cle to outline the World Bank’s strategy” (World Bank 2015a, 1). Guinea-Bissau presents a convincing rationale for use of a CEN as an interim measure. Despite a series of successful elections from 1994 to 2008, the country went through acute instability, and several governments were overthrown by coups. The last coup in 2012 paralyzed the country and led to a deep fiscal and poverty crisis. The use of the country as a transit point for international drug trafficking wors- ened its fragility. An elected government is now in place and is showing leadership to sustain peace and restart development. It is preparing a comprehensive development program, to be anchored by a list of priority investments. The CEN for fiscal 2015–16 was prepared in coordination with other donors 94 World Bank Group Country Engagement | Appendix F to support the government of Guinea-Bissau. The country is also among the first two proposed as a candidate for exceptional allocation from IDA17 for “turnaround” situations. The Guinea-Bissau CEN committed to delivering an SCD in FY16. This document was finalized in June 2016. For two other countries, Ecuador and Guyana, the CENs identify political transition as the primary reason for using this instrument. Both provide further justifications for the use of a CEN. The Ecuador CEN was preceded by an ISN for fiscal 2014–15, which allowed the World Bank Group to reengage with Ecuador in fiscal 2014 and demonstrate its technical rigor, operational effectiveness, and finan- cial benefits. Under the ISN, the World Bank Group produced a number of analytical products on informality and the labor market, productivity and competitiveness, conditional cash transfers, and a national strategy for water and sanitation (table F.1). Table F.1. Rationale for a CEN Country Justification Provided Fiji Reengagement; previous CAS was regional. CEN committed CPF in fiscal 2017, but regional CPF now under preparation. Guinea-Bissau Fragility and political instability. CEN for short-term support to consolidate transition and restore basic services. Central African Republic Conflict and political transition. Turkmenistan Reengagement. Follows fiscal 2014–15 ISN, which was the first formal engagement since 2001. Delivered reimbursable advisory services (RAS) program on macroeconomic statistics and financial sector development. Guyana Political transition. New government elected May 2015 and development strategy being prepared. CAS fiscal 2009–12 priorities continue. Ecuador Follows fiscal 2014-15 ISN. Forthcoming presidential elections in 2017 and macroeconomic risks cited to justify CEN. Although the World Bank Group has had limited engagement in Ecuador, in addition to the World Bank’s active portfolio of six projects ($718 million), IFC also has a committed/outstanding portfolio of around $250 million. The choice of a CEN is attributed to the upcoming presidential elections, scheduled for early 2017, and the adverse macroeconomic context, which the CEN argues poses a risk to the pace of implementation of a more comprehensive program. In this case, in IEG’s view the preparation of an SCD, even with some knowledge gaps, could have positioned the World Bank Group to offer more meaningful advice to the new government. The choice of a CEN for engagement in Guyana in fiscal 2016–18 is also attributed to uncertainty arising from political transition. In this case, however, the new government had already been elected in May 2015. The justification the CEN offers is that the government is in the process of preparing their national development strategy. The CEN was preceded by the fiscal 2009–12 CAS, following which the World Bank Group’s engagement was focused on implementation of activities committed under that CAS and on addressing requests for policy advice and technical support through adminis- trative funds or grant financing. Independent Evaluation Group | World Bank Group 95 New lending resumed in fiscal 2014, leading to a doubling of the IDA portfolio in fiscal 2014–15. The resumption of lending and the presence of a new government could also have been seen as an opportunity to prepare an SCD to inform the national development strategy. The CEN claims there was an absence of poverty data. However, the data cited on education, health, and water supply and the references to analytical work conducted by other agencies suggest there was sufficient knowl- edge to undertake an SCD, which could have identified further knowledge gaps to be filled in the next cycle. While a CPF could perhaps have awaited the government’s development strategy, delaying the preparation of an SCD is a missed opportunity based on underestimating its potential value. The rationale for both Turkmenistan and Fiji is reengagement. In Turkmenistan, the World Bank Group has had limited involvement, but it reengaged with the country in 2012 with a knowledge-based pro- gram financed largely through reimbursable advisory services (RAS). Prior to the CEN, engagement was based on the ISN for fiscal 2014–15, which covered RAS activities on macroeconomic statistics, financial sector development and support for privatization, and strategic studies on reform priorities. The previous ISN also included an IFC program supporting the government’s privatization program. While the country has maintained an impressive growth record largely based on extractive industries, the CEN argues that the paucity of poverty data and uncertainty about a medium-term engagement make a CEN a more appropriate vehicle for World Bank Group engagement. The CEN envisages continuing with the World Bank’s knowledge-based program with four main areas of engagement: improving data for policy making and governance, financial sector modernization, enhancing com- petitiveness, and managing natural resources efficiently. IFC will participate in the last two of these through advisory services, and possibly as an investor. MIGA has no current engagement but will explore opportunities for guarantees to potential investors. The CEN for Fiji was the first country strategy for the country and marks the resumption of IBRD lend- ing after a 20-year hiatus. It followed the democratic elections in September 2014, which provided civilian legitimacy to Commodore Bainimarama, who initially came to power through a coup in 2006. The World Bank Group had extended no new lending to Fiji since 2005. Following the 2014 elections, Fiji was readmitted to the Commonwealth, and the Pacific Island Forum Secretariat and Australia and New Zealand have lifted all sanctions. The Asian Development World Bank endorsed a new CPF in December 2014, and immediately approved a new transport sector project. The CEN aimed to rebuild the World Bank Group’s partnership with the government of Fiji, and for the World Bank Group to deepen its knowledge base. The CEN was accompanied by a $50 million transport sector loan, with a provision for another $75 million for another operation whose content has yet to be determined. However, it should be noted that the country department had previous- ly operated with a regional strategy for Pacific Island countries, and going forward it is preparing a regional SCD, to be followed by a regional CPF. 96 World Bank Group Country Engagement | Appendix F The Relevance of Time In this section, the CENs are assessed along three time dimensions: immediacy of the crisis or transi- tion, the time frame of the CEN, and the duration of the program proposed by the CEN (table F.2). On the first dimension, Ecuador seems questionable. On the second and third dimensions, the Fiji CEN seems inconsistent with the two-year limit for CENs. Most country teams justify CENs on the basis of political transitions. With the exception of Turkmen- istan, all the CENs provide a justification based on timing, which they argue prevents the formulation of a medium-term program. On this criterion, the Fiji, Guinea-Bissau, and Guyana CENs follow a recent election, and the Central African Republic CEN immediately precedes an election. Fiji and Guinea-Bissau are also countries where the World Bank Group is reengaging after a hiatus of several years, and Guinea-Bissau and Central African Republic have also been adversely affected by political instability, conflict, and fragility. Fiji has been relatively stable and performed well economically com- pared with its peers in the Pacific Islands, but was ostracized by development partners after the 2006 coup. In Ecuador, where elections are not scheduled until 2017, there is no immediate crisis. The argument that this is a preelection period, which provides a level of uncertainty, is unconvincing, be- cause it is not very different from the nature of World Bank Group relations over the past few years or from conditions in other World Bank Group clients from time to time. The CEN is clearly buying time in the hope that the 2017 election will bring about a change toward a more responsive government. Turkmenistan is in a different league. Timing is not a factor. Its knowledge-based program is driven largely by RAS, and the World Bank is trying to reestablish itself as a reliable and trustworthy partner. Most of the CENs have a two-year time frame. World Bank Group guidance says that CENs are ex- pected to have a 12–24-month cycle, with a maximum limit of two years. Except for Fiji, all the CENs are at the upper limit. The Fiji CEN was also put forward as a two-year CEN to build the World Bank Group’s “partnership with Fiji, deepen its knowledge base, and support critical investments” (World Bank 2015b, 1), but the program is actually longer. The timing of deliverables starts in March 2015 (implying it started earlier) and continues through June 2017, making it at least a 28-month CEN. The program duration indicates a short time horizon for the lending and advisory services and analytics programs in most CENs. Without prejudging whether it was feasible to propose a medi- um-term program in some of these countries, the actual programs put forth in five of the six CENs are consistent with the guidelines of focusing on short-term objectives—strengthening institutions, restoring basic services, providing short-term livelihood support, and undertaking analytical work to fill knowledge gaps in areas of mutual agreement. The program in Fiji, however, appears to be more like a medium-term program. The Transport Infrastructure Investment Project, financed by the IBRD, which accompanied the CEN, may not be completed within the time span of the CEN. And the provisioning of $75 million for another operation—either an investment project in agriculture, energy, tourism, or fisheries, or a development policy operation in public sector reform—which had not even been determined by the Board date, is even more likely to exceed the short-term objective of a CEN. Independent Evaluation Group | World Bank Group 97 Table F.2. Timing Aspects in CENs Country and Board Date Timing of CEN Time Frame Program Duration Fiji Reengagement following elections in fiscal 2015–17 Program exceeds 2-year time 3/11/2015 September 2014. frame. $50 million transport investment project approved with CEN; $75 million for an investment project or a development policy operation. Guinea-Bissau Reengagement. 2014 elections fiscal 2015–16 Two years; $30 million IDA 3/24/2015 restored legitimacy after 2012 coup plus enhanced funds as and prolonged history of instability and “turnaround regime.” Reached fragility. HIPC completion point in 2010. Central African Political transition. Election scheduled fiscal 2016–17 Two years; $42 million IDA— Republic in October 2015 after ethno-religious core public services, support 7/30/2015 conflict since 2013, under “Head of to livelihoods, health systems State of the Transition” with help from project; $12 million Health international peacekeepers. Results Trust Fund. Candidate for exceptional allocation as “turnaround regime.” Turkmenistan Knowledge-based country program fiscal 2016–17 Two-year RAS and advisory: 11/3/2015 follows ISN fiscal 2014–15. Limited improving data, financial country engagement. services modernization, enhancing competitiveness, managing resources efficiently. Guyana Political transition, following elections fiscal 2016–18 Debt relief (HIPC and 3/15/2016 held in May 2015 Multilateral Debt Relief Initiative) of $97 million under fiscal 2009–12 CAS. New lending resumed fiscal 2014; portfolio doubled in fiscal 2014–15. Program has $27.3 million for disaster risk management, education, and PSD + IFC support. Ecuador Political transition. Elections planned fiscal 2016–17 Sustaining gains in basic 3/15/2016 in 2017. Limited World Bank Group service delivery and safety engagement. nets, economic diversification. Climate and environmental mitigation is a cross-cutting theme. Note: CEN = country engagement note; HIPC = Heavily Indebted Poor Country Initiative. Consistency with Guidance on Content In this section, the CENs are assessed against guidance provided on the length of the document, the evidence base and knowledge gaps, the indicators proposed to monitor progress, and risk assess- ment. None of the six CENs reviewed met the page limit prescribed by World Bank Group guidance, which clearly provides for a maximum of 15 pages. The main body of the text alone ranged from 17 to 23 pages. With the annexes, the CENs ranged from 30 to 85 pages. Either the guidance needs to be revised, or more stringent systems introduced to prompt adherence to the guidance. 98 World Bank Group Country Engagement | Appendix F Most of the CEN countries had a paucity of current data on poverty. Despite the country’s fragility, analytical work in Guinea-Bissau was well under way, and this is the only CEN country to have un- dertaken an SCD, discussed in the Board in June 2016. The World Bank Group prepared a Country Economic Memorandum (CEM), a fragility assessment, and analytical work in the health sector. Further work on poverty is planned in the form of poverty mapping and a poverty and social impact analysis of the cashew sector, the country’s main cash crop. Analytical work in the Central African Republic has focused largely on fragility and the conflict con- text. The CEN draws on several relevant pieces of analytical work, which are also included as four separate annexes on conflict and fragility analysis, armed groups, displacement, and pastoralism and conflict. Although the CEN briefly describes the economic context, it appropriately focuses on stabili- zation and services to rebuild livelihoods. Poverty data in Fiji and Guyana are based on household surveys done in 2008/2009 and 2006, respectively. The CEN for Fiji commits to update the poverty assessment based on the 2013/2014 Household Income and Expenditure Survey. It also ambitiously plans a CEM and an SCD by June 2016, and a country gender assessment by December 2016. However, there is no evidence that these have been prepared, or are even well under way. The Country Management Unit (CMU) is now reported to be preparing a regional SCD and CPF. Since poverty data are not current, accurate measures of poverty are not available for Guyana. The CEN reports significant improvements in nutri- tion and child health, but maternal and reproductive health indicators remain poor. Improvements in education, access to water, and gender equality are reported, but significant geographical and ethnic disparities persist. During the previous CAS period, $97 million in debt relief was approved (see table F.2), but reform momentum subsequently slowed down, and the World Bank Group did not prepare a subsequent country strategy, although it resumed lending in fiscal 2014 and 2015. The CEN program envisages $27.3 million for disaster risk management, education, and PSD, in addition to IFC and MIGA efforts to support the private sector. The Ecuador CEN builds on the analytical work done under the previous ISN on informality and the labor market, productivity and competitiveness, the graduation process from conditional cash trans- fer programs, and the water and sanitation sector. The World Bank Group also initiated RAS activities on competitiveness in global value chains. Macroeconomic data support the analysis of drivers of progress and the challenges facing the country. The CEN also presents evidence of gains in growth, poverty reduction, and shared prosperity due to rising employment and income opportunities, as well as public transfers, although sustainability remains a concern. This suggests that paucity of analytical evidence was not a constraint for preparation of an SCD. Table F.3. Consistency with Guidance on CEN Content No. of Benchmarks and Country pagesa Evidence base and knowledge gaps indicators Risk assessment Fiji 22 Poverty data based on 2008/09 HIES. Rudimentary SORT used, CEN commits to new poverty assessment matrix of activities but technical based on 2013/14 HIES; CEM and SCD and outputs listed and institutional (in June 2016), CGA (December 2016). as milestones. challenges in CEN time frame underestimated. Independent Evaluation Group | World Bank Group 99 No. of Benchmarks and Country pagesa Evidence base and knowledge gaps indicators Risk assessment Guinea- 17 CEM, health economic and sector work, Monitoring SORT: High Bissau and fragility assessment done. Poverty framework fragility, political/ mapping, PSIA of cashew sector, and in annex— governance, SCD planned. SCD done, June 2016 milestones, some macroeconomic, benchmarks. institutional capacity risks. Central 19. Analytic and advisory assistance on Monitoring SORT: African conflict and fragility analysis; armed framework in Political and Republic groups; displacement; pastoralism and annex— process governance, and conflict. SCD and sector notes planned. and outputs; macroeconomic revenue outcome risks high, but risk indicator has of inaction also benchmark. high. Turkmen- 21 No mention of CEM, poverty assessment, No monitoring SORT: political/ istan CGA, or SCD. Growth record strong but matrix in CEN. governance, based on extractives. Macroeconomic institutional data in CEN but household survey capacity, and outdated. macroeconomic instability are moderate. Guyana 19 Poverty data from 2006 household No monitoring SORT: budget survey shows geographic and matrix. Macroeconomic, ethnic inequality. Macro data are current Performance institutional and reveal volatility to commodity prices. measured by capacity, and qualitative criteria environmental “derived from sustainability portfolio results risks rated frameworks.” substantial, which appears to be an underestimate. Ecuador 23 Macroeconomic and 2013 poverty and None. SORT: Overall inequality data included in CEN, suggests operational an SCD should have been possible. risks high: macroeconomic, institutional capacity, and natural hazards risks high. Note: HIES = Household Income and Expenditure Survey; CGA = country gender assessment; PSIA = Poverty and Social Impact Analysis. a. Page length compares length of main text to maximum limit of 15. Turkmenistan is the most extreme case, since the government shares very limited data. Recent ana- lytical work done by the World Bank Group generated useful data on macroeconomic trends and the financial sector, and the government is welcoming further analytical work on privatization and PSD. The CEN reports on the country’s social, political, and gender profile. However, there is no mention of a CEM, a poverty assessment, or a gender assessment, nor is there any commitment to prepare an SCD. Most CENs lack basic elements that would allow a thorough analysis of performance and lessons. CENs are not expected to have a detailed results matrix, but the document should describe bench- marks and performance monitoring indicators to assess progress. Among the six CENs, the only two that present a matrix with indicators to monitor program outputs are from the FCS countries—the 100 World Bank Group Country Engagement | Appendix F Central African Republic and Guinea-Bissau—while Fiji has a more simplified matrix of activities, and outputs labeled milestones. The CENs for Turkmenistan and Ecuador do not have any mechanism to monitor progress, and the Guyana CEN states that performance will be measured by qualitative criteria “derived from portfolio results frameworks” (World Bank Group 2015c, 14). Current World Bank Group guidance says that CENs are not expected to prepare a CLR and are not subject to validation by IEG. But the teams are expected to conduct a thorough analysis of World Bank Group performance and lessons learned at the end of the CEN period, which is expected to be incorporated into the following CPF or CEN. The absence of even a rudimentary results framework in three of the CENs renders even this limited objective out of reach. All the CENs use SORT to identify and summarize risks, but only half of them acknowledge high risks. Not surprisingly, CENs for the two FCS countries—the Central African Republic and Guinea-Bissau— assess political and governance risks and macroeconomic risks as high. The Ecuador CEN is also candid about high overall operational risks, and also rates macroeconomic, institutional capacity, and natural hazards risks as high. But the Fiji CEN underestimates operational and institutional capacity risks to the program, while Guyana underestimates institutional and governance risks (table F.3). Given the modest, knowledge-based nature of the CEN in Turkmenistan, it assesses the risks to the program as being moderate. While this may well be true, there is no evidence yet that Turkmenistan will be able to graduate to the SCD/CPF country engagement model in the foreseeable future. Main Messages The guidance for preparation of a CEN allows consideration of the instrument only for countries in crisis, when a country is going through an unusually uncertain period, or when the World Bank Group is reengaging with a country and lacks sufficient knowledge to formulate medium-term objectives. The World Bank Group has used this option for six countries, of which only two were FCS countries. Of those, only Fiji embarked on a program with a time frame that is likely to exceed the two-year time span. Lack of confidence about country ownership for a medium-term program seems to have been the primary motivation in Guyana, Ecuador, and Turkmenistan. In several countries, the World Bank Group also lacked the knowledge base to develop a full SCD that addresses the binding constraints on poverty reduction and shared prosperity. However, with the exception of Guinea-Bissau, the CENs have not made sufficient provisions to ensure that these knowledge gaps will be filled in time for the teams to prepare an SCD and CPF before the end of the CEN cycle. In comparison with the SCD/CPFs prepared under the new country engagement model, the CENs have had a rather cursory treatment of cross-cutting themes. Fragility received adequate attention in the two FCS countries, but fragility risks were underestimated elsewhere. The treatment of gender issues was rather superficial in the few cases where it was discussed. And despite references to climate risks in Fiji and Guyana, these risks were undervalued in the CENs. Independent Evaluation Group | World Bank Group 101 The creation of CENs has provided the World Bank Group with a nimble instrument to engage in client countries where a medium-term strategic program cannot yet be developed. But the CENs run the risk of emulating the previous pattern of repeated ISNs. If this cycle is to be broken, and countries currently operating under CENs are expected to contribute substantially to the World Bank Group’s twin goals, senior management will need to send clear signals that a CEN should not be followed by yet another CEN, unless there is an immediate crisis. One way to change the incentive structure would be to remove the discretion from country teams by mandating an external review of CENs. An alternative would be to institute an external validation by IEG of the performance review and lessons from the CEN prepared by country teams. References World Bank. 2014a. Selectivity in Country Strategies: The Evidence. Washington, DC: World Bank Group. ———. 2014b. World Bank Group Assistance to Low-Income Fragile and Conflict-Affected Countries. Washington, DC: World Bank Group. World Bank Group. 2015a. Central African Republic—Country Engagement Note for the Period FY2016– 2017. Washington, DC: World Bank Group. ———. 2015b. Fiji—Country Engagement Note for the Period FY2015–2017. Washington, DC: World Bank Group. ———. 2015c. Guyana—Country Engagement Note for the Period FY16–18. Washington, DC: World Bank Group. 102 World Bank Group Country Engagement | Appendix F Appendix G. IFC and MIGA Integration in the SCD and CPF Process Introduction The purpose of this summary note is to analyze the extent to which IFC and MIGA programs have been integrated into the SCD and CPF process as part of One World Bank Group, compared with the previous CAS approach. It also describes the early experience of IFC and MIGA staff with their participation in the SCD and CPF processes. Finally, this note examines the integration of private sector binding constraints, challenges and opportunities, and analysis of the sector’s contribution to the achievement of the twin goals in the SCD and the CPF. The note is based on the review of SCDs and CPFs of 22 countries that were approved by the Board of Directors as of April 30, 2016. IEG conducted visits to 10 countries to get firsthand feedback from the respective governments and other stakeholders. To implement the 2013 World Bank Group strategy more effectively, a number of operational and organizational changes were implemented. The changes were meant to better serve World Bank Group clients and support the goals of eliminating extreme poverty and increasing shared prosperity in a sus- tainable manner (the “twin goals”). One important change is the new country engagement approach, aimed at increasing the strategic focus of World Bank Group country programs. The approach builds on the previous CAS process and adds improvements and new features to enable the organization to operate effectively as One World Bank Group to help client countries achieve the twin goals. The new country engagement approach was also designed to overcome the challenges and learn lessons from the CAS experience, especially on the effective integration of IFC and MIGA into the country strategies and country ownership. Although IFC, and occasionally MIGA, participated in several CASs, the extent of their integration into the CAS process varied.1 Reviews of CASs indicate that IFC became an integral part of the CAS in countries where the institution already had a signifi- cant presence (World Bank Group 2009). In countries where it had limited or minimal presence, IFC’s contributions were relegated to a few paragraphs or sentences in the CAS. Several IFC staff inter- viewed for this evaluation also mentioned that under the previous CAS approach, World Bank team leaders would give them short notice—in extreme cases, a day—to draft a paragraph explaining IFC’s contribution to the CAS. IEG’s learning note on World Bank-IFC cooperation in country strategy development found that de- spite the increase in the number of joint CASs, the majority of the strategies failed to include specific proposals to implement planned World Bank Group cooperation (World Bank 2015b). IEG also noted that references to cooperation were often perfunctory, and joint programs were not reflected in the results matrixes. In addition, IFC’s country engagement in the CASs tended to be merely a listing of its planned investment and advisory projects in the country. In the majority of the strategies reviewed, IEG found that the discussions lacked a clearly defined structure and implementation plan. World Bank and IFC cooperation was presented without quantifiable metrics or planned division of labor be- tween the two institutions, and without any specific, identified outcomes. Monitoring and evaluation of the impact of IFC activities was also hindered by the absence of expected outcomes or indicators of IFC projects in most of the CASs. 103 The underlying reasons for the mixed results in achieving a truly joint World Bank Group country strategy under the CAS approach included: the differing importance of the CAS to the World Bank and IFC; different business models; lack of incentives for World Bank and IFC staff to cooperate; and even unfamiliarity among staff of the other institution’s procedures and decision-making processes. Concerns over conflicts of interest and limited resources of IFC strategy officers were also identi- fied as structural reasons for the low level of intra–World Bank Group cooperation under the CAS approach. Nevertheless, IEG’s learning note acknowledged the potential for selective World Bank-IFC cooperation to help improve the effectiveness and efficiency of World Bank Group operations and its development impact in client countries. The new country engagement approach is designed to remedy these shortcomings and strongly em- phasizes the integration of the World Bank, IFC, and MIGA as One World Bank Group at the country strategy level. IFC and MIGA are expected to participate in the preparation and delivery of the SCD, CPF, the Performance and Learning Review (PLR), and until the self-assessment or the Completion and Learning Review (CLR) at the end of the CPF cycle. To strengthen this remit, a World Bank Group directive, procedure, and guidance were issued, setting out the process, governance structure, orga- nizational responsibilities, and templates (World Bank Group 2014b, 2014c, 2015e). Training of World Bank, IFC, and MIGA staff in the CPF Academy and regular Board updates are also important pieces in ensuring the effective implementation of the new approach. These arrangements are reinforced by the Regional Coordination Mechanisms consisting of regional management teams of the World Bank, IFC, and MIGA.2 The guidance makes clear that “All CPF components and the CEN [Country Engagement Note] are joint documents and are approved by the World Bank, IFC, and MIGA” (World Bank Group 2014b, 5). Sign-off by the World Bank, IFC, and MIGA team leaders is required to send the SCD and CPF documents for formal reviews. In contrast, coordination among the three World Bank Group institutions under the CAS approach tended to be ad hoc and relied mostly on individual teams. It is crucial to the new approach that senior management of the World Bank, IFC, and MIGA be involved in the SCD and CPF process. A Review of IFC and MIGA Early Experience in the New Country Engagement Approach: What Has Changed? Substantive Integration of IFC and MIGA in the SCD and CPF Processes and in Strategy Documents IFC integration in the SCD and CPF. IFC’s participation in the SCD and CPF is broader than in the previous CAS approach, with clear internal structure and workflow. IFC was an integral part of the team in all 22 SCDs and CPFs reviewed by IEG, even in countries where it had limited operations (Bolivia, Botswana, Chad, Maldives, and Uruguay). IFC’s participation in both the SCD and the CPF is coordinated by nine senior strategy officers with global and regional strategy functions in the Country Framework Engagement Unit in IFC headquarters under the Office of the Chief Economist (figure G.1). For the first batch of SCDs and CPFs (Albania, Indonesia, Myanmar, and Panama), the unit’s senior 104 strategy officers served as the IFC team leaders. They consulted the relevant IFC regional and indus- try directors and staff, and country field offices, and gathered and summarized their inputs.3 Unlike the CAS approach, there is an IFC internal structure for the new approach. Ownership of the SCD and CPF in IFC rests with the IFC regional directors and country heads.4 IFC’s internal process involves the regional director appointing the IFC team leader in the SCD and the CPF, usually the country managers. IFC team leaders of most CPFs approved after April 2015 (Azerbaijan, Bosnia and Herzegovina, Botswana, Chad, Côte d’Ivoire, the Arab Republic of Egypt, Indonesia, Mali, Serbia, and Uganda) were senior staff from the country offices.5 Typically, IFC designates the same team leader for the SCD and CPF. Figure G.1. IFC Organizational Structure and Work Flow in the SCD and CPF IFC IFC WB CMU IFC Chief Economist IFC RD IFC VP Region/ Industry Strategy Units Initiate IFC Country Kick off: WB SCD or announces SO engages Framework CPF process, alerts/invites CH, alerts RD, RIHs Unit Manager Results and/or Measurement Process IFC IFC TL consults IFC Regional Unit Industry IFC-WB team Strategy units discuss Heads context, approaches & Provides input/ key issues participates Concept Note IFC-WB team drafts CN IFC industry units provide inputs QC/ Comments CN Review RD co-chairs Meeting (if Tier 1) IFC leads Pr. S Draft consultations; joins other IFC Industry staff join Document consultations sector/thematic and/or JIP Provides input/ brainstorming participates WB, IFC & Industry units MIGA draft comment report QC/ Comments RD co-chairs IFC VP Approval ROC Review Tier 1; approves SCD Approves final CPF Source: The World Bank Group Approach to Country Engagement: Working as One World Bank Group. CPF Fall Academy. October 21, 2015. The IFC team leader pulls in industry experts from its global or regional industry teams on the invest- ment side and the cross-cutting advisory services units to shape industry-specific discussions. Both the IFC team leader and the IFC strategy officers participate in internal World Bank Group deliber- ations, including in-country consultations, and lead private sector engagements in tier 1 SCDs and CPFs. The Results Measurement Unit supports in the structuring of the results framework. IFC’s chief economist or the manager of the Country Framework Engagement Unit provides oversight on IFC’s inputs into the SCD and the CPF. IFC’s vice president for Global Client Services approves and clears all strategy documents for IFC and co-chairs decision meetings and Board discussions for tier 1 SCDs and CPFs. The CBCCF team (Country Framework Engagement Unit) coordinates and partici- pates in the working group meetings with the World Bank’s Operations Policy and Country Services and MIGA teams on the overall implementation of the new country engagement approach. IFC’s global presence, with some offices co-located in the World Bank country offices, facilitates participa- Independent Evaluation Group | World Bank Group 105 tion in brainstorming meetings, stakeholder consultations, and working meetings with the World Bank country management teams. Integration of IFC in the SCD and the CPF is substantive compared with the previous CAS approach. But its contribution to the country strategy and program is more evident in the CPF than in the SCD, with the latter considered mainly as a World Bank document. IFC inputs and participation in the SCDs are mostly recognized by the World Bank teams, but the level of participation and integration of its contribution in the diagnostic document varied. In the Bolivia, Colombia, El Salvador, Haiti, and Panama SCDs, short industry notes prepared by the IFC teams were included as annexes, providing background and context to the diagnosis of binding constraints and priority areas for private sector development. In the stakeholder consultations for the Albania and Indonesia SCDs and CPFs, IFC led and moderated consultations with the private sector.6 In the Côte d’Ivoire and Uganda SCDs, IFC provided important inputs, albeit verbally, that were integrated by the World Bank team leaders into the SCD. Figure G.2. IFC Programs in 22 CPFs Will Contribute Sources: Results matrixes for 22 CPFs. Note: *Others include public sector management (Azerbaijan); productive capacity (Chad); productivity (Colombia); poverty (Myanmar); economic governance (Serbia). IFC’s ongoing and new engagements are well integrated in the 22 CPFs—in the main document and in the results matrix. The IFC program in the CPFs supports the countries’ inclusive growth objectives and social and economic inclusion goals as pathways to achieving the twin goals. The IFC program in 15 CPFs was clustered around the inclusive growth objectives (figure G.2). In most of the CPFs reviewed, IFC supports two or more objectives. In the Myanmar CPF, for example, IFC intends to support projects that are aligned with the country’s multiple objectives in the CPF: poverty reduction, social inclusion, competitiveness, and job creation. In the Albania and El Salvador CPF’s, IFC’s pro- grams are consistent with these countries’ inclusive growth goals. 106 World Bank Group Country Engagement | Appendix G Nearly 40 percent of IFC ongoing and new projects are in support of access to finance and financial in- clusion objectives in the CPF, and 25 percent support infrastructure development objectives (figure G.3). Programs Mainly Support Access to Finance and Infrastructure Figure G.3. IFC Development Access to Finance, Financial Inclusion (including MSME finance) 36% Infrastructure Development 25% Business Environment, Investment Climate & Trade Facilitation 10% Agriculture 8% Others 8% Multi-Sector 5% Health and Education 4% Climate Change & Environmental Management 3% 0% 5% 10% 15% 20% 25% 30% 35% 40% Sources: Results matrixes for 22 CPFs. MIGA integration in the SCD and CPF. Compared with the practice in the CAS approach, the integration of MIGA in the SCDs and CPFs is greater, but evidence of its inputs is more difficult to detect in the SCDs. The level of its involvement in both the SCD and the CPF reflects the scale of its operations and staffing relative to the World Bank and IFC. The volume of its existing exposure in the country and business prospects drive the extent of MIGA’s participation in the SCD and CPF processes. Countries where it has large exposures, such as Bangladesh, Côte d’Ivoire, and Pan- ama, are considered priorities, and its participation in the SCD and CPF has been more extensive. In the Côte d’Ivoire SCD and CPF, MIGA was present in all World Bank Group deliberations and provided verbal comments because Côte d’Ivoire was their largest country exposure in fiscal 2014.7 MIGA also attended World Bank Group meetings and provided written comments in the Bangla- desh and Myanmar SCDs and CPFs because of their large active portfolio and pipeline of projects in these countries. Its engagement in the Colombia, El Salvador, Egypt, and Uganda SCDs and CPFs was more calibrated, but MIGA’s ongoing projects and planned activities were mentioned in the CPFs. In the SCDs and CPFs for Azerbaijan, Albania, Bolivia, Botswana, Costa Rica, Chad, Haiti, Maldives, Mali, and Uruguay, MIGA participation was minimal. Nevertheless, potential or planned MIGA involvement was reflected in the respective CPF results matrix for Colombia, Costa Rica, and Uruguay. Independent Evaluation Group | World Bank Group 107 Full engagement by MIGA in the SCD and CPF processes is limited by its small staff complement relative to the World Bank and IFC. Insurers typically do not have large staff complements and their operating margins are thin. MIGA has a total of 120 staff in its headquarters and two regional hubs. The responsibility for contributing and coordinating activities related to the SCD and CPF rests with the Economics Unit of MIGA’s Economic and Sustainability Group. The unit consists of six staff and a manager, all of whom have served as the focal point and represented MIGA in the SCD and CPF process, just as in the previous CAS model (figure G.4). Figure G.4. MIGA Internal Workflow in the New Country Engagement Model MIGEX – EVP Final Sign-Off World Bank SCD MIGCO – VP ROC, Decision & Approval & CPF Team Inputs on On-going & Planned Program MIGOP - Operations MIGOP Sector Managers Group MIGEC – SCD-CPF FOCAL POINT & LEAD in MIGA Economics & MIGAH – Asia Sustainability Hub Group* (Singapore) MIGEA – Europe & Africa *Only the Risk Management Officers in Economics unit of the (Paris Office) Economics & Sustainability Group (MIGES). The personality of the World Bank team leader and World Bank country manager/director can make a difference in integration. Several MIGA and IFC team leaders mentioned the role of leadership behavior in facilitating their integration and engagement in the SCD and CPF process. When World Bank team leaders and country managers/directors have reached out to MIGA early in the process, it has made a difference in ensuring that the agency is well integrated into the SCD and CPF pro- cess, despite the agency’s constraints. In these instances, they brought MIGA into the SCD or CPF team as early as the brainstorming meetings for the SCD concept note, ensured that the MIGA team leader was copied in all communications and invited to all internal deliberations, with sufficient time to provide comments (Côte d’Ivoire SCD and Uganda SCD and CPF).8 An IFC coteam leader also noted that the welcoming attitude of the World Bank SCD team leader made the IFC’s participation in the SCD easier (Bangladesh SCD).9 On the other hand, lack of adequate communication and coordina- tion within the team can slow down the process, as it did in the Egypt SCD.10 108 World Bank Group Country Engagement | Appendix G MIGA activities are mostly aligned with CPF priorities related to inclusive growth. Of the five CPFs that included MIGA projects in the results matrix, the projects tend to cluster around CPF focus areas related to growth. The New Country Engagement Approach Brought Positive Changes IFC and MIGA staff interviewed by IEG recognized the benefits gained from the new country engage- ment model and the One World Bank Group approach. They acknowledged that the new approach has brought about several welcome changes: ƒƒ Improved coordination/collaboration of the World Bank, IFC, and MIGA compared with the previous CAS process. IFC and MIGA participation is actively sought by the World Bank and is not just pro forma. Several of the IFC and MIGA interviewees cited examples of World Bank staff reaching out to them (and vice-versa) regarding a related issue or potential collaboration as a result of their participation in the SCD/CPF exercise. ƒƒ Deeper understanding of the country’s development challenges, their underlying causes, and the complexity of the issues. ƒƒ Greater understanding of the challenges faced by each World Bank Group institution, although differences in perspectives and even institutional biases remain. ƒƒ Improved ability to fit and design IFC’s program according to SCD and CPF priorities and align these with the World Bank Group’s twin goals. It also allowed them to situate IFC interventions within a broader strategic context.11 ƒƒ Opportunity for IFC to integrate its priorities and perspectives in the World Bank Group’s country strategy. ƒƒ Greater involvement in shaping the country strategy, unlike in the previous CAS model, where the IFC strategy team was asked to write a paragraph on IFC activities for the CAS, often on short notice. ƒƒ Appreciation of World Bank research and analytical materials, which the IFC is not equipped to undertake due to limited resources and capacity. Also, generating research is not one of IFC’s core functions. ƒƒ Knowledge and learning gained from the stakeholder consultations. In countries where IFC led the private sector consultations, the SCD and CPF provided the platform for engaging with the country’s private sector over the long term.12 ƒƒ Recognition and increasing involvement of MIGA as part of the World Bank Group. The new approach has also provided MIGA with greater collaborative opportunities within the World Bank Group. Independent Evaluation Group | World Bank Group 109 The New Approach also Brought Fresh Challenges for IFC and MIGA The expectations of an integrated, One World Bank Group approach are not matched by the avail- able resources, and this is an unfunded mandate for some. There is particular asymmetry regarding the resources available to the World Bank compared with IFC, and especially compared with MIGA. While the CPF serves as the budget document for the World Bank’s engagement in the country, it does not serve the same purpose for IFC and MIGA. World Bank staff can charge their time against the budget code for each SCD and CPF, but IFC does not yet have a system for tracking costs relat- ed to country level SCD and CPF engagement. MIGA uses the time recording system to track staff time and the cost of its SCD and CPF engagement, but not at the country level. This time recording code (unfunded) was created to track staff time for participation in the SCD and CPF after concerns were expressed about the amount of time spent on these tasks compared with their main work program. MIGA does not yet have a separate budget for SCD and CPF activities. These costs are currently funded through its regular operating costs. IFC has a slightly different arrangement, whereby some of the expenses are absorbed by the country offices’ business development and operating budgets. Several IFC team leaders lamented the lack of dedicated budget for the SCD and the CPF. The level and depth of participation in the SCD, and especially the CPF process, also depended on the resources of the respective IFC in-region and in-country teams. IFC’s “hub-and-spoke” organizational structure can inhibit its integration and influence in the SCD and CPF. Although IFC has 104 offices in 94 countries, its Regional and country staff cover several coun- tries, and this had some effect on: (a) the extent of engagement in the SCD and CPF process; (b) the ability to influence and reflect its priority issues in the SCD and CPF; and (c) the participation in the consultations with stakeholders. An IFC manager reported contrasting experiences with the SCD and CPF process in two countries. She was heavily involved in one SCD, but felt excluded from another SCD’s consultative process. The local IFC office was not invited to the government consultations for the CPF, possibly because of the lack of IFC physical presence in the country. The absence of an IFC senior representative in residence in another country may have been an obstacle to a more influential IFC presence and larger role in the CPF and in that country’s economy.13 The new country engagement process needs greater flexibility with regard to the frequency of the SCD and CPF. The SCD pulled together what used to be conducted separately, but repeating the exercise every four to five years (on average) has high transaction costs, especially for IFC and MIGA, and takes away resources from implementation. Although the guidance for CPF products allows for some flexibility, IFC and MIGA staff interviewed by IEG expressed concerns over producing the SCD and the CPF every four to six years, in addition to the PLR every two years and a CLR at the end of the period (World Bank Group 2014b). Such frequency compresses the time for CPF implementation, for monitoring, and for IFC and MIGA to develop the pipeline of projects in support of the country program. The new country engagement could allow for greater flexibility than is outlined in the current guidance—for example, it could call for preparation of a new SCD and CPF after four years only if there are significant changes in country conditions (or on the World Bank side). Otherwise, the guid- ance could allow for even longer intervals. 110 World Bank Group Country Engagement | Appendix G Areas to Improve Balancing legacy and new programs in the CPF. The CPFs reflect mostly ongoing programs, high- lighting the difficulty of balancing the composition of ongoing programs and new projects. The guidance allows ongoing projects to be included in the CPF results framework and notes that the CPF document should indicate how the World Bank Group program will shift over time “even if the activities and objectives under the new direction will emerge only gradually over the course of the CPF or in later strategies” (World Bank Group 2014b, 10). However, the guidance is unclear about the appropriate balance between new and ongoing or legacy projects, as well as the time frame for ongoing projects that can be included in the CPF results matrix. This balancing dilemma is widely shared across the World Bank Group, but specifically by the World Bank and IFC. In general, the magnitude of the ongoing IFC portfolio and MIGA exposure and their business development pros- pects continue to guide the level of participation and the resources (staff and time) devoted to the SCD and CPF process. Sixty-three percent of IFC projects listed in the 22 CPF results matrixes are ongoing projects, with a few projects committed as far back as fiscal 2007. IEG found little evidence that the CPF teams rigorously assessed the continued relevance of the World Bank Group’s legacy programs. It seems appropriate that ongoing programs are not discon- tinued, so that the expected development results can be achieved. But absent a serious rethinking of their relevance, it becomes difficult to redeploy World Bank Group funding from poor-performing or low-priority ongoing projects to strategically relevant projects. It is important to note that if the CPF objectives are deliberately made to conform to ongoing programs that were not identified in the SCD as priority or focus areas, the World Bank Group runs the risk of supporting sectors, programs, or projects that do not address the twin goals. This gray area also creates attribution problems when assessing World Bank Group performance and results in the CLR. Expectations of World Bank Group joint programs and projects in the CPF. To the extent that the CPF envisions joint projects, there is already a history of joint (cofinanced), parallel and sequential projects by the World Bank Group in a country. For example, there are more World Bank Group joint projects ongoing and planned in the Bangladesh CPF than in the other CPFs reviewed by IEG. Most of the World Bank Group joint projects are in the power sector, where the three institutions are jointly coordinating their support. IFC and MIGA are also cofinancing power generation projects. However, this One World Bank Group approach to Bangladesh’s power sector preceded the CPF. There were previous World Bank Group joint investment projects in Bangladesh’s power sector such as Khulna (IFC-MIGA), Bibiyana (World Bank-IFC), and several others with advisory and analytics or Energy Sec- tor Management Assistance Program support. There were also past joint World Bank-IFC advisory services projects related to public-private partnerships and the investment climate in Bangladesh, as well as examples of World Bank and IFC advisory services joint projects envisioned in several CPFs that had their origins in past cofinanced projects. Improving the analytics on private sector development constraints and opportunities in the SCD and CPF. The new approach still reflects a one-size-fits-all treatment of the private sector, with limited new Independent Evaluation Group | World Bank Group 111 information coming from the diagnostics of private sector constraints and opportunities (hereafter referred to as PSD analysis). Private sector development is explicitly identified as a focus or priority engagement area in 10 CPFs reviewed by IEG. In the other 11 CPFs, PSD was subsumed in other CPF objectives.14 With few exceptions, and much like the previous CAS approach, the analysis of private sector contribution to the twin goals in the SCD is still very much from a government policy perspective of improving the business climate. This perspective is likely the result of relying primarily on the Doing Business reports and Enterprise Surveys to diagnose private sector binding constraints. The heavy public sector tilt in PSD analysis also reflects the World Bank’s lead role in drafting the SCDs, with IFC and MIGA providing sub- stantive inputs. Although the SCDs linked support to private sector development and growth to the achievement of the twin goals, the analysis of binding constraints for the private sector is predictable, presenting a run-down of the same recurring issues. The result is the continuity of World Bank Group programs related to access to finance through provision of credit lines for onlending to medium, small, and micro enterprises (MSMEs), as indicated in figure G.3, and investment climate approaches, which are then reflected as CPF objectives. Several stakeholders, including IFC and MIGA staff, inter- viewed by IEG for this evaluation admitted candidly that they were not satisfied by the private sector analysis in the SCD and CPF (Albania, Egypt, and Uganda). However, IFC and MIGA staff do not have the capacity or resources to undertake the analysis themselves. Analysis of Private Constraints and Opportunities in the Box G.1. Contextual Myanmar and Haiti SCDs Despite the data gaps and lack of World Bank analytical materials on Myanmar, the World Bank Group’s analysis of private sector constraints and potential sector involvement had more contextual information than the PSD analysis in the other SCDs reviewed by IEG. The Myanmar SCD identified the industries and economic sectors that present potential for private investment. In terms of constraints and challenges, the SCD described the nature of the private sector in Myanmar, which includes the dominant state-owned and military-owned enterprises in the economy that effectively stifled PSD. The SCD also analyzed which among the different groups of Myanmar’s poor would likely benefit from private sector–led off-farm employment. The private sector is expected to be the lifeline for the transitional poor. But for the traditional poor, agriculture will continue to be their lifeline, and the private sector is not expected to play a large role until other constraints facing this group are resolved. The Haiti SCD analyzed the historical development of the country’s business elites and the linkage to the elite capture of a large segment of the country’s economy today. The SCD presented a table on the import quotas held by 19 families and the effect that such arrangements have on expanding Haiti’s private sector. The Haiti CPF discusses the implications of this historical development for the failure to expand the formal private sector in the country and what is required to make this expansion happen. Sources: World Bank Group 2014a, 2015b, 2015c, 2015d. 112 World Bank Group Country Engagement | Appendix G The SCDs lack business-oriented diagnostics of the private sector that could help with selectivity in the CPF. Private sector consultations should be broader and deeper. Such weaknesses were prevalent in the previous CAS approach, and they appear to continue under the new country engagement approach. At minimum, the SCDs can include information on the structure and composition of the country’s business sector, the types of companies operating in the country, and differentiate the industry-specific constraints and opportunities according to the types and sizes of business enterprises. This implies knowledge about the country’s business world, and it requires preparatory work and consulting a wider range of business enterprises and business owners (including nonclients), not just heads of chambers of commerce or other such umbrella associations. Wider consultations also imply having time to consult stakeholders from outside the World Bank’s usual network and counterparts. With the exception of the Bolivia, Colombia, El Salvador, Haiti, Myanmar, and Panama SCDs, private sector analysis in SCDs provided little information on the profile, structure, and composition of the country’s private sector that could help with priority setting in the SCD and CPF. In the Azerbaijan SCD and CPF, for example, the private sector was described as “very underdeveloped,” without much granularity in the analysis. Yet the membership of the American Chamber of Commerce, whose executive director was interviewed by IEG, consisted of several private U.S. and Canadian compa- nies that have invested in Azerbaijan. The association of Turkish companies in Azerbaijan was also mentioned in IEG’s back-to-office report from its mission there. The Indonesia SCD and CPF did not include discussions of sufficient granularity about the country’s large and thriving private sector. Size Does Not Fit All: Analysis of Private Sector Constraints and Box G.2. One Opportunities in the Indonesia SCD and CPF Private sector analysis in the Indonesia SCD and CPF treated the private sector as a homogeneous entity, and the prescribed program assumed a one-size-fits-all strategy for PSD. The SCD and the CPF did not analyze the structure and characteristics of Indonesia’s private sector, nor were the binding constraints differentiated according to firm size, sector, or other characteristics. The SCD and CPF analyses did not mention the existence of Indonesian conglomerates that have been operating as large multinational enterprises—such as the Salim, Sinar Mas, Bakrie, Lippo, Astra, and Indorama Groups—some of which were IFC and MIGA clients. These conglomerates own diverse interests in the Indonesian economy, including cross-holdings as owners of banks, insurance and leasing, real estate, extractive industry companies, and businesses in other sectors. Private sector analysis in the Indonesia SCD and CPF seems to assume that the same problems are faced across the sector, whereas the challenges differ according to firm size, geographic location of firms and their markets, and other contextual factors. IFC’s short industry notes in the Bolivia, Colombia, El Salvador, Haiti, and Panama SCDs provided informative, granular analysis of the countries’ private sector constraints and opportunities that also facilitated prioritization in the CPFs. These industry notes could be improved and adopted in the next Independent Evaluation Group | World Bank Group 113 generation of SCDs and CPFs. In the Panama SCD, IFC’s industry notes became the basis of the SCD priorities and ensured consistency with the CPF focus areas and objectives. Nine CPFs have objectives related to PSD, while other CPFs integrated the private sector’s role into other objectives.15 Considering the private sector’s critical role in fostering inclusive growth, job creation, and eventually in reducing extreme poverty, World Bank Group country strategy documents can do better with their analysis of private sector constraints and opportunities. This would facilitate the selectivity exercise, as in the Panama SCD and CPF. Improving MSME analyses in the SCD and the CPF. Most SCD analyses of MSME binding constraints tend to be policy-oriented, do not provide direction on “how-to” and the kind of activities that would develop MSMEs, and do not consider differentiated demand. Increasing onlending facilities or provid- ing credit lines for MSMEs were listed as CPF objective indicators in 17 CPFs reviewed, but this is not often backed by an assessment of the needs of each MSME segment within the country context. The Egypt CPF intended to support MSMEs headed by women and youth, but other CPFs did not have even this level of targeting. The lack of specifics will likely present difficulty in determining the effec- tiveness of the World Bank Group’s MSME program at the end of the CPF period. As noted in IEG’s evaluation of World Bank Group support for financial inclusion (World Bank 2015a), the historic ap- proach of trying to lift the poor out of poverty through microcredit was built on a set of assumptions that proved overly optimistic. Previous support for financial inclusion focused entirely on the provision of credit. It assumed that the poor would invest these funds in productive assets, and eventually be lifted out of poverty. The evaluation showed that such assumptions were flawed and did not lead to success, because access to finance was not the binding constraint to start with. The loan recipients did not invest in productive assets, but used these funds to smooth consumption and manage their day-to-day finances, or recipients lacked the entrepreneurial skills required. As the evaluation noted, “only a very small fraction of recipients were able to develop their microenterprises and thus escape poverty” (World Bank 2015a, 36). Eighteen CPFs included objectives to help MSMEs as a means to reduce extreme poverty; create jobs; and achieve shared prosperity, inclusion, and growth.16 While these SCDs and CPFs provided the analytic justification (some more than others) for targeting MSMEs, there are gaps that can be strengthened along the lines of the recommendations in the IEG evaluations of the World Bank Group experience with targeted support to small and medium enterprises (SMEs; World Bank 2014a) and the World Bank Group’s support for financial inclusion (World Bank 2015a). In particular, the SCD and CPF diagnostics on MSMEs could be strengthened by adding: ƒƒ The profile of SMEs in the country and analysis could address each differently, based on existing country conditions. In many of the SCDs and CPFs reviewed, MSMEs are treated as a single group, even though different firm sizes face different constraints and have diverse needs. ƒƒ The definition of the SMEs being targeted could be using measurable, country evi- dence-based criteria, such as number of employees and annual revenues, along with 114 World Bank Group Country Engagement | Appendix G an explanation that the targeted SME group (some of which could be a subset of SMEs) suffers from size-specific market failures. Some CPFs (Costa Rica, El Salvador, Hon- duras, and Panama) provided a definition of MSMEs, albeit taken from IFC’s standard definition based on loan sizes—not based on country-specific evidence. The Colombia CPF had a footnote that referred to using the SME definition from the World Enterprise Survey. Although these are good starts compared with the other CPFs that did not de- fine the eligibility criteria for MSMEs, there is room for further improvement. ƒƒ In the results frameworks indicators, it can help to include the impact of World Bank Group intervention on the targeted group (not just a reference to a generic SME) and on the binding constraints. IEG’s review of the CPF result matrixes observed that the recom- mendation has not been followed. ƒƒ For MSME support channeled through financial intermediaries, a results framework that considers the evolution of the financial intermediary’s distribution of the MSME portfolio plus the composition of the beneficiary institutions and firms. This would ensure that the benefits are shifted from better-served firms and markets into underserved segments. In the CPF results matrixes reviewed by IEG, the indicators focus on outputs (number and volume of loans or MSME beneficiaries) instead of outcomes. Realism and quality of CPF results frameworks. The guidance requires a CPF results framework that “provides a concise summary of the results chains articulated in the text, showing the linkages between the country development goals, the development challenges, and the CPF Objectives” (World Bank Group 2014b, 12). The results matrix is a required annex in the CPF document that will be used to assess progress toward achieving the CPF objectives. Indicators can be either qualita- tive or qualitative, with quantitative indicators, with baselines and targets. The indicators need to be both achievable and measurable within the CPF period. The guidance also mentions that “It is best practice for the matrix to include information on data sources for indicators to assist future program evaluation” (World Bank Group 2014b, 13).17 In several CPFs, the World Bank Group program tends to overpromise on the outcomes, perhaps because there is a corrective mechanism in the PLRs. The Indonesia CPF presented ambitious targets, although most of the projects in the program are still to be determined. In the other CPFs, it is difficult to determine which of the projects in the World Bank Group program would contribute to the indicators because the entries in the CPF results matrixes are not arrayed well (Colombia, Côte d’Ivo- ire, and Mali are examples). CPF indicators were mostly outputs rather than outcomes, particularly for indicators relating to MSME access to finance. In the Azerbaijan CPF results matrix, IFC investments to nine financial intermediaries to support MSMEs were listed as part of the World Bank Group’s ongoing program to support access to finance. Achievement of this objective will use the number of MSME loans through IFC client banks as a metric. A more relevant outcome indicator would have been the increase in the share of MSME loans relative to the total loan portfolios of the IFC-supported financial institutions. Six CPFs presented the operational definition and data sources for monitoring and evaluation of their CPF results frameworks, which would greatly facilitate the preparation of the PLR and CLR.18 These can Independent Evaluation Group | World Bank Group 115 be improved by including explanations about how baseline and target indicators were calculated. The Bangladesh and Uruguay CPFs explained the indicators in footnotes, but the Bangladesh CPF results matrix stands out as best practice according to the guidance (World Bank Group 2014b) in explaining the calculation of the baseline and target indicators, which should also facilitate outcome monitoring. Box G.3. Support for SMEs in Bolivia’s CPF: Incorporating Lessons from the CLR In the Bolivia CPF, IFC’s program envisions providing credit lines to two financial institutions for onlending to MSMEs under CPF pillar 1, objective 3–Improve opportunities for income generation, market access, and sustainable intensification. IFC’s experience in supporting SMEs in Bolivia has been challenging, and its program in the CPF incorporated some lessons learned from recent experience. What is not so clear, however, is how IFC will ensure that its new/planned investment will have a different outcome. Lessons from the 2011–15 CAS CLR noted that IFC’s investments to support SMEs in the previous CAS did not achieve their objectives and were unsuccessful, including MSME credit lines provided to public banks, because the underlying problem was low productivity growth and not access to credit. The CLR lesson highlighted the need for appropriate sequencing of MSME support, especially credit support, until the more binding constraints are addressed. IFC also extended advisory support to its investee banks for expanding their SME and rural finance businesses, but its efforts have been stymied and all are on hold or terminated. Through its Business Edge training support, IFC tried to develop sustainable training services to provide local SMEs with access to applicable and customized business training. The program succeeded in strengthening the capacity of one local partner, but two others had to be dropped due to poor performance. Feedback from the local partner mentioned that the training only worked for large Bolivian companies because the time and cost involved made them prohibitive for Bolivian SMEs, which required an entirely different kind of business training. Under the newly established SME Facility in the Region, IFC initiated many pilot projects to explore assistance models. Some of the pilots were successful, others were not. Few were replicated due to dwindling donor funding. Perhaps due to the results of its advisory services program under the CAS, there is no new advisory services program in the CPF results framework. Source: World Bank Group 2015a. Inclusion of MIGA program in the CPF results framework. The MIGA program could be better reflect- ed in the CPF results matrixes, especially its ongoing projects. Private sector indicators in the CPF matrix relate to IFC ongoing and pipeline/potential investment and advisory services program and the private sector–related program of the World Bank. MIGA projects formed part of the World Bank Group program in the Bangladesh, Colombia, Côte d’Ivoire, Egypt, Honduras, Indonesia, Myanmar, and Panama CPF results matrixes, although there is no MIGA-specific CPF indicator. But matrixes 116 World Bank Group Country Engagement | Appendix G also have ongoing projects in Bolivia, Bosnia and Herzegovina, Serbia, and Uganda. MIGA’s reluc- tance to include indicators stems from (a) a business model that makes it difficult to project five years outward and (b) learning from experience when it was penalized by IEG in past CAS Completion Reports if its indicative targets were not met or the planned transaction did not materialize.19 Addressing IFC and MIGA’s lack of dedicated budget for the new country engagement model. The capacity to fully contribute in the new country engagement approach is also determined by the resources allocated for each SCD and CPF. The lack of dedicated budget has constrained the ability of IFC, for example, to influence the PSD diagnostics and priorities in the SCDs and CPFs. Resources are also needed to develop a realistic, measurable results framework although close attention by the CPF team leaders is also required. To a large extent, the quality of IFC inputs (less so for MIGA) in the CPF results matrix is determined by the resources available to the teams. The IFC team leaders for the Bangladesh CPF had the resources and staff that allowed the team to pay close attention to the results matrix. The IFC team included staff from its Results Measurement Unit, who assisted in the preparation of the CPF results matrix. In addition, the IFC Advisory Services South Asia Regional Operation is based in Dhaka, so the IFC team leaders were able to get additional support. The difference made by having addition- al staff to contribute to the preparation of the CPF results matrix is evident in the coherence of the Bangladesh CPF results matrix and the clarity of the baseline and target indicators, which included an explanation of how the baseline and target indicators were calculated. Other IFC CPF team leaders had resource constraints and were given extremely tight deadlines, in addition to the task of balanc- ing the demands of their regular responsibilities, including those as country managers (Uganda, Côte d’Ivoire, and Egypt CPFs). World Bank Group joint program expectations. Expectations of increased joint World Bank Group (cofi- nanced) projects need to be tempered by the reality in the field and lessons from previous joint projects. Ten CPFs envisioned joint projects by the World Bank Group. Most of those listed are still considered “potential,” while other CPF results matrixes included ongoing cofinanced projects by two or all the World Bank Group institutions.20 Mali and Myanmar had joint business plans or implementation plans for the agriculture and energy sectors, respectively.21 But the realization of cofinanced projects by the World Bank, IFC, and/or MIGA ultimately depends on public and private sector client demand. Greater clarity is needed in the guidance about what is meant by World Bank Group “jointness.” The October 2013 World Bank Group strategy referred to two types of joint projects: cofinancing of a single project and a sequence of projects with a shared objective and strategy (World Bank Group 2013). However, there is no such distinction in the 2014 Directive on Country Engagement or in the guidance on CPF products and expectations for joint programs need to be clarified (World Bank Group 2014b, 2014c). Is it sufficient to carry out programs in parallel, noting what the other part of the World Bank Group is doing and trying to mold programs to be broadly consistent and supportive? Or should the institution look for a more active synergy through the design, cofinancing, and implemen- tation of joint programs? IEG’s learning note on joint World Bank Group projects noted that, overall, Independent Evaluation Group | World Bank Group 117 cofinanced joint projects represented less than 1–2 percent of World Bank and IFC commitments (1.4 percent of combined commitments by the World Bank Group) for the past 20 years. The learn- ing note also found that joint projects had been useful and highly relevant in high-risk contexts, in supporting pioneering investments, and in complex projects. However, jointness also entailed costs and risks. More important, World Bank Group cofinancing of projects did not ensure successful out- comes, although such joint projects can be powerful tools and are effective under the right conditions and with robust due diligence, monitoring, and supervision. Emerging Lessons The integration of the World Bank, IFC, and MIGA as One World Bank Group has begun to take root as envisioned under the new country engagement approach. Unlike past initiatives to have an inte- grated World Bank Group country strategy, the new approach is reinforced by (a) strong World Bank Group senior management support; (b) requiring that the SCDs and CPFs be approved by the man- agements of the World Bank, IFC, and MIGA; (c) guidance notes that repeatedly emphasized the One World Bank Group nature of the SCD and CPF and outlined the processes, governance structure, organizational responsibilities, and templates; (d) including IFC and MIGA team leaders for the CPF and point persons for the SCD; and (e) establishing the CPF Academy to train World Bank, IFC, and MIGA staff. All 22 SCDs and CPFs reviewed by IEG reflected this One World Bank Group approach to country strategies. The new country engagement approach is in its early stages, and evidence from the 22 pioneering new country engagement documents show considerable improvements from the past CAS ap- proach. IFC and MIGA are integrated into the SCD and CPF processes, although institutional capacity and existing operations continue to determine the level of engagement. But there are areas for contin- ued attention—among them, improvements in private sector and MSME diagnostics, quality control over the CPF results framework, and further improvements in the SCD and CPF processes. Adequate resources are needed to produce quality country diagnostics and country strategies. Involving IFC and MIGA early in the process would allow for greater participation and ownership, especially of the CPF. However, it remains to be seen if the integration would result in increased business volume for IFC and MIGA, as well as support for projects with better development outcomes. 1 In fiscal year 2001, fewer than half of CASs were joint; by fiscal 2010–12, more than 83 percent of CASs and Interim Strategy Notes (ISNs) (63 of 76) were joint (World World Bank 2015e). 2 MIGA will be represented by the vice president/chief operating officer. 3 Based on the review of 22 SCDs and CPFs approved by the Board of Executive Directors as of May 2016. IEG conducted visits in 10 countries to interview World World Bank Group staff and other stakeholders. 4 IFC Regional directors and country managers are field positions. IFC does not have offices responsible for specific countries in its Washington, DC, headquarters. 118 World Bank Group Country Engagement | Appendix G 5 Six CPFs (Botswana, Chad, Côte d’Ivoire, Egypt, Indonesia, and Uganda) were led by IFC country managers. Eight CPFs had IFC country managers or heads as CPF co-task leaders (Azerbaijan, Bangladesh, Bolivia, Bosnia Herzegovina, Haiti, Maldives, Myanmar and Serbia). 6 The guidance note on CPF products (World World Bank Group 2014b, 7) mentions that “IFC teams will lead consultations in relation to IFC activities and will be fully engaged in consultations with the private sector in general, particularly in Tier 1 SCDs, CPFs/CENs and PLRs.” In the Albania and Indonesia SCDs, the IFC team leader was given full responsibility (including moderating) over the private sector consultations and summarizing the inputs in the documents. 7 Interview with MIGA team leader. 8 Interviews with MIGA team leaders. IEG back-to-office report for Côte d’Ivoire and Uganda country visits. 9 Interview with IFC CPF co–team leader. IEG back-to-office report for Bangladesh country visit. 10 Interview with IFC. IEG back-to-office report for Egypt country visit. 11 Interviews with IFC team and co-team leaders. IEG back-to-office reports. 12 IEG back-to-office report for Indonesia country visit. 13 IEG back-to-office report, Uganda country visit for the SCD and CPF process evaluation. 14 Private sector development was considered a cross-cutting issue in the Azerbaijan CPF, but not a focus area. The Bangladesh, Bolivia, Chad, Colombia, Costa Rica, El Salvador, Indonesia, Mali, Panama, and Uruguay CPFs did not have a CPF focus area pertaining to private sector development, but the CPF objectives involved private sector participation. The Albania, Bosnia-Herzegovina, Botswana, Côte d’Ivoire, Egypt, Indonesia, and Myanmar CPFs and the Uganda 15 SCD reflected private sector development as a priority or focus area. 16 The Albania, Bangladesh, Chad, and Uruguay CPFs did not prioritize MSME’s access to finance or financial services. 17 World World Bank Group 2014d states that “The CPF articulates a results-based engagement, centered on a results framework, that lays out the objectives that the WBG activities are expected to help the country achieve, and links those objectives to the country’s development goals” (paragraph 8, 4). 18 Costa Rica, Côte d’Ivoire, El Salvador, Haiti, Honduras. 19 IEG interview with MIGA. Bangladesh, Chad, Côte d’Ivoire, Colombia, El Salvador, Egypt, Honduras, Mali, Myanmar, Uganda, and Uruguay 20 CPFs. 21 The Myanmar Joint Implementation Plan for the Energy Sector started in fiscal 2013, before the SCD and CPF. References World Bank. 2014. The Big Business of Small Enterprises: Evaluation of the World Bank Group Experience with Targeted Support to SMEs 2006–2012. Washington, DC: World Bank Group. ———. 2015a. Financial Inclusion—A Foothold on the Ladder toward Prosperity? An IEG Evaluation of the World Bank Group Support for Financial Inclusion for Low-Income Households and Microenter- prises. Washington, DC: World Bank Group. ———. 2015b. Past and Future—World Bank-IFC Cooperation at the Country Strategy Level. Washington, DC: World Bank Group. Independent Evaluation Group | World Bank Group 119 World Bank Group. 2009. Country Assistance Strategies: Retrospective and Future Directions. Washing- ton, DC: World Bank Group. ———. 2013b. World Bank Group Strategy. Washington, DC: World Bank Group. ———. 2014a. Myanmar: Ending Poverty and Boosting Shared Prosperity in a Time of Transition. Washing- ton, DC: World Bank Group. ———. 2014b. World Bank Group Directive, Country Engagement. Washington, DC: World Bank Group. ———. 2014c. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. ———. 2015a. Bolivia—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015b. Haiti—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015c. Haiti: Towards a New Narrative. Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015d. Myanmar—Country Partnership Framework for the Period FY2015–17. Washington, DC: World Bank Group. ———. 2015e. World Bank Group Procedure: Country Engagement. Washington, DC: World Bank Group. 120 World Bank Group Country Engagement | Appendix G Appendix H. Gender Integration in the New Country Engagement Model Introduction The guidance note for the country engagement model has not integrated the gender commitments that the World Bank Group made under Operations Policy (OP) 4.20, the IDA commitments, or the commitments reflected in the corporate scorecard gender indicators (World Bank Group 2014c).1,2 Gender is only referred to once in passing in the SCD guidance note, when it lists the potential knowl- edge products that the SCD could synthetize; the identification of data gaps is considered optional (World Bank Group 2014a).3,4 The Gender Cross-Cutting Solutions Area (CCSA) has offered guidance on gender in SCDs, including World Bank Groupthe rationale for gender integration in SCDs, the relevant commitments, and how to integrate gender. It has identified the Albania SCD as good practice.5 There is no information on the extent to which this guidance is being used by teams. A roundtable was organized in September 2014 with gender experts and SCD task team leaders to discuss the challenges team leaders were facing in integrating gender in the first cohort of SCDs and to share strategies to overcome these challenges. The CPF guidance note has a paragraph on gender, stating that gender considerations can be integrat- ed in the CPF analysis, content of the program, and results framework (World Bank Group 2014d, 9). The guidance specifically refers to the World Bank Group’s commitments with regard to gender and to available guidance: Gender considerations can be integrated into the CPF in three places: the analysis, the content of the program, and the results framework. The findings of a gender assessment and constraints identified in the SCD can inform the CPF’s analysis of gender as a driver of poverty and development challenges. The content of the CPF program can include specific measures when the analysis has identified the need for gender-responsive interventions, and when there is country ownership and demand. Gender is integrated into the results framework by including gender-sen- sitive and sex-disaggregated indicators to measure outcomes. As part of the IDA17 Replenishment, the World Bank Group has committed to integrating gender considerations into the analysis, program content, and results framework of all IDA CPFs. The corporate scorecard monitors the integration of gender considerations in all IDA and IBRD CPFs, with a target of 100 percent satisfactory attention to gender by fiscal 2017. (Staff should refer to OP/BP 4.20, Gender and Development, and the guidance note.) 121 CPFs are subject to the requirements of OP/BP 4.20 on Gender and Development, which says that CPFs need to be informed by a Country Gender Assessment.6 Yet the CPF guidance says “can” and not “must” or “should,” which may confuse teams. The Gender CCSA is tasked with monitoring this commitment. Gender guidance to teams preparing CPFs has yet to be updated; existing guidance referenced in the OP or the Operations Policy and Country Services website is the material previously used for CASs.7 A New World Bank Group Gender Strategy In December 2015, the World Bank Group approved a new gender strategy, reviving its commitment to gender. The strategy focuses on economic opportunities (access to more and better jobs, and ac- cess to and control over assets and finance), human endowments (health, education, social protec- tion), and voice and agency (gender-based violence, representation of women in local service delivery governance structures; World Bank Group 2015f). Eight of the 22 CPFs included in this analysis were approved at the same time or after the new gender strategy. While the strategy explicitly refers to the new country engagement model, the CPFs approved before the gender strategy were still required to observe OP 4.20 and other gender commitments.8 The strategy, in line with OP 4.20, reconfirms the adoption of a country-level approach to gender. Balancing country-level objectives (and the coun- try-driven model) with the priorities outlined in the gender strategy, as well as the World Bank Group’s corporate commitments on gender, which require universal compliance (box H.1), could be a chal- lenging task for teams working on SCDs and CPFs, as contradictions may arise among these various principles. Box H.1. Corporate Commitments on Gender  OP 4.20 requires CPFs to be informed by a gender assessment.  IDA17 requires all CPFs to integrate gender in its analysis, program, and results framework.  Corporate scorecard tracks gender‐integrated CPFs, with a target of 100 percent. Referring to the new country engagement model, the gender strategy stresses the importance of including gender in SCDs, recognizing SCDs as the primary documents to inform CPFs: As the World Bank Group “will adopt a more systematic approach to assessing the adequacy of its country knowl- edge on gender issues” (World Bank Group 2015f, 63). The gender strategy sets high expectations: “SCDs should highlight how closing the key gender gaps in endowments, economic opportunities, and voice and agency would boost the attainment of the twin goals” (63). The strategy aims to move teams away from mere diagnostics of gender equality gaps to a “better understanding of why gaps exist, and the key constraints that must be lifted for the gaps to narrow and how the proposed [World Bank Group] support contributes to lifting these constraints individually and as a whole.” Moreover, diagnostics should highlight heterogeneity among men and women: “SCDs should also document the availability of sex-disaggregated data and identify which gender data gaps should be filled” (World Bank Group 2015f, 63). A gender assessment, when available, should inform the SCD, which in turn will inform the CPF. Alternatively, the SCD analysis itself could constitute a gender assessment according to OP 4.20.9 But problems arise in moving from statements of principle to implementation. There is nothing in the 122 World Bank Group Country Engagement | Appendix G text of the strategy that states clearly who is responsible for including gender in SCDs. The strategy requires that “at a minimum, CMUs should map the alignment between the key gender gaps high- lighted in the SCD, the CPF gender equality objectives, and the portfolio of operations. . . . CMU will look across operations to ensure they reflect appropriate actions to close key gender gaps identified in the CPF” (World Bank Group 2015f, 66). The strategy is silent about what to do if gender gaps are not identified in the SCD or the CPF or if the CPF does not outline gender equality objectives. As part of the CPF preparation process, the strategy expects wide consultations on gender with the govern- ment and civil society to help identify the objectives with regard to gender. But it is silent on the role, if any, of Regional or Country Gender Action Plans, despite acknowledging their existence in some Regions.10 Because of limited staffing, the Gender CCSA involvement in the production of SCDs or CPFs is limited to providing comments at formal reviews. It remains to be seen how the CPF and the SCD guidance will be revised to take into account the commitments and recommendations of the gender strategy. Methodology to Assess Gender Integration in SCDs and CPFs IEG reviewed the universe of SCD and CPF documents included in this evaluation to assess the treat- ment of gender. Criteria for assessment of the treatment of gender in SCDs and CPFs focus more on the process than on the substance and were developed based on the guidance note for CPFs, the guidance note on Gender for SCD developed by the Gender CCSA, and the spirit of the World Bank Group Gender Strategy: ƒƒ Whether the SCD and CPF draw on (or explicitly refer to) gender analysis that already exists (CPFs have to be informed by a gender assessment and SCDs are based on exist- ing material and are not expected to produce new knowledge). ƒƒ Whether gender dimensions were raised during consultations (consultations could pro- vide important input to identify gender priorities in a country). ƒƒ How the gender discussion is woven into the document: whether it is seamlessly inte- grated or if it appears as an add-on (relegated to a box or separate paragraph that does not appear to be linked to the rest of the document). ƒƒ How gender is treated: whether gender is equivalent to women, whether women are identified as an excluded group, whether key gender gaps are identified or not, wheth- er heterogeneity among men and women or boys and girls is discussed, whether the discussion of gender is merely descriptive or analytic, whether the gender discussion focuses on the priorities identified in the gender strategy, and similar concerns. ƒƒ Whether a person assigned to specifically look at gender issues is part of the SCD team (assuming that this relaxes the technical constraints and the limited time). ƒƒ Whether gender issues are not covered in SCDs due to a stated lack of data and, if so, identified as data gaps. Independent Evaluation Group | World Bank Group 123 ƒƒ Whether CPFs clearly outline the objectives with regard to gender, its rationale, the World Bank Group activities planned to achieve the objectives, and whether there are corre- sponding indicators in the results framework. The methodology was centered on the review of documents (at both the concept note and final stag- es), comments received at ROC meetings, and interviews with task team leaders and gender experts. The review also benefited from the recent IEG assessment of gender integration in CASs conducted for the 2015 Results and Performance of the World Bank Group (World Bank 2016, chapter 1). The aim of this assessment was not necessarily to find gender issues listed among the most pressing binding constraints to growth and shared prosperity in many countries. But, instead, to review the extent to which gender dimensions of key binding constraints were analyzed in SCDs, and the impli- cations for the country program, and discussed in CPFs. The selected criteria provided descriptive categories for the analysis. Criteria were not meant to select “good practice,” as this is highly context- and case-specific. Gender Integration in SCD The depth and quality of gender analysis in SCDs varies. In all, 19 of the 22 SCDs reviewed discussed gender issues (16 on labor, and 8 on education and health), while 8 SCDs highlighted gender issues among the main challenges. Among the earliest group of SCDs, Albania is a good example of gender integration, because it has a comprehensive discussion of the most pressing gender issues (youth unemployment, labor force participation, land rights). The discussion is not merely descriptive; issues are carried through to the prioritization exercise (land, labor, access to financial markets), and relevant data gaps in the labor markets were identified. Among the most recent cohort of SCDs, Chad stands out because it discusses in detail several constraints to women’s agricultural productivity (land, labor, cash crops), as well as the impacts of high fertility and early childbearing on women’s economic ac- tivity and maternal health. Women’s limited access to land and lack of time were included in the final list of constraints. Indonesia is typical of cases where gender seems to be an add-on. Despite the existence of a great deal of recent literature to draw on, the SCD makes limited reference to it in a single (but dense) para- graph that describes a few of the disadvantages women face in terms of economic opportunity. The issues described in this paragraph, though, seem an add-on—they are not picked up anywhere else in the document: Women tend to be more vulnerable than men due to differences in job status. Women constitute most of the self-employed and unpaid family workers, making them more susceptible to personal and financial insecurity. The gender wage gap in Indonesia is larger than in other countries in East Asia, with women only earning about 70 percent of what men earn, in part because female workers tend to have less secure terms of employment and are more likely to be self-employed, doing unpaid family work or working in the informal sector. Being a woman increases the probability of working in the informal sector by 24 percent. Women-owned SMEs 124 World Bank Group Country Engagement | Appendix H are mostly self-employed by necessity. Indonesia’s social assistance programs favor female-headed households (FHH), but as typically the sole income earner in the household, FHHs tend to be more vulnerable to shocks and their poverty rates tend to be more volatile. Furthermore, out of the large labor pool of 37 million working-age Indonesians that is inactive, 86 percent of which are women (World Bank Group 2015b, 36). Some SCDs include a summary box describing a few gender issues in the country (Bolivia, Côte d’Ivoire, Haiti, El Salvador, Mali, Uganda), but little (if any) of what is mentioned in the boxes is in- cluded in the prioritization or policy discussion. This merely descriptive treatment of gender issues is similar to what happens with social inclusion in SCDs.11 The majority of SCDs refer to women when discussing gender issues. In the Latin America and the Caribbean Region, at-risk behaviors of boys and men are sometimes discussed, but their omission is most noted (as in the case of violence associated with gangs in El Salvador or the issue of youth not in school, or working in Honduras—the so-called ninis). Several SCDs singled out women as part of excluded groups (Albania, Azerbaijan, Botswana, Chad, Costa Rica, the Arab Republic of Egypt, Haiti, Mali, Myanmar, Serbia). But this identification of women as part of groups in need of attention does not necessarily derive from a detailed gender analysis. The Serbia SCD, for instance, only briefly refers to women’s constraints in accessing economic opportunities in the labor market; the Myanmar SCD has one sentence referring to gender-related reasons for social exclusion. Detailed gender analysis in SCDs is most commonly found in the labor markets and school-to-work transition discussions (see table H.1). It is not clear why gender dimensions do not come up in other detailed sector analysis. One possible explanation might be the availability of data. Yet gender data gaps were identified in only a few countries (Albania, Bangladesh, Egypt, El Salvador, Costa Rica, and Uruguay), and some of these countries also have a good integration of gender. Moreover, SCDs that do not include a discussion of gender issues do not necessarily identify gender data gaps (table H.1). Maybe this is because identifying gender data gaps is only optional according to the SCD guidance note (World Bank Group 2014a). Another reason gender issues are disproportionately discussed in the context of labor markets and school-to-work transitions may be the nature of the SCDs them- selves; they are required to focus more on the economic constraints to (shared and sustainable) growth rather than the human development aspects.12 Another possible explanation may be that this is driven by a deliberate choice of the Gender CCSA, through its comments, given their limited resources. Gender inequality may not be recognized among the most binding constraints to the twin goals in many countries even if education, skills, and access to jobs might be important dimensions of the pri- orities identified by SCDs. Differences in constraints and opportunities between males and females, boys and girls, are very rarely included among SCD priorities. In the best cases, they made it to the long list but are lost in the prioritization exercise (Colombia). In other cases, even when the list of priorities includes gender issues discussed in the text, these linkages were not explicitly spelled out in the prioritization (Bangladesh). In several instances, when the priorities are listed, their gender impli- cations are not described. There are also cases where a single sentence is added in the prioritization chapter referring to the importance of gender in generic terms (Mali). Independent Evaluation Group | World Bank Group 125 Half of the SCDs reviewed had identified a person responsible for gender (table H.1). In two cases, the person responsible for gender was also the team leader of the SCD (Albania and Egypt). In at least two cases the person responsible for gender was also the team leader for the gender assess- ment (Albania, Bangladesh). Persons responsible for gender are not mapped to the Gender CCSA, except in one case (Botswana). Having a dedicated person responsible for gender ensures integration, but it does not necessarily guarantee good integration. While some of the best examples of gender integration in SCDs had a dedicated person for gender (Albania, Chad), there are also examples of good integration of gender, even when no person was identified as responsible (Colombia). In other cases, a person responsible for gender was identified, but the SCD included only brief references to gender (Serbia, Mali). Country gender assessments were available in 11 of the 22 countries, but two dated back to 2006 (Indonesia and Mali) while three (Bolivia, Costa Rica, and Honduras) were prepared subsequent to the SCDs. SCDs referred to gender assessments when they were current and available during preparation, three of them to an existing report (Azerbaijan, Bosnia, and Myanmar); Panama SCD to a gender note; and Bangladesh and Maldives to a forthcoming report (table H.1). Table H.1. Gender integration in Systematic Country Diagnostics Gender Reference Issues High- to Gender lighted Gender Assess- Gender Issues among Main Gender Data Gaps Region Country Expert ment Discussed Challenges Identified ECA Albania Yes No Youth Land, labor, Yes (within labor unemployment, access to markets) labor force financial participation, markets land rights ECA Azerbaijan Yes Yes Labor, education, Labor, No sex-birth ratio, education gender-based violence SAR Bangladesh Yes Yes Labor, education, No Yes (Country Gender access to credit, Assessment update recommended) LCR Bolivia Yes No Labor No No ECA Bosnia and Yes Yes (text Labor, access to Labor No Herzegovina only) financial services Gender Reference Issues High- to Gender lighted Gender Assess- Gender Issues among Main Gender Data Gaps Region Country Expert ment Discussed Challenges Identified 126 World Bank Group Country Engagement | Appendix H Table H.1, continued. AFR Botswana Yes No Labor, HIV, No No maternal health, voice AFR Chad Yes No Labor, fertility, Insufficient No access to time for productive economic resources, activities; land agency/ tenure empowerment, gender-based violence. LCR Colombia No No Labor, access to Equality of No financial services opportunities in labor markets was listed but not of highest priority LCR Costa Rica No No Labor, short No Yes (factors driving reference to secondary dropout; financial access constraints in labor market) AFR Côte d’Ivoire Yes No Land, education, Education No labor, health (population control) MNA Egypt, Arab Yes No Labor mostly, Only partially Yes Rep. education, referred to access to credit, education, health health, and credit LCR El Salvador No No Migration, No Yes (reasons for low education, teen female participation in pregnancy, labor markets) LCR Haiti No No Box summarizing No No gender inequalities LCR Honduras No No Labor, youth No No violence, ninis (youth out of work and out of school) Gender Reference Issues High- to Gender lighted Gender Assess- Gender Issues among Main Gender Data Gaps Region Country Expert ment Discussed Challenges Identified EAP Indonesia No No No discussion of No No gender issues, but brief reference in labor. Independent Evaluation Group | World Bank Group 127 Table H.1, continued. SAR Maldives No Yes General Labor No descriptive stats, some reference in labor, GBV AFR Mali Yes No Box summarizing No (but one No gender sentence inequalities interventions some data to take gender disaggregated into account) EAP Myanmar No Yes No discussion No No of gender issues (one sentence referring to gender reasons for social exclusion but no analysis) LCR Panama No Yes (note) Labor, some No No reference to GBV ECA Serbia Yes No Labor No No AFR Uganda Yes No Labor, education, Addressing No fertility, access to high fertility; finance increasing women’s participation in labor market; targeted programs; vocational training and skills development LCR Uruguay No No Labor, ninis (youth No Yes (gender-related out of work and issues of excluded out of school) groups) Note: Gender expert is noted “yes” if a person responsible for gender is explicitly identified in the SCD. Gender Assessment is noted “yes” if there is a reference to it in the SCD. AFR = Sub-Saharan Africa; EAP = East Asia and Pacific; ECA = Eastern Europe and Central Asia; GBV = gender-based violence; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. Gender Integration in CPFs CPFs include less discussion of gender issues in the background and context section than CASs used to do, probably because CPFs include less diagnostic information in general (this is supposed to be covered by SCDs). Yet CPFs need to be informed by a gender assessment, according to OP 4.20. While such reference is usually noted, it is not coming from the SCDs in most cases. CPFs seldom refer to country or Regional Gender Action Plans. The Indonesia CPF is an exception: the Country Gender Action Plan will continue to “identify, coordinate and monitor activities to promote 128 World Bank Group Country Engagement | Appendix H gender issues in the broader engagement in Indonesia as well as in specific operations and knowl- edge products across four dimensions of gender equality: endowment, economic opportunities, voice and agency, and emerging risks areas” (World Bank Group 2015a, 46). The Myanmar CPF notes that a country-level gender action plan will be developed during the CPF period. Consultations could provide important input to identify gender priorities in a country, but they are not commonly used (or their outcome was not reported). Bolivia, Colombia, Egypt, El Salvador, Mali, and Uruguay are among the few CPFs that refer to consultations touching on gender issues. Yet only the Bolivia and Egypt CPFs outline a response to issues raised in these consultations in the proposed program. Little discussion of gender is found in CPF focus areas and program objectives. This lack was also observed in CASs in the past.13 Most CPFs have a generic reference to gender mainstreaming in the portfolio, regardless of whether or not gender is identified as a cross-cutting issue. When it comes to identifying concrete actions in the work program, gender references are more common in the advisory services and analytics program than in the lending program. This limited presence of gender in CPF focus areas may have to do with the noted absence in SCDs of a discussion of the reasons underlying gender disparities, thus limiting its policy implications. Many CPFs do identify what the existing portfolio is doing to address gender considerations; this is an improvement over previous CASs (table H.2). For instance, the Chad CPF refers to ongoing health and education operations as well as to the Regional Sahel Women’s Empowerment and Demo- graphic Dividend Project to improve rural access to and quality of reproductive health and education services for girls. While it falls short of fully addressing the constraints to women’s economic empow- erment raised in the SCD (limited access to land and other productive resources, limited availability of time for economic activities due to high fertility), it is definitely an improvement. In some cases, the importance of addressing gender is emphasized in the CPF, but little is included in the proposed program. For instance, the CPF for Mali calls for more commitment, but nothing concrete is included in the work program: “transformation requires substantially improved human capital, contained demographic growth and reduced gender inequalities…. The high-value impact of gender approaches would benefit Mali, but it requires bolder political commitment” (World Bank Group 2015c, 2 and 12). Good integration of gender issues in an SCD does not necessarily translate into good integration of gender in CPFs. This might be expected when the integration of gender is seen in a topic not consid- ered a priority for the CPF (as in the school-to-work transition in Bolivia). Motivations for this absence are much less clear in other cases (such as Colombia, where the CPF does not refer to any of the gender issues discussed in the SCD, even when some of these issues were also highlighted during SCD consultations). CPFs tend to align with SCDs in the identification of key gender issues, but they do not necessarily identify strategic priorities for gender. In some cases (Albania, Haiti, Uganda, Uruguay), CPFs outline clear areas of action (not necessarily identifying them as key gender gaps). In other cases, they just refer to gender mainstreaming activities in the World Bank Group portfolio and analytical work (Colombia, Maldives, Mali; table H.2). The Panama CPF is a good example of the typical integration. Independent Evaluation Group | World Bank Group 129 While the CPF points to opportunities to integrate gender in existing operations (“Gender disparities may create barriers for taking full advantage of growth, as they affect women’s and men’s endow- ments, economic opportunities and agency” [World Bank Group 2015e, 11]), it ignores gender issues when referring to the priorities. In a few cases, exactly the same information on gender is included in both the SCD and CPF. This is especially the case when gender is discussed in summary boxes or summarized in one paragraph, as in Honduras or El Salvador, for instance. Only a few CPFs pick up on gender data gaps identified in the SCD and propose analytical work to address them (Azerbaijan, Costa Rica). In Bangladesh and Egypt, the SCD raised the need to update the gender assessment, and the corresponding CPF was diligent in including it among the proposed analytical work. In other cases (Albania, El Salvador, Uru- guay), data gaps clearly identified in SCDs are not mentioned in CPFs. The Myanmar CPF highlights gender data gaps not previously identified by the SCD (Myanmar CPF). Sex-disaggregated indicators are commonly found in CPFs, regardless of whether the CPF refers to gender in the proposed program or not (table H.2). The majority of these indicators are project ben- eficiaries’ indicators (such as percentage female). This is not surprising, considering that the number and share of beneficiaries who are female are included in the corporate scorecard, and therefore are more frequently found in projects, and thus are readily available as a potential indicator for the CPF.14 These indicators are of little value. The majority measure outputs, tracking the percentage of female recipients or beneficiaries living in projects’ areas.15 Very few indicators aim at measuring gender gaps (this is most commonly done in education). The Uruguay CPF, for instance, has an indicator on the burden of care for poor young women. Maternal mortality is not mentioned as often as it was in CASs. Other indicators included in CPFs are more focused on process, such as “gender assessment disseminated,” “government policy or programs gender informed,” and “new labor force survey that analyzes sex-disaggregated employment data and sex-disaggregated data on the wage gap and its determinants conducted” (table H.2). Emerging Findings This analysis has used descriptive categories (explicitly and implicitly identified in the gender strate- gy, some also reflected in the SCD and CPF guidelines) to assess the quality of gender integration in SCDs and CPFs. Based on these categories, IEG found little difference in the treatment of gender in CPFs compared with previous CASs as a result of the new country engagement model. Most SCDs are not providing a comprehensive gender diagnosis. Gender issues highlighted in SCDs tend to be echoed by CPFs. The majority of the SCDs and CPFs reviewed describe some gender issues countries may be facing but do not identify key gender gaps to be addressed, nor do they refer to an existing gender assessment. In SCDs, the gender gaps that receive the most attention relate to labor markets, followed by skills and school-to-work transition. In the CPFs as well there seems to be an evolu- tion beyond human development, commonly found in the previous generation of CASs, toward econom- ic empowerment, and especially related to labor issues. Poverty among female-headed households is also commonly discussed. While “gender issues” are not limited to “women’s issues,” the discussion on gender, when present in SCDs and CPFs, is generally associated with women’s issues only. 130 World Bank Group Country Engagement | Appendix H Some SCDs and CPFs are stronger than others. Implications for the World Bank Group country pro- gram of the gender considerations discussed in SCD priorities are only partially addressed in CPFs. CPFs are much more likely to refer to what is being done in the existing portfolio to address gender issues than to propose future actions to address them (except with regard to analytical work). Compared with previous CASs, CPFs are better at articulating World Bank Group actions to address specific gender inequalities and include many more sex-disaggregated indicators. But the alignment between the actions proposed and the indicators included in results frameworks is still weak. Moreover, indicators included in CPFs tend to be the percentage of female beneficiaries of existing projects, which provides little new information beyond what is already being monitored by individual projects. The absence of major change in the treatment of gender in CPFs compared with CASs is not sur- prising, considering that the guidance for the new country engagement model did not fully adopt the gender commitments that the World Bank Group made under OP 4.20, the IDA commitments, or the corporate scorecard with regard to gender. While the CPF guidance refers to these commitments, the SCD guidance note only mentions gender in passing (World Bank Group 2014b, 2014d). The World Bank Group gender strategy, finalized after the guidance was completed, provides more spe- cific directions on how gender dimensions can be incorporated in the SCD and CPF, but these have yet to be internalized in the CPF guidance (World Bank Group 2015a). Box H.2. Gender CCSA Guidance to Teams Preparing SCDs and CPFs ƒƒAddressing gender is not only a matter of inclusion (that is, women are agents of change in economic, social, and political life and not only a vulnerable group). ƒƒWomen are a heterogeneous group (only certain interactions with gender—race, education, and income, for example—may be key to address). ƒƒSex-disaggregate all individual-level indicators and identify data gaps if needed. ƒƒRecognize gaps and constraints (concrete differences between females and males), and not only in social sectors (that is, segmentation of industries, occupations, division of labor in productive and reproductive activities). ƒƒIdentify both explicit and implicit gender biases in policies, regulations, and market processes. The responsibility for gender integration in SCDs and CPFs is not clear. The Gender Leadership Council, which includes representatives from all Regional vice-presidencies; IFC; Development Eco- nomics; and the Poverty and Social, Urban, Rural, and Resilience global practices, could potentially play a key role.16 While the council does not have the specific role of overseeing the integration of gender in SCDs or CPFs among its functions, it identifies regional gender focal points that could help with this. Council members are supposed to meet quarterly to review implementation progress on the World Bank’s gender equality strategy, corporate targets, IDA 17, and Regional Gender Action Plans, among other matters. In addition, the Council is expected to “play a pivotal role in setting priorities for the World Bank’s agenda on gender equality and women’s empowerment and in fostering regional and country ownership of its activities.” The Gender CCSA is tasked with monitoring compliance with OP 4.20, but recognizing that mostly focusing on compliance with an operational policy diminishes the value of the input of the gender ex- Independent Evaluation Group | World Bank Group 131 perts, the Gender CCSA is paying little attention to this (compared with what Poverty Reduction and Economic Management Gender used to do). Instead, the Gender CCSA devotes a great deal of time to providing upstream comments to both SCD and CPF documents. Comments seem to have im- proved over time, and recent contributions are more focused and specific, many providing references and data sources (table H.2).17 Yet these comments are seldom addressed. Comments provided by the Gender CCSA to CPF and SCD teams tend to focus on the pillars of economic opportunities of the gender strategy: access to more and better jobs and access to and control over assets and finance. Much less attention is given to human endowments and voice and agency. Access to jobs is a priority identified by the strategy, but this is not the only priority. Plus, one has to square the principle of the priorities with the other principle also identified in the strategy of the country as the entry point for gender. Thus, it is insufficient to only focus on gender issues in the economic domain. Concrete inputs are often given by the Gender CCSA at the concept note and ROC stages to strengthen the discussion on gender in SCDs. In the case of CPFs, the Gender CCSA comments at ROC tend to recommend strengthening the narrative and alignment with the CPF’s overall priorities and goals. With the more streamlined process of comments, the Gender CCSA does not see the final CPF before it goes to Board; this limits the Gender CCSA ability to ensure compliance with OP 4.20. In the case of SCDs, comments from the Gender CCSA aim to encourage the diagnostics to move beyond establishing that there is a gap between males and females in a domain to discuss underly- ing reasons and what could be done about the gap (that is, to translate into policy recommendations). There is no systematic process that can guarantee a minimum level of quality with regard to gender integration, nor a clear definition of what a minimum threshold looks like. The Gender CCSA and the Poverty Global Practice are discussing the possibility of working with selected upcoming SCDs to come up with good practice guidance to teams. Maybe what is needed is a more proactive role of the Gender CCSA. For instance, it might be useful to decide whether it is best to aim at being systematic, even if that implies doing a basic analysis but covering a broad range of potential issues, or whether it is worth choosing two or three transformation elements for SCDs, and discussing the implications for CPFs. Maybe the Gender CCSA needs to do more than just comment and issue guidance notes (which are of limited value if they do not actively participate in the process). Task team leaders and gender experts interviewed for this evaluation mentioned several challenges to detailed integration of gender in SCDs. Challenges most often mentioned include: lack or limited techni- cal capacity in the team; lack or limited availability of data, tendency to rely on household surveys/quan- titative data and limited use of qualitative data, absence of Regional or country strategies or strategic plans with regard to gender, having to address multiple topics under severe time constraints, and lack of realistic support by the Gender CCSA. Many of these challenges were also raised at a roundtable in September 2014 (a few months after the first round of SCDs were completed). The lack of incentive mechanisms and clear responsibility and accountability systems are potential reasons for the limited change with regard to gender integration in the new country engagement model.18 132 World Bank Group Country Engagement | Appendix H Timing may also be an issue for integrating gender more systematically in SCDs or CPFs. SCDs (and CPFs) are supposed to rely on existing material that might not always be readily available. In some cases, the Regional Gender Action Plan is finalized after the CPF. In others, the Country Gender Assessment is an action planned during CPF implementation. Space is also often a concern, since CPFs are becoming shorter and more concrete. 1 OP 4.20 requires the CPF to be informed by a gender assessment. Gender assessments may be stand-alone products or carried out as part of other World Bank economic and sector work, such as poverty assessments (see OP 1.00, Poverty Reduction), country economic memoranda, Public Expenditure Reviews, development policy reviews, poverty and social impact assessments, or institutional analyses. Analyses carried out by other donors may also count as gender assessments, provided their quality is acceptable to the World Bank. 2 In particular, a commitment reiterated since IDA16 and reflected in the corporate scorecards requires that 100 percent of country strategies integrate gender (identify gender gaps and discuss potential policy responses). The recent World Bank Group Gender Strategy confirms that the country is the main entry point for addressing gender, hence the requirement that country strategies discuss and address gender issues of importance for the country. 3 Gender assessments are mentioned among the list of World Bank Group knowledge products that the SCD could synthetize (World Bank Group 2014b, 3) 4 The data diagnostics in the SCD guidance note suggest that teams, on an optional basis, can include information on whether each of the surveys/censuses has sex-disaggregated data (World Bank Group 2014b, 3). 5 The World Bank commitments refer to gender integration in the country strategies and the existence of a gender assessment as a diagnostic tool; SCDs are not formally required to integrate gender. However, considering that the SCDs are meant to provide the diagnostic for properly guiding the preparation of the country strategy, and that the World Bank often needs to play a proactive role to generate and sustain country demand for gender engagement, functionally speaking, it appears that the SCD should discuss gender issues for a meaningful integration of gender in the CPF to take place, as required. 6 OP 4.20: Gender and Development. https://policies.worldbank.org/sites/ppf3/ PPFDocuments/1680090224b08231c012.pdf 7 See http://siteresources.worldbank.org/INTGENDER/Resources/CAS_gpn.pdf (accessed September 8, 2016) 8 Available at https://openknowledge.worldbank.org/handle/10986/23425 9 OP 4.20 states that gender assessments may be stand-alone assessments or may be carried out as part of other World Bank economic and sector work, such as poverty assessments (see OP 1.00, Poverty Reduction), country economic memoranda, Public Expenditure Reviews, development policy reviews, poverty and social impact assessments, or institutional analyses. 10 Country Gender Action Plans are prepared in the East Asia and Pacific Region. Regional Gender Action Plans are in need of an update for all Regions except Africa http://globalpractices.worldbank.org/gender/ Documents/Forms/AllItems.aspx?RootFolder=%2Fgender%2FDocuments%2FRegional%20Gender%20 Action%20Plans&FolderCTID=0x012000790AA73A60831A49973AF32D66F5B801&View=%7BE98114D6%2- D34B2%2D41E3%2DB4A8%2D46C8D71C1CA0%7D (accessed September 28 2016) 11 This paper reviews if and how social inclusion is addressed in the first 17 SCDs. The paper concludes that while data on social inclusion is often presented, there is little discussion about the reasons for such exclusion, and policy recommendations to address it are even less common. http://imagebank.worldbank.org/servlet/WDSContentServer/ IW3P/IB/2016/08/02/090224b0844b281b/3_0/Rendered/PDF/Social0Inclusi00country0diagnostics.pdf 12 SCDs aim to address the question: what are the most critical constraints (and opportunities) facing a country in accelerating progress toward the goals of ending extreme poverty and promoting shared prosperity in a sustainable manner? Independent Evaluation Group | World Bank Group 133 13 See World Bank 2016 for a more detailed discussion of gender in CASs. 14 For the most recent World Bank Group corporate scorecard report see http://pubdocs.worldbank.org/ en/331941477328080420/World-World Bank-Corporate-Scorecard-2016-full-version.pdf 15 For a more detailed discussion on the limitations of the information coming from these indicators, see W 2016a. 16 Representatives from a subset of other global practices and CCSAs are also included on a rotating basis. 17 The same person in the CCSA is now responsible for giving comments at the concept note and ROC stages for both the SCD and CPF to ensure coherence. 18 The director-general, evaluation, statement to the Board on the occasion of the presentation and discussion of the World Bank Group Gender Strategy in December 2015 includes the observation that accountability and incentive mechanisms play a crucial role in the implementation of the gender agenda. IEG observed that “issues like inadequate project preparation work, poor quality at entry, and reduced budgets for supervision, are common and may affect the quality of gender integration as much as they do affect other aspects of project performance. More specific to gender, issues related to prioritization among competing corporate pressures, proper definition and enforcement of quality standards for what are often perceived ‘add-on’ topics, and management ‘buy-in’ are also important. All these factors need to be explicitly analyzed, and incentive mechanisms and clear accountability systems need to be defined, to address them effectively.” References World Bank. 2016. Results and Performance of the World Bank Group 2016. Washington, DC: World Bank Group. World Bank Group. 2014a. Interim Guidelines for Systematic Country Diagnostics. Washington, DC: World Bank Group. ———. 2014b. Systematic Country Diagnostics: An Overview of Current Thinking. Advisory Group for SCD. Washington, DC: World Bank Group. ———. 2014c. World Bank Group: A New Approach to Country Engagement. Washington, DC: World Bank Group. ———. 2014d. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. ———. 2015a. Indonesia—Country Partnership Framework for the Period FY2016–20. Washington, DC: World Bank Group. ———. 2015b. Indonesia Systematic Country Diagnostic. Connecting the Bottom 40 percent to the Pros- perity Generation. Washington, DC: World Bank Group. ———. 2015c. Mali—Country Partnership Framework for the Period FY2016–19. Washington, DC: World Bank Group. ———. 2015d. Myanmar—Country Partnership Framework for the Period FY2015–17. Washington, DC: World Bank Group. ———. 2015e. Panama—Country Partnership Framework for the Period FY2015–21. Washington, DC: World Bank Group. ———. 2015f. World Bank Group Gender Strategy (FY16-23): Gender Equality, Poverty Reduction, and Inclusive Growth. Washington, DC: World Bank Group. 134 World Bank Group Country Engagement | Appendix H Table H.2. Gender Integration in Country Partnership Frameworks Gender Actions Included in Gender Actions Included in CPF ASA Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks ECA Albania Gender-disaggregated data IFC Investment to finance n.a. Increase in the percentage of properties registered and monitoring backed by underserved segments including in the digital registration system 15 percent (2018)— gender-sensitive analysis small and medium enterprises gender-disaggregated data. People in targeted will be systematically used (SMEs) and women; expanded forest and adjacent communities with increased during project preparation and inclusive land registration. monetary/nonmonetary benefits from forests: 500 and implementation; (2017), 1,000 (2018); and from agricultural lands: gender and inclusion will 500 (2017), 800 (2018)—all data disaggregated by also be addressed in a gender. cross-cutting manner in all relevant ASA. ECA Azerbaijan Mainstreaming gender n.a. Analytical work that target knowledge gaps Increased number of customers annually using considerations in project identified by the SCD; awareness raising on the the Azerbaijan Service and Assessment Network’s design; disaggregation economic, demographic, and social effects of service centers, mobile units and e- and m-enabled of results of investment the skewed sex ratio at birth; policy dialogue services and applications to access selected public projects by gender. on topics highlighted by the Country Gender (30 percent women); Increase in percentage of Assessment and the regional study on missing vulnerable groups (including 20 percent of women) girls in the South Caucasus. among legal aid beneficiaries services; Justice Sector gender strategy and Action Plan prepared and publicly disclosed (2017); number of internally displaced persons directly benefiting from World Bank Group program exceed 30 percent of total number of internally displace persons (including 50 percent women); number of rural poor, including women, benefiting from World Bank Group– supported economic activities at community level. Independent Evaluation Group | World Bank Group Table H.2. continues on the next page. 135 Gender Actions Included in Gender Actions Included in CPF ASA 136 Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks SAR Bangladesh World Bank Group activities IFC will provide investment to IFC will provide advisory services to strengthen Number of river ports with women-friendly to consolidate equity and strengthen women’s access to women’s access to finance. facilities built/ improved; number of previously access gains in health and finance; Ongoing IDA operation Planned AAA (including a gender assessment) out-of-school children enrolled in primary education with continued will combine livelihood support to help address constraints on women’s schools in underserved upzilas and urban slums due regard to gender. with women’s empowerment employment and asset ownership. (sex-disaggregated); Increased secondary gross and access to finance and enrollment ratio for the poorest quintile (sex- community development; disaggregated); percent of deliveries by skilled IDA financing to contribute to birth attendant among the two lowest quintiles of increased agricultural productivity the wealth group; proportion of women receiving of smallholder farms and improve postnatal care within 48 hours (at least one visit); their access to markets, with number of targeted poorest beneficiaries receiving special emphasis on women social protection benefits (sex-disaggregated); farmers. 360,000 females participating in community- based decision-making structure by 2020; additional people with access to multipurpose shelters in vulnerable coastal districts by 2020 (50 World Bank Group Country Engagement | Appendix H percent female); number of people protected from tidal and storm surges in select areas (50 percent female); number of farmers who adopt improved agricultural technologies (35 percent female). LCR Bolivia Most new projects in The Access and Renewable n.a. Increased number of people provided with access the rural and urban Energy Project includes to electricity by household connections (50 development areas contain preliminary assessments on percent women); percent of the population in the gender-disaggregated data gender inclusion. bottom quartile of municipalities (according to the in their results matrixes. The Improving Employability and UN vulnerability index) benefiting from expanded Labor Income of Youth Project access to basic services (45 percent women); provides a greater stipend for number of households in alliances that receive young women and mothers to financing support (30 percent women). cover transport and child care. The Community Investment in Rural Areas Project has an explicit mechanism to benefit sub- projects designed, implemented, operated, and maintained by women’s groups. ECA Bosnia and n.a. n.a. Skills Measurement Program (STEP) to Additional enterprises reached with financial Herzegovina understand gender-related skills gaps, including services (percent owned by women); decrease the employers’ perception on women’s skills. The number of inactive persons (sex-disaggregated); World Bank will continue to support these Increase in private sector formal employment efforts through country dialogue and advice, (sex-disaggregated); reduced by 20 percent and subject to client demand, through new number of women who are registered with the financing, for example, to support pension Public Employment Services as active job seekers, reform implementation. but are not looking for work because they are IFC Advisory Services proposed: SME banking informally employed or inactive. advisory (risk management, gender, and agri- finance). AFR Botswana Gender-disaggregated n.a. Analytical work to assess the extent and causes Improved transition rate from junior secondary to indicators to be introduced of gender, poverty, and inequality will propose senior secondary education, including enrollment in new lending operations actionable recommendations for addressing the in the vocational/professional training stream in education, health, and main gender challenges. The World Bank will (girls transition rate); Increased intake into senior social protection. conduct a gender assessment early in the CPF secondary school in underserved districts (intake period to address needs in gender-responsive rate for girls); percent children (under five years) policies and actions for poverty reduction and with suspected pneumonia taken to appropriate economic growth, and to inform World Bank health provider—boys/girls ratio per age group; project and task-level interventions. percent children (under five years) with diarrhea treated with oral rehydration solution—boys/ girls ratio per age group; percent reduction of households in Q4 and Q5 receiving benefits from more than one program (female beneficiaries); percent of poor beneficiaries Q1 and Q2 supported by social protection programs (female beneficiaries). Independent Evaluation Group | World Bank Group Table H.2. continues on the next page. 137 Gender Actions Included in Gender Actions Included in CPF ASA 138 Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks AFR Chad n.a. Improve rural access to and quality n.a. Number of community projects financed on of reproductive health services: productive assets, rural infrastructure, and the World Bank Group program Sustainable Land and Water Management (of which will build on the ongoing operation women’s groups); in health and the Regional Sahel women of reproductive age in targeted regions Women’s Empowerment and using modern methods of family planning; pregnant Demographic Dividend Project women in targeted regions receiving antenatal care (emphasis on maternal health). during a visit to a health provider; births (deliveries) Improve rural access to and in targeted regions attended by skilled health quality of education: the World personnel; percent of participating girls and women Bank Group program will include (10–19) with improved knowledge on Reproductive, ongoing operations in education Maternal, Neonatal and Child Health and Nutrition; and women’s empowerment, primary completion rate in targeted regions (sex- combined with possible new disaggregated); percent of dropout from secondary investment in technical and schools in participating (adolescent girls). vocational education and training World Bank Group Country Engagement | Appendix H and postbasic skills (emphasis on girls’ education). LCR Colombia There has been an n.a. n.a. People provided with access to improved water increasing focus on sources under the Plan Pacifico project (of which gender-related activities in percent women); the portfolio, but a more share of women benefiting from ACCESS loans; strategic and consistent number of job placements achieved through the approach to promoting new system (of which women). gender equality through the World Bank’s interventions is needed. LCR Costa Rica n.a. n.a. Analytical work to identify drivers of low Increased number of students enrolled in secondary completion and developing options priority areas in four public universities (sex- for improving retention using a gender lens; and disaggregated); increased quality of services, to better understand gender disparities and as measured by decreased patient hospital stereotypes in education and labor markets (low readmissions (disaggregated by gender); female labor force participation and higher rates increased efficiency, as measured by reduced of participation in informal sector). unnecessary hospitalizations for surgical IFC will continue to provide support and advisory procedures eligible to be performed in an services to help identify areas for improvement in outpatient setting (disaggregated by gender). current MSME credit operations and strengthen institutional capabilities to better serve this market, including women-owned businesses. Targeted technical assistance on good practices in development finance to contribute to improving access to finance for MSMEs, entrepreneurs (particularly women and youth) and small agricultural producers. AFR Côte d’Ivoire Mainstream gender n.a. Policy-focused gender diagnostics and Number of farmers reached (gender- into projects in human analytical work; disaggregated); land parcels with use or development, employment, support reforms to enhance women’s social ownership rights recorded as a result of the microfinance and agriculture and economic inclusion. project (gender-disaggregated); ratio of girls (no specific objective). For IFC, this would include strengthening to boys in primary and secondary education; Gender-disaggregated women’s access to credit and business percent of students in science, engineering, data and gender-specific opportunities. indicators will continue and tertiary) (gender-disaggregated); percent and mathematics courses (upper‐secondary to be included in relevant of participating graduates who are employed projects. (gender-disaggregated); number of households or self‐employed 6 months after completion registered with a unique identification number in national registry (gender-disaggregated); number of people with access to a basic package of health, nutrition or reproductive health services (gender-disaggregated); additional urban poor with access to improved water source (through social connection program) (gender-disaggregated). Independent Evaluation Group | World Bank Group Table H.2. continues on the next page. 139 Gender Actions Included in Gender Actions Included in CPF ASA 140 Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks MNA Egypt, Arab Rep. Gender-disaggregated The design of the Upper Egypt Update gender assessment. Women served through the line of credits; monitoring of results will be Local Development Project beneficiaries of Active Labor-Intensive Public an integral part of World will be informed by the gender Works program (40 percent female); number of Bank Group initiatives under focused ASA, which highlighted targeted households accessing ownership and the CPF. the challenges of labor market rental housing units with support from the Program The benefits in focus area 3 participation and opportunities for (of which FHH); female project beneficiaries from (inclusion) will accrue, to a women. Household Gas Connections Project. large extent, to women. LCR El Salvador n.a. Interventions for violence n.a. Number of students in schools adopting the prevention, as well as those for Integrated Full-Time School Model (percent employment and employability, female); number of youth that receive training would be directed to youth and under new “employability and skills” program women. (percent female). LCR Haiti n.a. Inclusive Growth: address the n.a. Number of jobs created in targeted value chains obstacles that hinder women’s (disaggregated by gender); number of small World Bank Group Country Engagement | Appendix H access to income opportunities, businesses activities supported by tourism access to finance, and overall their development grant (disaggregated by gender and economic exclusion. youth); percent of adult women (age 18+) with an Human Capital specifically aims to account at a formal financial institution; number support poverty reduction through of direct beneficiaries with electricity services investments in education and financed by World Bank Group operations health, with the objective of closing (disaggregated by gender); number of permanent geographical and income gaps in jobs (of which women); number of primary service delivery. Attention to women students attending effective nonpublic schools and girls is critical to realizing these (disaggregated by gender); percent of pregnant goals. women having prenatal consultations before birth, Resilience: The World Bank Group among women living in the catchment areas. will help Haiti prepare for and prevent natural disasters and strengthen climate resilience, which pose a major threat to women, including a large percentage of households headed by women. LCR Honduras Cross-cutting theme. n.a. The World Bank Group will provide technical Percent of program beneficiary children aged 16-18 As gender is a key assistance on the implementation of violence years that completed lower secondary education socioeconomic category, a prevention initiatives that reduce specific risk (ninth grade), disaggregated by gender; number of gendered lens is being used factors, particularly those related to youth-at- children under age two attending the growth monitoring both to identify barriers and risk and gender-based violence. program in GAFSP-targeted areas in the Corredor design policies to eliminate Seco (disaggregated by gender); number of new rural or mitigate them. producer organizations linked to emerging markets (for example, cacao) (members disaggregated by gender); number of rural poor who benefit from increased food production and rural household income generation (disaggregated by gender); neighborhoods where gender-based violence prevention interventions are being implemented; Beneficiaries reached through school- based violence prevention initiatives (disaggregated by gender); youth-at-risk participating in employability initiatives (disaggregated by gender). EAP Indonesia Cross-cutting theme. Areas Continuation of IFC programs n.a. Number of people with improved access to urban with a particular gender for female entrepreneurs and infrastructure and services in the targeted slums, focus include financial SME finance will target women in disaggregated by gender; percent of mothers and sector development, financial inclusion programs. children receiving maternal and child health and village law, and poverty nutrition services in community health center and targeting, education, its network in targeted areas; number of students health, environmental enrolled in schools meeting minimum service sustainability, and disaster standards in targeted areas, disaggregated by mitigation. Gender-specific gender; number of people having access to improved monitoring and evaluation water services in targeted areas, disaggregated by will be further strengthened gender; number of people having access to improved in the CPF engagement sanitation services in targeted areas, disaggregated areas, including outcome by gender; number of individuals/ microenterprises indicators where possible reached with financial services, disaggregated by and relevant. gender; number of households benefiting from Program Keluarga Harapan (Indonesia’s Conditional Cash Trasfer Program, disaggregated by gender). Independent Evaluation Group | World Bank Group Table H.2. continues on the next page. 141 Gender Actions Included in Gender Actions Included in CPF ASA 142 Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks SAR Maldives Cross-cutting; no specific n.a. n.a. Number of students who will benefit from access objective. and quality enhancing activities increased (number female); government policy or programs on youth or gender informed; gender assessment disseminated; direct beneficiaries of Atoll/Island Councils that directly benefit from training on mainstreaming environmental management and climate change into island development planning and monitoring (number female); new labor force survey (that analyses sex-disaggregated employment data and sex-disaggregated data on the wage gap and its determinants) conducted. AFR Mali No specific objective n.a. n.a. Cereals held in stock by women; New Loans and mentioned, noted the Outstanding Portfolio (Microfinance), of which are importance but said it for women; percent of births (deliveries) attended World Bank Group Country Engagement | Appendix H requires “bolder” political by skilled health personnel; number of adolescents, commitment. women and men who have been reached by social and behavior change communication interventions; out-of-school youth who are employed or self- employed within one year of completion (30 percent female); out-of-school youth who are enrolled in dual apprenticeship programs supported by the project (percent female). EAP Myanmar Gender considerations will Support for access to finance Raising awareness and capacity, and improving Population in townships benefiting from improved be integrated in all IDA- includes particular emphasis on analysis and data to inform policy; a country- access to and use of rural infrastructure and services supported operations. reaching out to women and FHH level gender action plan will be developed. under World Bank Group interventions (at least 40 [similar on support to farmers]. percent female); deliveries with skilled birth attendant (percent); deliveries which are followed by adequate payments (at least 40 percent female). post‐natal care (percent); students receiving stipend LCR Panama n.a. Gender-specific transport needs; n.a. Percent of the bottom quintile benefiting from social emphasize women’s roles as assistance increased (disaggregated by gender); citizens, entrepreneurs and number of beneficiaries in the integrated social professionals (through Red de assistance registry receiving training and access to Oportunidades); enhancing market opportunities (disaggregated by gender); the voice of women in water number of youth enrolled in Red de Oportunidades boards and decision-making who benefit from training or employment services mechanisms. (disaggregated by gender); additional people with access to reliable water supply or to improved sanitation services in targeted lower-income urban areas and rural areas (disaggregated by gender). ECA Serbia n.a. Property registration and records; Technical assistance and economic and sector Number of participants in public works in regions inclusion and productivity (Skills work on skills, labor market regulation; technical where most of state-owned enterprises were Project). assistance on Gender and Roma. located will increases (to be monitored by gender); five mobile teams operational in major registration offices to assist people with disabilities; Roma; women in rural areas and others with difficulties accessing land administration services; Increased number of registered unemployed who found a formal job (of which female); increased number of registered unemployed women who found formal job; 98 percent of all children attend preschool education at age 6 (to be disaggregated by gender). AFR Uganda n.a. Improving educational Legislation and policies on sexual and gender- Gross enrollment rate, lower secondary school opportunities for girls at the based violence against women and girls. (percent, of which girls); primary school completion secondary school level; integrate rate (percent, of which girls); deliveries with skilled gender aspects into social attendants (percent); gender-disaggregated data protection activities; responding on rural land registration available; to inequalities in land use and gender-disaggregated data on urban land housing finance, labor market registration available. readiness, and access to finance for MSMEs. Independent Evaluation Group | World Bank Group Table H.2. continues on the next page. 143 Gender Actions Included in Gender Actions Included in CPF ASA 144 Region Country Gender Mainstreaming CPF Lending Program Program Indicators in Results Frameworks LCR Uruguay n.a. Skills training with a focus on Potential ASA/RAS on care. Enrollment in initial education (3-year-olds of total bridging the gap between jobs population), data to be disaggregated by gender if and schooling. available; reduce the percent of women between 18 and 29 years old from quintiles 1 & 2, with at least one child less than 4 years old who do not study, work and are not looking for a job because they dedicate their time to take care of children or older/disabled people in their home; increase the percentage of teenagers between 15 and 17 years who participate in formal education, data to be disaggregated by gender if available. Note: Gender mainstreaming refers to areas that the document highlights as receiving special attention when it comes to gender, in addition to what is mentioned in the specific program (listed in the other two columns). ASA = advisory services and analytics; AFR = Sub-Saharan Africa; ASA = advisory services and analytics; EAP = East Asia and Pacific; ECA = Eastern Europe and Central Asia; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; n.a. = not applicable; SAR = South Asia. World Bank Group Country Engagement | Appendix H Appendix I. Assessment of Governance and Political Economy Issues in SCDs and CPFs Purpose The purpose of this appendix is to assess how SCDs have addressed governance and political econ- omy issues, and how this analysis affected the design of the country program in CPFs in ways known to enhance the chances of achieving the objectives sought in their operations. The review considers the adequacy of the guidance and directives in this area (World Bank Group 2014a, 2014b), and the content of the SCDs and CPFs. Guidance and Directives Guidance for SCDs points out that political economy factors are a crucial part of country context, to be used in identifying constraints, solutions, and opportunities. Issues include weak incentives or official capacity, low payoffs to political actors, predatory behavior, political volatility, transparency, accountability, access to information, quality of core government systems, contestation over land access and use rights, and other political risks. Guidance for CPFs also suggests three key aspects of governance that may be considered: the cred- ibility and legitimacy of the government, incentives for pursuing public interest policies, and capacity of the public administration to implement the policies. Governance may be a cross-cutting theme mainstreamed throughout the CPF. It may not address all the priorities of the SCD when there is no political traction to address that challenge, but if deemed important enough, there may be need for a tough dialogue and investment in political consensus building. Latest Thinking on Governance and Political Economy The World Development Report 2017 presents the latest thinking on governance for achieving the World Bank’s twin goals (World Bank Group 2017).1 In this paragraph, IEG summarizes some perti- nent points from the literature.2 Briefly, there are three key points: “think of function, not only form,” “think of power, not only capacity,” and “think of the role of law, not only the rule of law.” Whether a policy functions and is carried out is more important than the form that it takes. Function depends on commitment, cooperation, and coordination of the key parties concerned. Often extensive behavioral change is needed to make this happen. When commitment is not credible and cooperation is diffi- cult, key parties may stop bargaining and withdraw, or even turn to conflict. Reform success comes from addressing the key constraints to commitment, cooperation, and coordination that stand in the way of promoting development by nurturing prodevelopment coalitions. Progress is more likely when building on processes that have worked in the past, and avoiding processes that have previously failed to gain traction. Security issues are critical for fragile states, and there are methods for improving security outcomes: power sharing, resource redistribution, dispute settlement, and sanctions. Power sharing can take place in legislatures and through other means to get elites and their followers to compromise. Re- distribution can take place through budget transfers, targeted service delivery, and jobs programs. 145 Dispute settlement can be done through truth and reconciliation commissions, arbitration, and cus- tomary councils. Sanctions can include increased policing, with safeguards for citizen’s rights. Failures in policy functionality often come from powerful interests. While capacity building is import- ant, new capacities will not be effective or sustained unless powerful interests let them. The adoption of laws by countries is no guarantee that they will be enforced, and weak capacity often suits pow- erful interests. Moving from form to function is a process of contest and bargaining among powerful interests. The effectiveness of policies depends not just on what policies are chosen but also on how they are chosen and implemented through bargaining and contest among different actors. Donors need to take into account the openings and constraints presented by shifting politics in supporting governance reforms. Overall Assessment Taking into account the SCD and CPF guidance, and the latest thinking as presented in the World Development Report 2017, this appendix qualitatively assesses the governance and political econ- omy aspects of SCDs and CPFs using a purposive sample of seven countries: Azerbaijan, Bangla- desh, Chad, Colombia, the Arab Republic of Egypt, Indonesia, and Uganda. These countries were selected because among the countries reviewed in the evaluation, they are known to have serious governance issues. Two are from the Africa Region: one Anglophone and one Francophone, and one from each of the other World Bank Regions. Based on this, there is considerable evidence that World Bank teams are taking into account the guidance, with governance and political issues analyzed and addressed in SCDs and CPFs, but at different levels of intensity. For example, weak incentives for reform feature in numerous ways, with proposals for incentivizing professionals to take hard-to-fill positions and to improve results through performance-based management. Weak capacity is also widely discussed, pointing to causes of limited human capital and weak educational and health systems. However, this analysis may miss the point that regimes may intentionally focus their best people on priority areas, such as security, while limiting capacity in areas where it is not wanted, such as in au- dits of high-value procurements or environmental safeguards. Regimes may well support new laws, regulations, and computerized systems, but not their functional implementation. In this way, regimes can take on the appearance or form of reform, without improving governance outcomes. As pointed out in the World Development Report 2017, moving from form to function often means bargaining among powerful interests (World Bank Group 2017). The World Bank needs to consider the openings and constraints presented by shifting politics in supporting the bargaining process. This was not called for in the guidance, and not explicitly done in the reviewed SCDs and CPFs. Governance features prominently in the SCDs of five of the seven countries as a key constraint (Azerbaijan, Chad), overarching priority (Egypt), main cross-cutting issue (Indonesia), and intrinsic part of sustainability (Uganda). There is less emphasis in the other two. For Bangladesh, there is some 146 World Bank Group Country Engagement | Appendix I discussion in a two-page annex of strengthening public institutions. However, governance is not a key priority, and political economy analysis was dropped from the SCD at the request of the government.3 In the case of Colombia, governance is less prominent than for other countries analyzed for this report, although some aspects are covered in pillar 1 on balanced territorial development. Governance features in all CPFs, but at different levels of priority. The Egypt CPF supports a transfor- mative program to renew the social contract to support private sector job creation, social inclusion, and enhanced governance. Governance is at the heart of all the areas, with policy and institutional reforms, increasing transparency and accountability at the sector level, and enhancing citizen voice and grievance redress. Likewise, the Uganda CPF states, “the CPF will, in every activity and through- out the portfolio, focus on understanding political economy dynamics, institutional and capacity bottlenecks in implementation, and aim to improve transparency, accountability, and service delivery” (World Bank Group 2015d, 17). Governance is also a key priority of the Azerbaijan CPF, with a focus on “better public resource management, service delivery, and a rules-based relationship between the public and private sectors. The idea is to build on this a basis for broader reforms and engagement when the government is ready” (World Bank Group 2015d, ii). At the same time, the Bangladesh CPF views governance reform as a long-term agenda rather than an immediate priority. It includes main- taining ongoing public sector reforms, capacity building within ongoing operations, and enhanced fiduciary oversight and use of forensic audits; any transformation will be a long way off. Governance is also addressed as a risk factor. The main risk (high) to the Bangladesh CPF emerges from the political and governance environment and weak fiduciary capacity. There is a substantial political and governance risk in the Azerbaijan CPF. It is mitigated by taking governance and institu- tion building into consideration in every World Bank Group activity, use of diagnostic tools such as Public Expenditure Reviews and Financial Accountability Assessments to monitor improvements, and limiting corporate management risks through IFC advisory activities. There is a moderate political and governance risk in Colombia’s CPF, including possible lack of political support at the local and provin- cial levels that could lead to delays, combined with low administrative and governance capacity at the subnational levels. These risks will be mitigated by strengthening subnational institutions, simplifying governance and fiduciary structures for project implementation, and addressing issues of conflict and displacement, drawing on lessons of the World Development Report 2011: Conflict, Security, and Development. Governance and political economy are sensitive issues everywhere, and this sensitivity appeared in the SCD process in some countries. Political economy analysis was dropped from the Bangladesh SCD on the insistence of the government, and governance reform is not an immediate priority in the CPF. A draft of the Egypt SCD was leaked after the decision review meeting, leading to concerns about the focus on governance issues among some in the government. While this led to a 10-month delay in revising the SCD, governance reform was retained as a central priority in the CPF. Weak incentives to implement reforms and provide quality public services are discussed in many SCDs and CPFs—for example, informal networks between business and administrative officials, Independent Evaluation Group | World Bank Group 147 which provide incentives for weak implementation and enforcement; reticence to implement reforms at the middle and lower levels due to lack of incentives; misaligned incentives and informal payments in health and education (Azerbaijan); a political context in which project selection and approvals take place, limiting incentives to produce technically high-quality projects; and a system of promotion based on years of service rather than performance, with no adequate incentives to encourage initia- tive and reward excellence (Bangladesh). To address these constraints, proposals are offered: providing incentives for the gradual engagement of the private sector in the management and maintenance of transport assets (Azerbaijan); linking postgraduate admission for health professionals with service in hard-to-reach areas; reexamining the institutional incentives for Department of Environment staff, based on performance indicators, and transparency and accountability; and performance-based incentives for secondary cities, towns, and union parishes that demonstrate sustained progress in planning, public financial management, revenue mobilization, and social accountability (Bangladesh). Making the state credible is an idea featured in many of these reports. Thus, the Azerbaijan SCD calls for strengthened accountability, transparency, and efficiency of public spending by instituting credible fiscal rules, particularly regarding oil revenues. The Azerbaijan CPF adds improving internal and ex- ternal audit capacity and enhancing independent and credible procurement complaint mechanisms. The Uganda SCD finds that public financial management systems are weak with regard to budget credibility and budget execution controls. Improving the quality and credibility of budget planning, a priority in the CPF, will help deliver national economic objectives and support effective service deliv- ery and development projects. Political volatility is also featured in many reports. The Uganda SCD states, “focus on the northern and eastern regions will be needed for Uganda to end extreme poverty and boost shared prosperity as well as to reduce social and political tensions that can emerge from stark differences across regions” (World Bank Group 2015e, 7–8). The Colombia SCD calls for a three-step transition: “(a) away from violence toward the rule of law; (b) away from a wartime to a peacetime economy that successfully enhances basic services and opportunities; and (c) away from turning to violence in the face of political differences to accepting participatory democracy” (World Bank Group 2015b, 19). The Chad SCD states, “political stability has been maintained by forging alliances between the regime and key constituencies, such as the security forces and other powerful interest groups, rather than relying primarily on institutional devel- opment and improved public service delivery for all” (World Bank Group 2015a, 4). Contestation over land access and use is also a common theme. Thus, the Colombia SCD highlights land use as critical for the peace process, the Chad SCD discusses women’s weak land tenure, and the Uganda SCD states that “only 20 percent of Uganda’s land is registered and most of the registered land is mailo (tenure) which is unsuitable as collateral since it cannot be liquidated by banks” (xiii). Improving agricultural productivity will take “accelerating systematic land registration, developing capacity of land dispute resolution institutions” (World Bank Group 2015e, xv). The CPF has a target of increasing the percentage of rural land registered from 5 percent in 2014 to 10 percent in 2021. The Egypt SCD notes, 148 World Bank Group Country Engagement | Appendix I “Land use laws are rigid and outdated, and prohibit the formal expansion of cities into rural areas, while rent control policies and height restrictions have also limited the supply of housing” (World Bank Group 2015c, 57) and proceeds to make a number of policy recommendations. Access to information is another theme that is often discussed. The Uganda CPF states, “the country has continued policy and legislative reforms largely led by technocrats and supported by develop- ment partners. In some areas, such as disclosure of public information, this has made Uganda a regional leader” (15). It goes on to say, “The World Bank will take steps to strengthen country systems for transparency, participation and accountability” (World Bank Group 2015d, 21). The Egypt SCD calls for “increasing government transparency both by making more data publicly available and by making progress toward a more comprehensive right to information law” (World Bank Group 2015c, xiv). The Colombia SCD advocates for the “need to invest in improving communication and infor- mation media on indigenous communities, as there has been a general tendency to focus on the conflicts and challenges that they face. Participants emphasized that it is imperative to create more awareness on the positive attributes of these communities, including their culture, identity, and phi- losophy” (World Bank Group 2015b, 126). Emerging Messages In summary, most SCDs and CPFs take governance and political economy into account in both anal- ysis and program recommendations, but some refinements to the guidance are suggested: ƒƒ Governance features prominently in most reports as a key constraint, overarching pri- ority, main cross-cutting issue, or intrinsic part of sustainability. There are analyses and recommendations addressing weak incentives and capacity for reform, making the state credible, addressing political volatility, increasing access to land, and providing informa- tion to citizens. ƒƒ Governance and politics are also addressed as risk factors, rated as moderate to high in the sample countries. ƒƒ Political economy analysis is sensitive, which in one case led to removal of sections from the SCD, and in another to delay of the SCD process until the government’s concerns could be addressed. ƒƒ However, some of the latest thinking in the area, as reflected in the World Development Report 2017, could be made more prominent. For example, analysis of and recommen- dations to address weak capacity would be strengthened with deeper analysis of the bargaining among powerful interests needed for such improvements to succeed. ƒƒ Consideration should be given to updating the SCD/CPF guidance so that it incorporates these and other issues raised in the World Development Report 2017. Independent Evaluation Group | World Bank Group 149 1 The World Development Report 2017 is expected to be released in January 2017. 2 See, for example, Porter et al. 2012. 3 High-quality political economy studies cited by the SCD include Mahmud, Ahmed, and Mahajan (2008); Parnini (2006); Rudra and Sardesai (2009); and World Bank (2009). These are cited in the reference list, but not in the text. References World Bank Group. 2014a. Interim Guidelines for Systematic Country Diagnostics. Washington, DC: World Bank Group. ———. 2014b. World Bank Group Guidance for Country Partnership Framework Products. Washington, DC: World Bank Group. ———. 2015a. Chad—Priorities for Ending Poverty and Boosting Shared Prosperity: Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015b. Colombia—Systematic Country Diagnostic. Washington, DC: World Bank Group. ———. 2015c. Egypt—Promoting Poverty Reduction and Shared Prosperity: A Systematic Country Diag- nostic. Washington, DC: World Bank Group. ———. 2015d. Uganda—Country Partnership Framework for the Period FY2016–21. Washington, DC: World Bank Group. ———. 2015e. Uganda—Systematic Country Diagnostic. Boosting Inclusive Growth and Accelerating Pov- erty Reduction. Washington, DC: World Bank Group. ———. 2015e. World Bank Group Procedure: Country Engagement. Washington, DC: World Bank Group. ———. 2017. World Development Report 2017: Governance and the Law. Washington, DC: World Bank. 150 World Bank Group Country Engagement | Appendix I Appendix J. Integration of Fragility Assessments in SCDs, CPFs, and CENs This evaluation reviewed the contribution of fragility assessments to Systematic Country Diagnostics (SCDs), Country Partnership Frameworks (CPFs), and Country Engagement Notes (CENs) in seven countries. Of the 22 countries that prepared SCDs and CPFs during the evaluation period, 4 are clas- sified by the World Bank as “fragile” and have a recent fragility assessment— Chad, Colombia, Côte d’Ivoire, and Mali. Uganda, though not classified as “fragile,” had a risk and resilience assessment prepared in June 2016. Five additional countries with CENs were included in the evaluation sample— the Central African Republic, Ecuador, Fiji, Guinea-Bissau, and Turkmenistan. Of these five, two are classified as “fragile” (Central African Republic and Guinea-Bissau) and have prepared recent fragility assessments, but do not have SCDs. Five (the Central African Republic, Colombia, Côte d’Ivoire, Guinea-Bissau, and Mali) of the seven fragility assessments contain some form of recommendations. Drivers of fragility were comprehensively discussed in four of the five SCDs (Chad, Colombia, Côte d’Ivoire, and Mali) and were aligned with the discussion in the fragility assessment (table J.1). The Côte d’Ivoire and Mali SCDs explicitly referred to their respective fragility assessments and include deep discussions on the historical nature of fragility in the country, as well as the current drivers of fragility. Fragility issues were also discussed in the same four country CPFs (not covered in the Uganda CPF), though not in as much detail as in the SCDs. The fragility assessment is featured quite prominently in the Mali CPF. The two CENs—for the Central African Republic and Guinea-Bissau—do not refer to their fragility assessments, though the CEN for the Central African Republic was completed in mid- 2015 and the fragility assessment was not completed until the first quarter of 2016. There were several common key drivers of fragility across the seven countries. Political and gover- nance issues, such as weak institutional capacity, weak judicial systems, abuse of political power, insecure political power and coups, and violent rebel uprisings and mutinies, have been regular oc- currences (in the Central African Republic, Chad, Guinea-Bissau, and Mali) and are common drivers of fragility. Land, particularly disputes over the use and ownership of land, is a key driver of conflict fragility in all five countries. In Colombia, rural areas, disconnected from the national economy and political life, are rich in natural resources and represent potential rents that perpetuate armed conflict. In Côte d’Ivoire, insecure access to land exacerbates divisions between groups and is at the root of the vast majority of local conflicts. Increasing pressure on land and the inability of customary land tenure arrangements to deal with rapid change have caused massive violent outbreaks. 151 Table J.1. Integration of Fragility Assessments in SCDs, CPFs, and CENs Specific Specific reference reference Fragility in SCD to in CPF to discussed in fragility as- Fragility dis- fragility as- Key drivers of Country SCD sessment cussed in CPF sessment fragility Côte d’Ivoire Yes Yes Yes No Injustice, security, violence, land issues, high levels of unemployed youth. Chad Yes No (though Yes No Poor fragility governance, land assessment conflicts, warfare was done in and insecurity, 2012) regional insecurities, severe environmental factors. Mali Yes Yes Yes Yes Insecurity and conflict, weak governance, youth, land and environmental issues. Uganda No, and SCD No No. Land issues No The assessment done (December discussed in the discusses risk 2015) before general context and resilience. fragility section but not Issues like assessment as a driver of political and (called risk fragility. Security regional and resilience is mentioned a insecurity, land, assessment) few times related and historical done in June to dealing with conflict are 2016. regional issues/ listed. Uganda’s neighboring “fragility” stems countries. primarily from issues of neighboring countries. Colombia Yes No Yes No Subnational fragility (such as land issues, isolated areas, and others) and organized crime from arms, drugs, and money. Table J.1 continues on the next page. 152 World Bank Group Country Engagement | Appendix J Specific Specific reference reference Fragility in SCD to in CPF to discussed in fragility as- Fragility dis- fragility as- Key drivers of Country SCD sessment cussed in CPF sessment fragility Central African Not applicable Not CEN done in Joint fragility Violent political Republic applicable 2015 includes assessment history and weak four annexes (World Bank institutions, on fragility: Group, United regional system conflict and Nations, of conflict, fragility analysis; Department violent clashes at armed groups; for community level, displacement; International locally armed pastoralism and Development, groups, conflict. African Development World Bank) done in April 2016 Guinea-Bissau Not applicable Not CEN done in Fragility Political applicable March 2015. assessment transformation, Refers to fragility done in June military assessment done 2015 after interference in 2012/13. CEN was (assassinations finalized. and coups), weak justice sector, weak institutions. Regional instability is also a key driver of fragility in these countries. Chad has suffered considerable upheaval spilling across its borders in the form of displaced persons, and it has also been drawn into regional conflicts over trafficking (in humans, firearms, food, or drugs) and recruitment into terror groups—with the Central African Republic, Sudan, and Libya. Uganda faces a number of external threats to stability (terrorism; border disputes; and refugees from South Sudan, Somalia, Democratic Republic of Congo, and other areas). Mali faces the same regional insecurity with its neighbors (Nige- ria, Niger, Cameroon, Chad, and the spread of terrorist organizations, among others). Issues related to youth were also considered drivers of fragility. Youth are easily mobilized and manipulated. In Mali, youth unemployment has motivated young people to join the networks of organized crime as the only way to move from rags to riches. In Côte d’Ivoire, high levels of unem- ployed youth are at the center of the conflict, and some of the disenfranchised youth have proven useful to the main actors in the crisis, who have found them easily mobilized and manipulated for their political gain. Independent Evaluation Group | World Bank Group 153 The World Bank 1818 H Street NW Washington, DC 20433