Approach Paper Evaluation of the World Bank Group Engagement on Strengthening Subnational Governments May 31, 2018 Background and Context 1. Decentralization has been at the center of the public policy reform agenda all over the world - a process driven by both economic and political factors. Long-run structural transformations – mainly economic development and urbanization – have been associated with increasing demand for the provision of public services at the local level, especially in rapidly growing urban centers.1 This has often been translated into an assignment of public functions from national to subnational governments (SNGs), a process which, together with the transfer of the respective structures, systems, resources and arrangements, amounts to what is generally understood as decentralization. The pace and specific form taken by decentralization have been driven by powerful and country-specific political dynamics, including for example democratization processes and efforts to address regional separatist movements.2 In this context, the real question today for development practitioners is not whether to decentralize – such decisions are taken by the countries based on multiple motivations, including political, social, and economic – but rather how to support decentralization processes strategically to make the most of them, in partnership with national and subnational governments. 2. While decentralization reforms have often been expected to improve provision of public services through better resource allocation and enhanced accountability, the empirical evidence suggests that the results have been mixed. The view that decentralization has the potential for improving public service delivery and infrastructure has been based on two premises, namely that SNGs are more aware of local conditions, and that they are closer and therefore more directly accountable to local populations as compared to central governments. However, the evidence suggests that these premises may not always hold. Indeed, SNGs are not immune to the governance risks, such as elite capture, clientelism, and capacity constraints, that have plagued many national governments and often motivated the need for decentralization3. In addition to governance challenges, SNGs have in many cases suffered from other practical challenges to deliver on their responsibilities, such as unclear functional assignments, and insufficient human and financial resources, among others.4 3. With decentralization, SNGs are playing a significant role in public service provision. Many SNGs are charged with the primary responsibility for providing many of the public services that contribute to the well-being of the poor and their capacity to lift themselves out of poverty. 1 See White and Smoke (2005). 2 See Faguet and Pöshchl (2015). 3 See Campos and Hellman (2005). 4 See Dyer and Rose (2005), Smoke (2013, 2015a). 1 The specific “division of labor” between central and subnational governments differs from one country to another. Globally, in 2013 SNGs accounted for about one-quarter of government spending – corresponding to 9 percent of GDP – and for almost 40 percent of public investment – equivalent to 1.5 percent of global GDP (OECD-UCLG 2016). Traditional sectors where the bulk of SNG spending is focused include education, health, water supply and sanitation, solid waste management, maintenance of local roads and provision of public transportation. The share of SNGs in total government spending at the national level tends to rise with the level of development, as measured by income per capita, suggesting a growing importance of the SNG strengthening agenda as countries move up the income ladder (OECD-UCLG 2016). 4. Given their important public service provision responsibilities, SNGs are expected to play a critical role for the achievement of the Sustainable Development Goals (SDGs). The SDG document (UNDP 2015) recognizes the role of local governments, emphasizing their contribution to such areas as economic planning, employment generation, and environmental management, among others. Moreover, beyond the role of SNGs in the achievement of individual SDGs, the development community has increasingly recognized that meaningful progress in the SDG agenda will require full ownership at the local level, both by SNGs and by local civil society organizations (UNDP and World Bank, 2016). Based on this view, international SNG networks have been developing approaches for the “localization” of SDGs, to contribute to the alignment of work programs as well as to facilitate knowledge exchange and local monitoring and reporting (UCLG 2017). 5. For SNGs to meet expectations on their service delivery and infrastructure responsibilities, central and subnational governments need to address important design and implementation challenges, with possible support from development partners. In this regard, a preliminary review of the literature5 (see Attachment 1) suggests the following key types of challenges that need to be addressed for SNGs to contribute to improved development outcomes: 5 Adams (2016), Ahmad E. and Brosio (2009), Ahmad J. et al. (2005), Azfar et al. (1999), Boex and Yilmaz (2010), Bruyninckx et al. (2012), Cheema and Rondinelli (2007), Dreher (2006), Eaton et al. (2010), Kaiser (2005), Litvack et al. (1998), Maaroof et al. (2008), Martinez-Vasquez et al. (2017), Robinson (2007), Saavedra (2010), Smoke (2007, 2013, 2015 (a), (b), (c), (d)), Sow and Razafinahefa (2015), Torrisi (2011), Webb and Giuigale (2000), White and Smoke (2005), World Bank (2000, 2002 (b), 2004). 2 • Clear and consistent intergovernmental frameworks, which may vary across countries, and define: the principles and practices for sharing public powers and functions among levels of government, along with the structures, systems, resources, and procedures that support their implementation to meet public sector goals. These frameworks can include administrative, fiscal and political dimensions and specify the relationships among and within different levels of government (e.g. interjurisdictional cooperation in metropolitan areas); • Strong SNGs’ economic and financial management policies and institutions: adequate systems and capacity for managing public revenues, expenditures, assets and debt, as well as good public financial management capacity more broadly; • Strong SNGs’ policies, institutions and incentives for good governance, transparency, and accountability: adequate human resources and civil service management, sound transparency, accountability and anti-corruption arrangements. The Role of the WBG 6. In the 2015 From Billions to Trillions: Transforming Development Finance Post-2015 paper, multilateral banks and the IMF acknowledged the increasing role of SNGs in delivering public goods and services. The paper includes a commitment to strengthen support and coordination in a number of focus areas including, inter alia: (i) strengthening the capacity of SNGs to raise their own revenues, to manage expenditures (including investment programming) and service delivery, and to borrow and manage debt prudently; and (ii) developing inter- government fiscal transfer arrangements that take into account the investment needs of SNGs, equalize fiscal capacity and expenditure needs across levels of government, and clearly delineate spending responsibilities between the different layers of government. Signatories also committed to expand policy guidance and technical assistance to increase the amounts, quality and efficiency of resource mobilization and public expenditure by national, subnational and municipal entities. 7. The WB has been expanding its engagement on the intergovernmental relations agenda as well as its direct support to SNGs since the early 1990s, although direct lending to SNGs declined in the last 8-10 years (Figure 1). This trend has been manifested in increased levels of WB lending to and partnership with SNGs. While a significant part of WB support to SNGs has focused on public finance reform, the WBG has also supported investments and policy and institutional strengthening across other sectors, ranging from human development to infrastructure and natural resource management. Although the volume of subnational lending has fluctuated over time, WBG engagements in many middle-income countries (MICs) have been increasingly shifting to supporting SNGs (as recognized in the 2013 WBG Strategy). Effectively responding to growing demand for WBG support at the subnational level across a broad range of sectors and country contexts will require a strategic approach informed by evidence on the effectiveness of past engagements. This evaluation aims to contribute to building such knowledge base. Ultimately, the goal is that of informing WBG strategies and approaches for helping client countries turn the promise of decentralization into better development results. 8. While the WBG does not have an explicit strategic approach for supporting intergovernmental relations and SNGs, several strategy papers and flagship reports have recognized the importance of this agenda. The 1999/2000 World Development Report (WDR) 3 on Entering the 21st Century highlights the unprecedent opportunities for growth and development opened by localization but also the new or greater challenges in terms of economic and political stability. Localization is praised for raising participation and voice and for its potential for a more responsive and efficient local governance. However, if poorly designed and/or poorly implemented, decentralization can result in overburdened local governments without the resources or the capacity to fulfil their basic responsibilities of providing local infrastructure and services. In some large countries, where SNGs have a significant role, it can also threaten macroeconomic stability if local governments, borrowing heavily and spending unwisely, need to be bailed by national government. The 2000 Public Sector and Governance Strategy portrayed decentralization as one of eight key elements of public sector reform. The Strategy positioned the Bank “ to be considered one of a very few leading authorities worldwide in several core areas where we have a track record or a comparative advantage, including (a) the role of the public sector, (b) the broad structure of government (including decentralization and intergovernmental fiscal relations).” The 2004 WDR on Making Services Work for the Poor emphasized the importance of fostering subnational capacity for service provision. The 2007 Governance and Anticorruption Strategy highlighted the importance of working with local governments. The Strategy recognized that in many countries, governance and corruption challenges have shifted to the local level. It placed a special emphasis on capacity building in countries with strong commitment to governance improvements, but with severe shortfalls in skills and organizational capabilities. More recently, the 2012 paper on The World Bank’s Approach to Public Sector Management 2011-2020: Better Results from Public Sector Institutions does not identify decentralization and subnational public sector reforms as a specific area of intervention. 9. Several WBG sector strategies adopted over the last 15 years considered local government reforms among key sectoral trends and suggested specific interventions to maximize their impact. For example, the 2009 Urban and Local Government Strategy prioritized focusing on the core elements of the city system (city management, infrastructure service delivery, finance, and governance) among business lines considered critical for cities and local governments. The strategy noted that capacity building to improve local government management was essential and that it needed to go beyond the provision of training to include reforms that change the “rules of the game,” using incentives and rule-based policy frameworks. These reform measures could include financial management, accountability, local revenue collection, economic performance, and a host of other areas. The 2003 Water and Sanitation Strategy emphasized the importance of broader decentralization and local government reform initiatives to improve accountability, ensure sustainability and increase the flow of finance to the sector. The 2012 World Bank Group Education Strategy 2020 factored in the global trend of decentralizing education systems. 10. The World Bank has provided support for intergovernmental frameworks and subnational governments strengthening through both lending and Analytical and Advisory Services (ASA). The evaluation team conducted an initial portfolio review (FY2008-2017) covering both decentralization and subnational government activities.6 Based on this review, the 6 The lending and ASA portfolio relevant to the evaluation was identified based on the thematic and sector codes assigned by task managers. Note that the “decentralization” theme code has been discontinued as part of the FY2016 modernization of the WB operational coding system and the scope of the “subnational government” sector code has been broadened (from “subnational government administration”) to cover activities previously coded for decentralization. To overcome this discontinuity and lack of clarity of the operational coding system in the subject 4 relevant portfolio includes 563 lending projects for a total commitment of US$71 billion, or 18 percent of the total World Bank lending. Lending has been lower than in previous decades (Figure 1): it averaged 33 percent of total WB commitments in the 1990-2007 period. Only about 25 percent of lending is coded as support to decentralization, with support to SNGs amounting to 90 percent of total commitments – about 15 percent of lending is coded as contributing to both decentralization and SNGs during the 2008-2017 period. Close to 80 percent of commitments were delivered by 5 Global Practices (GPs): Social, Urban, Rural and Resilience (34 percent); Macroeconomics and Fiscal Management (19 percent)7; Governance (11 percent); Water (8 percent); and Transport & ICT (7 percent).8 Advisory Services and Analytics (ASA) portfolio included 1,194 products, of which Technical Assistance (TA) accounted for 56 percent. 11. IFC has supported subnational government entities with financial and advisory services through various initiatives. IFC’s support was initiated through the Subnational Finance Program (2003) – a joint World Bank/IFC program that provided subnational financing without the need for a sovereign guarantee. The program aimed to support investments across infrastructure sectors and public services, working through subnational governments, nationally- owned SOEs, financial intermediaries, and public private partnerships. The Subnational Finance Program was mainstreamed in IFC’s operations in 20099. More recently, IFC has begun to roll out a global Cities Initiative, which aims to make cities more competitive by: (i) strengthening institutions and regulations; (ii) improving critical infrastructure and environmental sustainability; (iii) fostering skills and innovation; and (iv) expanding access to finance.10 The initiative seeks to combine financial and advisory support through support to both subnational governments and private investors. IFC has supported subnational governments and SOEs through 46 investments totaling $2.1 billion (FY08-17) mainly for infrastructure projects – about 2 percent of IFC’s total commitments during this period. IFC’s financial support was concentrated in transport, ports, the power sector and water/wastewater. Most of the financing occurred during FY09-14 and was in non-IDA countries. 12. In addition to the financial and advisory support linked to subnational finance projects, IFC has also been active through investment climate advisory for subnational government entities, and has deployed its Public Private Partnerships transaction advisory to support the introduction of private sector participation in the delivery of infrastructure services through PPPs. The evaluation will focus on the main areas of IFC’s support described above: subnational and municipal finance and linked advisory services, investment climate advisory, and PPP advisory. It will exclude from of this evaluation, the team is conducting a systematic identification of the Bank portfolio relying on criteria beyond sector and thematic codes. 7 This GP is being renamed Macroeconomics, Trade, and Investment (MTI). 8 These shares are based on the operations’ lead GP. 9 In the context of 2012 IFC reorganization, subnational finance is housed in IFC’s global infrastructure department (municipal and environmental infrastructure unit), whereas support to SOEs was mainstreamed within the sectoral units of the infrastructure group. 10 IFC Cities Initiative. Factsheet. Accessed at: https://www.ifc.org/wps/wcm/connect/8bfdc50048b5bbfea84dfe6c57b0ebf6/Cities_factsheet_IFC.pdf?MOD=AJPE RES 5 its scope IFC’s support to nationally-owned SOEs and to subnational entities through financial intermediaries. 13. Since the late 2000s, MIGA has issued credit enhancement products to sub-sovereign entities. In 2009, MIGA’s Board of Directors approved a new cover for non-commercial risks leading to the introduction of a new product line for MIGA: the non-honoring of sovereign financial obligations (NHSFO) coverage. Through this new product, MIGA can provide credit enhancements to assist credit-worthy sovereigns that aim to reduce their cost of financing when borrowing from cross-border lenders. This product can also be applied to credit worthy municipalities and sub-sovereigns without the requirement of a sovereign guarantee. Since its creation, NHSFO cover has been issued for several infrastructure projects at subnational levels e.g. provincial or municipal governments including Istanbul Metro, Izmir Tramway (Turkey), Panama City Metro, and Sao Paulo Sustainable Transportation (Brazil). Figure 1. World Bank Lending, FY1990-2017 12.0 140 10.0 120 100 No. of projects 8.0 US$ billions 80 6.0 60 4.0 40 2.0 20 - 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Subnational (BH) Decentralization (26) Commitments (US$ billions) Commitments (US$ billions) Subnational (BH) No. of Projects Decentralization (26) No. of Projects Source: WB Business Intelligence. Subnational commitments identified by the subnational government administration sector code (BH); decentralization commitments identified through the respective theme code (26). Evidence from Previous IEG Evaluations 14. The 2008 IEG evaluation of Decentralization in Client Countries (World Bank 2008b) identified some areas of success, mostly in fiscal decentralization, and highlighted several shortcomings in Bank support that had at the time hampered its effectiveness. The most successful parts of the World Bank decentralization program included strengthening legal frameworks for intergovernmental fiscal relations, improving public financial management at the local level, and helping central governments establish transparent fiscal transfer systems. The Bank, however, was much less successful in helping countries enhance subnational revenue 6 mobilization, clarifying responsibilities of different government levels, and strengthening citizen oversight of SNG operations. The evaluation noted that while in a few cases Bank support had been built on an explicitly cross-cutting approach, in most of cases it had taken a sector-specific route. In the case of education, the evaluation found that sector-level decentralization efforts were less effective when not embedded in country level cross-cutting support for improving decentralization frameworks. 15. The 2008 IEG evaluation offered recommendations that focused on the importance of country ownership, result-orientation, strong integrated analytical underpinnings, and cross-sectoral collaboration. The evaluation offered four recommendations: (i) ensure that Bank support is underpinned by genuine client commitment to decentralized service delivery; (ii) encourage the adoption of a more results-oriented approach to decentralization by helping develop in-country and Bank capacity for monitoring and evaluation that focuses on local outcomes rather than just the process of decentralization; (iii) ensure Bank support at the country level is based on a clear analytical framework grounded on an integrative understanding of the various factors at different level of government and across sectors affected by decentralization, and accompanied by support to develop local government capacity; and (iv) strengthen institutional arrangements within the Bank to ensure that an integrative view underpins Bank interventions, particularly those based on sector-specific entry points. As per the last Management Action Record (MAR) for this evaluation, released in 2012 (IEG 2012), it was only the second recommendation on adopting a more results-based approach that was rated Substantial for the level of adoption and Complete for status of implementation. On strengthening WBG institutional coordination, the annual MARs presented various attempts and initiatives with limited evidence of actual follow-up. The proposed evaluation will make an effort to assess how much progress was made on these recommendations. 16. The 2010 IEG Study on “World Bank Engagement at the State Level: The Cases of Brazil, India, Nigeria, and Russia (World Bank 2010b) highlighted the challenges of combining support to leading and lagging states and the importance of strengthening SNGs’ public finance capacity. Two tendencies—often in tension—were featured in most approaches for the selection of states for direct engagement. One was to support better-performing, reformist states (“lead or focus states” approach), while the other was to support the poorest states as a more direct route to reducing poverty. The initial area of engagement was typically fiscal reform, where the Bank for the most part helped enhance the capacity of state governments in the area of public finance. For the four country cases reviewed, the study confirmed the desirability of continued selective lending in a few focus states, but suggested giving greater weight to the needs of the poorest states. It also supported a continued focus on improving public finance at the state level, irrespective of whether WBG engagement is confined to this area or it serves as an entry point for broader engagement. Purpose, Objectives, and Audience 17. The main objective of the proposed evaluation is to assess the role and contributions of the WBG to the strengthening of subnational governments (SSNG)’ ability to fulfill their public service provision responsibilities. The evaluation will focus on WBG support to core government policies and institutions necessary for SNG to deliver services and infrastructure (see below and Figure 2). The evaluation aims at distilling lessons from past WBG engagement in these areas with a view to inform WBG strategic approaches in SSNG support. The evaluation is 7 expected to make specific recommendations that could feed into relevant country strategies and project design. This evaluation is of strategic relevance from the perspective of implementing the Maximizing Finance for Development (MFD) approach, which called for enhancing financial leverage of the WBG. In addition to raising domestic resource mobilization, Bank and IFC support to SNGs, has been designed to create the conditions for increased private development finance at the subnational level. The potential audience for this evaluation includes WBG management, WBG task teams, clients (at national and subnational levels), development partners and practitioners. 18. The evaluation will address existing evidence gaps and contribute to both learning from past WBG experience and accountability on the results achieved through the WBG’s support to decentralization and SNGs. The 2008 IEG evaluation of Decentralization in Client Countries focused on the extent to which the World Bank effectively supported the improvement of decentralization frameworks. It covered the World Bank portfolio of lending and analytical activities with decentralization components. The present evaluation will have a different, although related, focus. Considering the large size of the portfolio of activities to be covered, the evaluation is expected to fill an important accountability gap in terms of providing evidence on the results achieved through the large portfolio of WBG SSNG activities. At the same time, the evaluation will place an important emphasis on learning from experience to inform future SSNG engagements. Conceptual Framework 19. As the WBG does not have an explicit strategy for support to decentralization and SNGs, the proposed intervention logic (IL) is based on a preliminary literature review of decentralization (fiscal and administrative), intergovernmental systems, local governance, and public sector management and public service provision at the subnational level.11 In addition to the conventional technical considerations, the literature points to the critical importance of political economy considerations - national, intergovernmental and local - in shaping how decentralization emerges and evolves. The IL also considers the conclusions derived from a preliminary review of WBG analytical work, country strategies, lending portfolio, and ASA covering decentralization and engagement with SNGs. CONCEPTUAL UNDERPINNINGS 20. Effective multilevel governance structures and public-sector capabilities at all levels of government are essential to foster effective decentralized public service provision by subnational governments. Many WBG and external publications have pointed to a critical link between institutional capacity and the achievement of development results, and highlighted that subnational authorities can be efficient providers and regulators of local services provided the right institutional incentives are in place and when there is clarity about who does what. The 2002 WDR on Building Institutions for Markets underscored that to make institutional changes happen, improvement of knowledge, skills, and ability of public organizations is critical. The 2004 WDR 11 In addition to references listed in footnote 5, see Ahamad et al. (2014), Ahmad and Brosio (2015), Bahl (1999), Canuto and Liu (2013), Charbit (2011), Da Cruz and Gary (2015), Schoburg (2016), Shah (2006), Smoke (2016), United Cities and Local Governments (2010), and UNDP (2009). 8 on Making Services Work for the Poor emphasized the importance of building partnerships with central governments in fostering subnational capacity and described decentralization as one of the areas with the biggest payoffs for improving service delivery. Other areas emphasized by the 2004 WDR as also having large payoffs (in addition to decentralization) include improvements in the allocation of public expenditures – “spending wisely and predictably in line with priorities and coordinated across sectors” – as well as the development of administrative capacity, the curbing of corruption, and learning from success and failure. 21. Strengthening both upstream and downstream government functions (at all government levels) is important to achieve better development outcomes (World Bank, 2000). Upstream activities include: (i) making and implementing economic policy, including maintaining discipline in fiscal and monetary aggregates, effectively setting priorities among competing demands for resources, and mobilizing revenues; and (ii) ensuring accountability for the use of public resources and public regulatory power. Downstream activities are all those directly required for front line service delivery. The importance of strengthening both upstream core ministries and central agencies, and downstream bodies – including sector ministries, and non-executive state institutions – was again highlighted in the more recent Paper (World Bank 2012b) The World Bank Approach to Public Sector Management 2011-2020: Better Results from Public Sector Institutions. 22. SNGs cannot play a developmental role unless they are empowered to act autonomously. A good intergovernmental framework is expected to include formal and clear assignment of functions and revenues as well as systems and processes to support implementation. Clarity on their functions enables SNGs to act more confidently and allows central governments and constituents to hold them accountable. Autonomy needs to be disciplined by transparency and accountability. This includes downward accountability to citizens through elections and non- electoral means, and upward accountability to ensure, for example, compliance with basic financial management procedures and to facilitate responsiveness to legitimate national goals and standards. Although such regulatory/oversight functions are essential, they can hinder SNG performance and downward accountability if they are too inflexible or inconsistently applied. A main concern in decentralization policy is establishing an appropriate balance between upward and downward accountability, which can evolve as SNGs prove their ability to perform well. 23. The entire process of decentralization is conditioned by political economy factors-- socioeconomic characteristics of the country, national and intergovernmental political and bureaucratic dynamics, and subnational political decisions, among others. Such considerations influence the motivation for and form of decentralization, specific design features, and the extent to which decentralization is implemented as designed. Equally important, decentralization reform efforts are often focused heavily on designing systems that meet normative standards, but such systems are often a long way from what currently exists. Moving too quickly with decentralization can overwhelm SNGs and test the tolerance of central actors reluctant to lose power and resources, while moving too slowly can lead to stalled and unproductive reforms. Thus, developing an appropriate and pragmatic strategy for implementing decentralization--taking into account political economy as well as resource and capacity constraints--is increasingly recognized as being as important as good design. 9 DEFINITIONS: DECENTRALIZATION AND SUBNATIONAL GOVERNMENTS 24. Decentralization involves the transfer of authority and responsibility for public functions from the central government to intermediate and local governments or quasi- independent government organizations12. Decentralization can take various forms, and can be implemented through devolution, delegation or deconcentration. This process implies a change in the relationship between the central government and SNGs. It requires the development of a new set of “rules of the game” and the provision of adequate resources for SNGs to fulfill their new roles. These new rules cover the political space, the allocation of fiscal resources, mechanisms to ensure accountability of government actions and the participation of citizens in the organization of public service provision. 25. For the purposes of this evaluation, the term “subnational government”, or SNG, is defined as an institutional unit whose fiscal, legislative, and executive authority extends over a geographical area, below the central government level, distinguished for administrative and political purposes.13 These include various forms of SNGs, such as states, provinces, departments, metropolitan cities, municipalities, communes - depending on each country subnational government structure. It excludes the lowest tier of local governance systems (community-level). Decentralization can also take various forms (see Attachment 2). INTERVENTION LOGIC 26. Overall, WBG engagement on SSNG has focused on addressing the enabling factors for better service provision, based on two inter-related pillars. These pillars include: (i) helping the countries to establish properly functioning intergovernmental frameworks and systems guiding relationships between different layers of government - in particular, various aspects of intergovernmental finance; and (ii) helping to provide services through direct support to strengthening subnational governments’ policies and institutions14. 27. The intervention logic underpinning the evaluation is shown in Figure 2, where WBG decentralization and SNG-related activities are classified in three thematic areas that broadly correspond to the main types of challenges for realizing decentralization’s potential for improving service delivery: (i) Intergovernmental frameworks (IGF); (ii) Economic and financial management (EFM); and (iii) Governance, transparency, and accountability (GTA). The objectives of the WBG support in each of these areas are summarized below. 12 A similar definition is used in the WB’s Public Sector Governance webpage on decentralization (http://www1.worldbank.org/publicsector/decentralization/what.htm) and the 2008 IEG evaluation of Decentralization in Client Countries. 13 Adapted from Government Finance Statistics Manual (IMF, 2014). 14 Strengthening SNGs’ policies and institutions is often referred to as SNG’s capacity building. In this context, capacity refers to: (a) the skills public officials need to deliver on various mandates; (b) the resources (capital and financial) they require to support their efforts; and (c) the systems and mechanisms that enable large numbers of bureaucrats to work together effectively. The same considerations hold for members and staff of civil society organizations (Ahmad J. et al. 2005). 10 • Intergovernmental frameworks: Improve intergovernmental frameworks including clarity of assignments across levels of government, alignment of resource allocation with assigned responsibilities, inter-government fiscal transfers, fiscal rules for SNGs. • Economic and financial management: Improve upstream government functions of SNGs including strategic planning, economic and fiscal management, debt policy and debt management, access to finance, revenue mobilization, public expenditure management, public investment management, and public financial management. • Governance, transparency, and accountability: Improve human resource management, transparency (information, monitoring and evaluation), civil society and private sector participation in SN governance processes, and accountability mechanisms. 11 Figure 2. Intervention Logic WBG ACTIVITIES / AREAS OF DIRECT OUTCOMES OUTCOMES INPUT INTERVENTION Intergovernmental Frameworks Intergovernmental Frameworks • Diagnostic studies • Decentralization policies (allocation of • Improved decentralization policy frameworks and introduced clear functions assignment of responsibilities for public services across various • Intergovernmental fiscal relations levels of government, supported by adequate resource allocation • SNG capacity needs assessments • Operationalized inter-government fiscal transfers that are LINKS TO TWIN GOALS AND SDGS • Political economy analysis transparent, predictable, and fiscally sustainable • Improved fiscal rules for SNGs are set in a responsible and effective manner • Established and effective public accountability mechanisms of Economic and Financial Management government workers to elected officials at SN level Lending • Policy planning and budgeting • Improved public service • Regulatory diagnostics (ICA, Doing delivery Business etc.) Economic and Financial Management • Regulatory capacity • Macro-fiscal policy reform • Improved strategic planning • Improved equity • Access to financing • Improved economic management Technical • Medium-term fiscal planning • Established and prudent fiscal management and borrowing • Increased investment, Assistance, • Monitoring and rationalization of SN policies growth, employment Capacity public expenditures, PEFA; PERs • Improved debt policy and debt management Building • Performance-based budgeting • Improved revenue mobilization • Improved access to finance • Improved governance • Public investment planning, design and • Established and operationalized public financial management execution systems (budget planning, preparation and execution incl. • Public procurement reform public investment management and public procurement, • Physical upgrade of FM systems (IFMIS) accounting and reporting, and financial accountability and transparency) Analytical Governance, Transparency, and Support Governance, Transparency, and Accountability Accountability • Improved human resource management capacity • Public administration reform (civil service • Improved transparency (information, monitoring and rules etc.) evaluation) • Public accountability frameworks • Improved civil society and private sector participation in SN • Participatory approaches (budgeting, governance processes and accountability mechanisms policy formulation, access to information, etc.) • Anti-corruption frameworks Domestic Context Global Context: Level of development, demographic and social Global Partners: Global economic cycles, characteristics, urbanization, political IMF, Regional Development Banks, OECD, UN agencies, commodity prices, financial economy, aid dependence, etc. Bilaterals, etc. markets etc. 12 Evaluation Scope and Evaluation Questions 28. The evaluation will review the modalities of the WBG engagement in support of the SSNG agenda, including the choice of instruments, the synergies between them and the design of the respective interventions. The evaluation will consider the full range of potential WBG products and interventions – the “inputs” and “activities” depicted in Figure 2 – that have been used in support of strengthening SNGs. It will cover only World Bank and IFC activities in support of SSNG. Evaluation of MIGA activities in the SSNG area will be covered by specific ongoing evaluation work on MIGA’s Non-Honoring of Sovereign Financial Obligations. The evaluation will thus cover World Bank ASA, IFC Advisory Services, as well as WB and IFC lending and investments. The evaluation will assess the instruments used by the Bank and IFC in support of the SSNG agenda and the possible synergies between them. It will also assess the relevance of design of the respective Bank and IFC interventions, thereby considering their consistency with client needs, evidence from literature and corporate strategic priorities. 29. The main evaluation question is the following: How and to what extent has the Bank and IFC support to subnational governments contributed to strengthening their ability to deliver public services and infrastructure? This main question will be addressed through three main sub-questions: 1. What has been the nature and volume of Bank and IFC support to strengthening SNGs? 2. How relevant and coherent has been the Bank and IFC support to strengthening SNGs? 3. How and to what extent has the Bank and IFC support to subnational governments contributed to strengthening their ability to deliver services, through: a. Improving their economic and financial management policies and institutions; and b. Improving their policies and institutions systems for good governance, transparency and accountability? 30. The evaluation will cover support extended to the central government – as relevant to strengthening SNGs, in particular on developing effective intergovernmental frameworks– as well as the relevant second and third tiers of government (states, provinces, departments, major metropolitan areas, municipalities, communes, etc.) – in relation to the strengthening of SNG capabilities for delivering public services and infrastructure. The focus on second or third tiers of government depends on the subnational government structure of each country and will be explained in detail when selecting case study countries (see below). Targeting primarily the second and third tiers of government will help sharpen the focus of the evaluation and avoid possible overlaps with other ongoing or recent evaluations. This excludes the lowest tiers of local governance systems (e.g. community-level), and hence will also exclude community-driven development (CDD) type of engagements except when these engagements include support to SNGs. In the area of SNGs’ strengthening, the evaluation will focus on core 13 upstream policies and institutions in EFM and GTA. These exclude sector specific policies and institutions which would be better treated as part of sector evaluations. 31. The evaluation will coordinate with other thematically related ongoing IEG evaluations, in particular, the Urban Resilience evaluation and Engaging Citizens for Better Development Results to avoid possible duplication and ensure synergies. The evaluation will also ensure complementarity with the IEG Institutional Capacity Strengthening evaluation planned for FY20 which is expected to have a much broader scope. The evaluation will cover the period FY2008-2017. 32. The evaluation will emphasize the gathering of evidence on the achievement of direct outcomes15 at the subnational government level and treat intergovernmental frameworks as an enabling factor16. Direct outcomes are critical steps toward the achievement of the expected final outcomes of WBG SSNG engagements. They are fundamental for enabling SNGs to effectively fulfill their public service functions. Moreover, causal relationships between intervention activities/outputs and direct outcomes are easier to establish within the life cycle of Bank and IFC projects than for final outcomes. 33. The evaluation will assess the achievement of results in the two direct outcome areas at subnational government level identified in Figure 2 -- EFM and GTA. In addition to identifying the nature and relevance of the Bank and IFC activities in each direct outcome area, the evaluation will seek to produce evidence on the extent to which key expected outputs and direct outcomes were achieved and the evidence on the Bank and IFC’s contribution to those results. Except for cases where the WBG has been supporting the strengthening of SNGs for an extended period, the evaluation will not cover the causal links from direct outcomes to final outcomes as these final outcomes depend on many factors beyond EFM and GTA policies and institutions. Existing evidence including WBG analytical work (See above) supports the conclusion that strong EFM and GTA policies and institutions are important determinants of better development outcomes, but they are not sufficient. In this regard, the evaluation will review and incorporate several contextual factors affecting the effectiveness of WBG engagement with SNGs and modalities thereof. Those will include political economy factors and the impact of global economic and financial context. The evaluation will be based on stated objectives as defined in WBG documents and measurement of success will use WBG indicators and targets provided in these documents when relevant and adequate. Evaluation Design 34. The evaluation will adopt a multi-level, multi-site evaluation design. The multilevel dimension of the evaluation refers to the different data collection and analysis activities conducted at the global (portfolio), country, SNG, and intervention levels. The multi-site aspect concerns the 15 The causal pathways from activities/outputs to outcomes are much more refined. For example, technical assistance may affect knowledge levels of civil servants (outcome) which in turn affects their attitudes (outcome), which then influences their behavior (outcome), e.g. in terms of the practice of policy formulation or program design. To keep things simple (and comprehensive), at the level of the overall intervention logic the former types of outcomes are all part of what we refer to as direct outcomes. 16 Or constraining factor depending on the nature of these intergovernmental frameworks. 14 purposive selection of countries, SNGs, and interventions for more in-depth data collection and analysis. The evaluation will be organized around four levels of analysis (Figure 3). The evaluation methods have been designed to support data collection needs linked to the main evaluation questions at the four levels of analysis (see details in Attachment 3, Table 1). OVERVIEW OF MAIN METHODS 35. Evaluative analysis will be undertaken at four levels, using a nested approach; a broad portfolio lens supplemented by case-based analysis at the levels of selected countries, selected SNGs in selected countries, and selected interventions targeting selected SNGs. Figure 3. Multi-level Case-based Evaluation Design Global Portfolio Level Selected Country Level Subnational Government Level Selected Intervention Level 36. The methods include structured-literature reviews, portfolio review analysis, and case studies (see Attachment 3 for details on methods, including building blocks and sources of evidence by evaluation question). Given the complex multifaceted nature of the topic, the evaluation will cover: • Global portfolio level. The evaluation will identify the global portfolio, conduct portfolio descriptive analysis, a desk review of corporate strategies, and semi- structured interviews with Bank and IFC Management and operations staff. Depending on the availability of evidence additional avenues for evaluative analysis using portfolio data will be explored. • Selected countries level. The evaluation will conduct portfolio analysis, desk review of country strategic and project documents and semi-structured interviews with Bank, IFC staff, government officials and other relevant stakeholders (in selected countries). • Selected SNGs (in selected countries) level. The evaluation will conduct portfolio analysis, desk review of any relevant Bank/IFC strategy documents and semi- structured interviews with Bank and IFC staff, SNGs and other relevant stakeholders (in selected countries, for selected SNGs). 15 • Selected interventions in selected SNGs. The evaluation will conduct in-depth portfolio analysis of relevant project documents (including IEG evaluations) and semi-structured interviews with stakeholders. 37. Cross-case comparative analysis of key issues across countries, SNGs and specific interventions will be conducted on the basis of structured data collection templates. Taking into consideration the highly contextualized nature of SNGs’ ability to deliver services, the evaluation will rely on two principles to enhance the external validity findings. First, the selection of countries, SNGs and interventions will be based on informed purposive sampling strategies (see below). While this does not guarantee the generalizability of findings (and can generate its own biases), in combination with a certain level of convergence of findings across countries/SNGs an informed claim to generalizability of findings can be developed. 38. To support evaluative analysis at the different levels, the evaluation team will also take advantage of other completed IEG evaluations (thematic, corporate, country, and project level -- see Attachment 4 a preliminary list of relevant IEG evaluation work). In addition, structured literature reviews of existing evidence on strengthening SNGs’ policies and institutions in economic and financial management (EFM) and in governance, transparency and accountability (GTA) as well as a background papers on the evolution of Bank and IFC SSNG strategies and approaches over the evaluation period will inform the evaluation. SAMPLING STRATEGY 39. To support the generalizability of the evaluation’s findings as well as to manage inevitable trade-offs between the depth and breadth of the evaluation, the team will follow explicit criteria for the selection of cases at the country, SNG, and intervention levels of analysis. • Selection of countries for case-based analysis. Selection of countries will be based on criteria set to identify a group of countries that will provide both reasonable diversity and common patterns within Bank and IFC areas of engagement to allow for some generalization. Attention will be paid to capture the relevant heterogeneity of interventions, and socio-economic, political economy and decentralization contexts, both of which are likely to affect outcomes. The selection criteria will be based on the initial global portfolio analysis, the review of country strategies, a literature review, and consultations with Bank and IFC staff. It will include cases with: (i) decentralization and SNG-level engagement featured as important strategic areas in country-owned development strategies and/or Bank and IFC documents; (ii) sufficient coverage and depth of the three thematic engagement areas (IGF, EFM, and GTA); (iii) distribution of cases across regions and client income- groups; (iv) FCV countries; and (iv) distribution across unitary and federal country structures. Given the long list of differentiation factors among countries and to have sufficient coverage of the selected interventions for deeper analysis, the selection of countries will be done in two steps. As a first step, a long list of countries will be selected based on a set of generic criteria (country and portfolio characteristics). Then additional selection criteria related to SNG and intervention diversity will be added for each country in the long list (see Attachment 5). 16 • Selection of SNGs for case-based analysis. Within selected countries, further purposive sampling will be applied to SNGs to reflect the variation in: (i) the structure of subnational government; (ii) the socio-economic context. • Selection of interventions for case-based analysis. Intervention is defined as a package of activities targeted to a specific sub intermediate outcome. For example, under EFM, Bank support to strengthening Public Investment Management (PIM) at the subnational level or Domestic Revenue Mobilization (DRM) would constitute an intervention. Selection of interventions for a more in-depth analysis in each of EFM and GTA thematic areas will be based inter alia on the following criteria: (i) volume in portfolio, (ii) stakeholder demand, and (iii) availability of evidence on effectiveness. DESIGN LIMITATIONS 40. The evaluation design contains both strengths and weaknesses. The triangulation of the proposed sources of evidence offers a good base to answer the evaluation questions—particularly in relation to relevance and strategic role. The evidence base will provide broad guidance and lessons drawn from the past and present support to help enhance the WBG approach, role, and positioning vis-à-vis central and subnational governments throughout the process of engagement on SNGs strengthening. Causal analysis of WBG effectiveness will also rely on triangulation but will be even more constrained due to the inherent complexity of causal processes, the heterogeneity in Bank and IFC interventions and the contexts in which they are embedded, data availability, and the evaluation’s resources to address these issues. Overall, the following factors will constrain the evaluation: • First, limited evaluation evidence is available as only 30 percent of lending operations identified by the initial portfolio review have been evaluated by IEG17. • Second, limited data will be generated from the completion reports of the World Bank’s closed operations included in the sample, as relevant support often comprise components or sub-components, rather than full operations. • Third, data collection and its reliability at the subnational level is traditionally more complicated than at the national level. • Fourth, there is a risk of implicit selection bias, i.e. the evaluation team will need to select not only countries, but also a small sample of subnational units at both the second and third tier of government within these countries. • Fifth, data, time and budget constraints inevitably influence the depth and breadth of empirical analysis at the selected country, SNG and intervention levels, as a result of which 17 Among the 563 SSNG projects approved over FY2008-2017, 310 have been closed (or 55 percent of the total number of SSNG projects) and 172 have been evaluated by IEG. The evaluation rate for DSNG projects over the evaluation period is higher than the average for the total World Bank portfolio (30 percent compared to 23 percent). 17 the evaluation has to strike an informed balance between feasibility and credibility considerations. Quality Assurance Process 41. This evaluation will be subject to IEG’s standard quality review. The external peer reviewers are: • Joel Hellman, Dean of School of Foreign Service, Georgetown University, former Director and Chief Institutional Economist, World Bank. • Lili Liu, visiting professor at the Central University of Economics and Finance in Beijing; former Global Lead on Decentralization and Intergovernmental Relations in the Governance Global Practice of the World Bank, and co-chair of the Decentralization and Subnational Regional Economics Thematic Group; • Luiz de Mello, Deputy Director of the Public Governance and Territorial Development Directorate, OECD. 42. In addition, the approach and scope of the evaluation has been further discussed at a workshop with Bank staff and management to help ensure relevance of the evaluation questions and definitions. The intervention logic and evaluation questions, will be further refined and finalized based on the final portfolio identification and in close consultation with Bank and IFC counterparts. Expected Outputs and Outreach 43. The main output will be an evaluation report to the Board’s Committee on Development Effectiveness (CODE), which will contain the main findings and recommendations. The final evaluation report will be disseminated both internally and externally through a series of open learning events and BBLs. IEG will develop, as appropriate, additional presentations, bite-sized products showcasing key findings, blogs, videos, and other products for relevant audiences, including Bank staff, clients and partners, academia, multilateral development banks (MDBs), UN agencies, bilateral organizations, and NGOs. During the preparatory stages of the evaluation, the team will solicit feedback and comments from stakeholders, with World Bank Group staff working on decentralization and subnational government development. During field missions, IEG will meet with a diverse set of stakeholders to inform the evaluation. To maximize the value and use of findings and recommendations to strengthen development outcomes, IEG will develop a detailed outreach strategy prior to the one-stop meeting, which will be implemented after the report’s disclosure. IEG will take special consideration to ensure that dissemination efforts reach Bank and IFC staff and stakeholders located in countries, as well as other multilateral development banks, donors, and partners. 44. Regular stakeholder interaction with relevant units in the Bank and IFC will be sought to enhance the evaluation process. Throughout the evaluation, the team will have ongoing dialogue with Bank and IFC staff working on strengthening subnational governments. The team has already hosted an engagement workshop to solicit feedback on the draft approach paper and is committed to host a similar workshop at the end of the evaluation cycle to receive informal feedback on 18 evaluation recommendations. In addition, the team will set up learning and knowledge sharing sessions while undertaking the evaluation to create feedback loops with Bank and IFC experts. These sessions could cover issues such as the selection of case studies, evaluation questions, early findings etc... The team will also reach out to and engage with partner organizations - particularly other MDBs, UN agencies, and bilateral organizations. Resources 45. The evaluation will be prepared by a team led by Claude Leroy-Themeze (Senior Economist), under the direction and guidance of Auguste Tano Kouame (Director) and Pablo Fajnzylber (Manager). Konstantin Atanesyan (Senior Evaluation Officer) has been co-leading the preparation of the approach paper and will remain involved in the evaluation work. Other team members include Victor Vergara (Lead Urban Specialist), Stephan R. Wegner (Senior Evaluation Officer) and Yumeka Hirano (Economist). Lev Freinkman, Basil Kavalsky and Paul Smoke are short term consultants working on this task. Other team members and consultants will be identified as needed. The portfolio review and analysis will be conducted by a team led by Maria Shkaratan (consultant) with the support of Jacqueline Andrieu, Evaluation Officer. The PRA team comprises Corky De Asis (Evaluation Assistant), Shakib Noori (Consultant), and Hassan Wally (Consultant). Jos Vaessen will advise the team on methods. Yasmin Angeles will provide administrative support. 46. The evaluation will be prepared with an estimated net budget of US$1,000,000 (including dissemination). Costs associated with conducting the evaluation will be incurred in FY18-19. Funds allocated for dissemination will be incurred in FY19. The report will be finalized and submitted to CODE in the fourth quarter of FY19. 19 List of Attachments 1. References 2. Decentralization forms 3. Evaluation methods 4. List of relevant IEG evaluations 5. Main criteria for case selection (Countries, SNGs, Interventions) 20 Attachment 1 References Adams, Brian. 2016. “Assessing the Merits of Decentralization: A Framework for Identifying the Causal Mechanisms Influencing Policy Outcomes.” Politics & Policy 44 (5): 820–849. Ahamad, Farrah, Simon Boehler, Zahra Khan, and Ruvinda Pilapitiya. 2014. “The Politics of Decentralized Service Provision: A Conceptual Framework.” Working Paper 05/08/2014, Urban Institute, Washington, DC. Ahmad, Junaid, Shantayanan Devarajan, Stuti Khemani. and Shekhar Shah. 2005. Decentralization and Service Delivery.” Policy Research Working Paper 3603. Washington, DC: World Bank. Ahmad, Ehtisham, Giorgio Brosio, and Vito Tanzi. 2008. “Service Provision in Selected OECD Countries: Do Decentralized Operations Work Better?” Working Paper 08/07. Washington, DC: IMF. Ahmad, Ehtisham, and Giorgio Brosio. 2009. Does Decentralization Enhance Effective Service Delivery and Poverty Reduction? Edward Elgar, Northampton, MA. Ahmad, Ehtisham, and Giorgio Brosio. 2015. Handbook of Multilevel Finance. Edited by Center for Development Research, University of Bonn, Germany and Asia Research Centre, London School of Economics, UK, Department of Economics and Statistics, University of Turin, Italy. Chelteham, UK; Northampton, MA, USA: Edward Elgar. Azfar, Omar, Satu, Kähkönen, Anthony, Lanyi, Patrick Meagher, and Diana Rutherford. 1999. Decentralization, Governance and Public Services: The Impact of Institutional Arrangements: A Review of Literature”. IRIS Center, University of Maryland, College Park, MD. Bahl, Roy. 1999. “Implementation Rules for Fiscal Decentralization.” Working Paper 99-1, International Studies Program, School of Policy Studies, Georgia State University, Atlanta, Georgia. Boex, Jamie and Serdar Yilmaz. 2010. “An Analytical Framework for Assessing Decentralized Local Governance and the Local Public Sector.” Working Paper 2010-06, Urban Institute Center on International Development and Governance, Washington, DC. Bruyninckx H., S. Happaerts, and K. Van Den Brande, 2012. Sustainable Development and Subnational Governments: Policy-Making and Multi-level interaction. Palgrave Macmillan, United Kingdom. Caceres, Susan, Robert Yin, Anthony Tyrrell, Judy Hahn Gaubatz, and Sian Williams. 2016. “Introducing a Framework for Evaluating Service Delivery in Sector Evaluations”. IEG Working Paper 2016/ No.3. Washington, DC: World Bank. 21 Campos Jose Edgardo and Joel S. Hellman. 2005. “Governance Gone Local: Does Decentralization Improve Accountability”. In East Asia Decentralize: Making Local Government Work. A World Bank Study. Washington, DC: World Bank. Canuto, Otaviano, and Lili Liu, Eds. 2013. Until Debt Do Us Part: Subnational Debt, Insolvency, and Markets. Washington, DC: World Bank. Charbit, C. 2011. “Governance of Public Policies in Decentralised Contexts: The Multi-Level Approach.” OECD Regional Development Working Paper 2011/04. Paris: OECD. Cheema, Shabbir and Dennis Rondinelli. 2007. Decentralizing Governance: Emerging Concepts and Practices, Series: Brookings / Ash Institute Series, "Innovative Governance in the 21st Century", Ash Institute for Democratic Governance and Innovation, John F. Kennedy, School of Government, Harvard University. Brookings Institution Press, Washington DC. Kaiser, Kai. 2005. Decentralization Reforms. In Analyzing the Distributional Impact of Reforms: A Practitioners’ Guide. Coudouel, Aline and Stefano Paternostro, Eds. Washington, DC: World Bank. Da Cruz, Nuno Ferreira and Michel Gary. 2015. “Local Governance Integrity- Principles and Standards.” Transparency International, Germany. De Mello, Luiz. 2003. “Fiscal Decentralization and Subnational Expenditure Policy.” Public Finance Review, Vol. 29 No. 2, March 2001 108-138. Dreher, Axel. 2006. “Power to the People? The Impact of Decentralization on Governance.” Working Paper 121, Swiss Institute for Business Cycle Research, Zurich, Switzerland. Dyer, Caroline and Pauline Rose. 2005. “Decentralisation for Educational Development? An Editorial Introduction.” Compare: A Journal of Comparative and International Education, Volume 35, 2005 - Issue 2: Decentralisation for Educational Development: 105–113. Eaton, Kent, Kai Kaiser, and Paul Smoke. 2010. The Political Economy of Decentralization Reforms: Implications for Aid Effectiveness. Directions in development; public sector governance. Washington, DC: World Bank. Faguet, Jean-Paul and Caroline Pöschl, Eds. 2015. Is Decentralization Good for Development? Perspectives from Academics and Policy Makers. Oxford University Press, 2015, Oxford, England. Gilbert, Roy, Ramachandra Jammi, Kavita Mathur, Heather Dittbrenner, and Roy Gilbert. 2009. Improving Municipal Management for Cities to Succeed: An IEG Special Study. Washington, DC: World Bank. Ivanyna, Maksym, and Anwar Shah (2014). “How Close Is Your Government to Its People? Worldwide Indicators on Localization and Decentralization”. Economics: The Open-Access, Open-Assessment E-Journal, 8 (2014-3): 1–61. 22 Liu. L. 2017. “Fiscal Decentralization”. Paper presented at MFM Forum Session. Washington DC, March 29. Litvack, Jennie, Junaid Ahmad, and Richard Bird. 1998. Rethinking Decentralization in Developing Countries. A World Bank Sector Study Series. Washington, DC: World Bank. Maaroof, Ludwig, Jessica, Adrianus Hendrawan, and Aurelien Kruse. 2008. “Making Decentralization Work for Development: Methodology of the Local Government Performance Measurement (LGPM) Framework.” Washington, DC: World Bank. Martinez-Vasquez, Jorge. 2011. “The Impact of Fiscal Decentralization- Issues in Theory and Challenges in Practices”. Asian Development Bank, Mandaluyong, Philippines. Martinez-Vazquez, Jorge, Santiago Lago-Peñas and Agnese Sacchi. 2017. “The Impact of Fiscal Decentralization: A survey.” Journal of Economic Surveys, Volume 31, Issue 4, OECD-DAC, 2004. Lessons Learned on Donor Support to Decentralization and Local Governance. Michelsen, James. 2016. “Financing Municipalities in Latin America: IFC Subnational Finance.” CCAC LatAm Regional Solid Waste Workshop. Washington, DC: World Bank. OECD. 2008. "The Challenge of Capacity Development: Working Towards Good Practice", Journal on Development, vol. 8/3, Paris: OECD. ---.2010. “Does Fiscal Decentralisation Strengthen Social Capital? Cross-Country Evidence and the Experiences of Brazil and Indonesia”, OECD Economics Department Working Papers, No. 825, Paris: OECD. OECD-UCLG. 2016. Subnational Governments around the World: Structure and Finance. Paris: OECD. Robinson, Mark. 2007. “Does Decentralization Improve Equity and Efficiency in Public Service Delivery Provision?” Institute for Development Studies, Bulletin. Volume 38, Number 1, January, pp 7-17. Saavedra, Pablo A. 2010. "A Study of the Impact of Decentralization on Access to Service Delivery." Dissertation, Georgia State University. Schoburgh, Eris and Roberta Ryan, Eds. 2016. Handbook of Research on Subnational Governance and Development. IGI Global, Hershey, PA. Shah, Anwar. 2006. Local Governments in Developing Countries. A World Bank Study. Washington, DC: World Bank. Smoke, Paul. 2007. Fiscal Decentralization and Intergovernmental Relations in Developing Countries: Navigating a Viable Path to Reform. In G. S. Cheema, & D. Rondinelli (Eds.), Decentralized Governance: Emerging Concepts and Practice. Brookings Institution Press Washington, DC. 23 ---. 2013. The Role of Decentralisation/Devolution in Improving Development Outcomes at the Local Level: Review of the Literature and Selected Cases. Local Development International LLC, Brooklyn, NY. ---. 2015a. “Financing Local and Regional Governments - The Missing Link in Sustainable Development Finance.” UCLG. Global Taskforce on Local and Regional Governments. ---. 2015b. “Rethinking Decentralization: Assessing Challenges to a Popular Public Sector Reform.” Public Administration and Development 35, 97–112. ---. 2015c. “Managing Public Sector Decentralization in Developing Countries: Moving Beyond Conventional Recipes.” Public Administration and Development 35, 250–262. ---. 2015d. “Accountability and Service delivery in Decentralizing Environments: Understanding Context and Strategically Advancing Reform.” In a Governance Practitioner’s Notebook: Alternative Ideas and Approaches. OECD, Paris, France. Sow, Moussé and Ivohasina F. Razafimahefa. 2015. “Fiscal Decentralization and the Efficiency of Public Service Delivery.” IMF Working Paper. Washington, DC: IMF. Torrisi, Gianpiero. 2011. “Decentralization Outcomes: A Review of Evidence and Analysis of international data.” Centre for Urban and Regional Development, Newcastle University and Department of Geography and Environment, London School of Economics. London: Department for Communities and Local Government. United Cities and Local Governments. 2010. Local Government Finance: The Challenges of the 21st Century. United Cities and Local Governments, Spain. United Cities and Local Governments. 2017. National and Subnational Governments on the Way Towards the Localization of the SDGs. United Cities and Local Governments, Spain. UNDP. 2009. “A Users’ Guide to Measuring Local Governance.” New York: United Nations. UNDP. 2015. “Sustainable Development Goal Agenda.” New York: United Nations. UNDP and World Bank. 2016. “Transitioning from the MDGs to the SDGs.” Washington DC: World Bank; United Nations Development Programme, New York. Webb, Steven B., Marcelo M. Giugale. 2000. Achievements and Challenges of Fiscal Decentralization: Lessons from Mexico. A World Bank Study. Washington, DC: World Bank. Wescott, Clay. 2008. “World Bank Support for Public Financial Management: Conceptual Roots and Evidence of Impact”. Background Paper for IEG evaluation on Public Sector Reform: What Works and Why? Independent Evaluation Group (IEG)Washington, DC: World Bank. White, Roland and Paul Smoke. 2005. “East Asia Decentralizes”. In East Asia Decentralize: Making Local Government Work. A World Bank Study. Washington, DC: World Bank. 24 Wilde, Alexandra, Shipra Narang, Marie Laberge, and Luisa Moretto. 2009. A Users’ Guide to Measuring Local Governance. United Nations Development Programme. New York: United Nations. World Bank. 1997. World Development Report 1997: The State in a Changing World. Washington, DC: World Bank. ---. 1999. World Development Report 1999/2000: Entering the 21st century. Washington, DC: World Bank. ---. 2000. Reforming Public Institutions and Strengthening Governance: A World Bank Strategy November 2000. Washington, DC: World Bank. ---. 2002. World Development Report 2002: Building Institutions for Markets. Washington, DC: World Bank. ---. 2003a. Reforming Public Institutions and Strengthening Governance: A World Bank strategy - An implementation update. Washington, DC: World Bank. ---. 2003b. Water Supply and Sanitation Sector Business Strategy. Washington, DC: World Bank. ---.2004. World Development Report 2004: Making Services Work for the Poor. Washington, DC: World Bank. ---. 2005. Capacity Building in Africa: An Independent Review of World Bank Support. Independent Evaluation Group (IEG). Washington, DC: World Bank. ---.2006. Sub-National Development Program- A Proposed WBG Initiative to Scale Up and Mainstream Technical Assistance and Financial Support to Subnational Entities. Sub-National Development Working Group. A Report to the Board of Executive Directors. Washington, DC: World Bank. ---.2008a. Public Sector Reform: What Works and Why? An IEG evaluation of World Bank Support. Independent Evaluation Group (IEG). Washington, DC: World Bank. ---. 2008b. Decentralization in Client Countries: An Evaluation of World Bank Support, 1990- 2007. Independent Evaluation Group (IEG). Washington, DC: World Bank. ---. 2009a. The World Bank Urban and Local Government Strategy. Washington, DC: World Bank. ---. 2009b. Local Government Discretion and Accountability: Application of a Local Governance Framework. Washington, DC: World Bank. ---. 2010a. Subnational Development Policy Lending. OPCS Good Practice Note for Development Policy Lending. Washington, DC: World Bank. 25 ---.2010b. World Bank Engagement at the State Level: The Cases of Brazil, India, Nigeria and Russia. Independent Evaluation Group (IEG). Washington, DC: World Bank. ---. 2012a. Learning for All: Investing in People's Knowledge and Skills to Promote Development – A World Bank Group Education Strategy 2020. Washington, DC: World Bank. ---.2012b. The World Bank Approach to Public Sector Management 2011-2020 : Better Results from Public Sector Institutions. Washington, DC: World Bank. ---.2013. Results and Performance of the World Bank Group 2012. Independent Evaluation Group (IEG). Washington, DC: World Bank. ---. 2014. “Decentralization in Education, Health, and Social Protection-A Stocktaking Review of World Bank Human Development Network Portfolio”. Office of the Human Development Chief Economist. Washington, DC: World Bank. ---. Forthcoming. “Independent Evaluation of Development Interventions: An Overview of Approaches and Methods.” Independent Evaluation Group (IEG). Washington, DC: World Bank. Yilmaz, Serdar, Yakup Beris, and Rodrigo Serrano-Berthet. 2008. “Local Government Discretion and Accountability: A Diagnostic Framework for Local Governance”. Social Development Working Paper; No. 113. Local Governance and Accountability Series. Washington, DC: World Bank. 26 Attachment 2 Decentralization Forms18 1. Deconcentration refers to decentralization of central government ministries and arrangements whereby subnational governments act as agents of the center. Sometimes regional branches of central offices and agent governments have some authority to make independent decisions, usually within central guidelines. Often, though, deconcentrated local government lacks authority over the scope or quality of local services and how they are provided. 2. Delegation is an arrangement by which subnational governments rather than branches of central government are responsible for delivering certain services, subject to some supervision by the central government. It may improve efficiency when subnational governments can better administer programs of national interest—including certain aspects of education, water, and health—in ways that better reflect local circumstances. The center, or sometimes intermediate levels of government, determines what should be spent, and may also set minimum service standards, while subnational governments define the details. The design of intergovernmental fiscal transfers, and the degree and nature of central monitoring, influence the balance between central and local decision making under delegation. 3. Devolution is the most complete form of decentralization. Independent or semi- independent and, typically, elected subnational governments are responsible for the delivery of public services and for imposing fees and taxes to finance those services. Subnational governments have considerable flexibility in selecting the mix and level of services they provide. Other levels of government may provide intergovernmental transfers. For devolution to work, the central and local governments must act as partners, with the former keeping its commitment to devolve functions, and local officials agreeing to make difficult choices and develop the capacity to exercise their powers effectively. 4. Intergovernmental systems usually have some elements of each of these variants. In principle, devolution should improve efficiency by giving citizens more influence over the mix and level of services, and by giving local governments greater incentives to mobilize resources. The associated efficiency gains—combined with the ability to mobilize untapped revenues at the subnational level (from the formal and informal sectors)—may prove particularly significant in countries with diverse economies, cultures, geography, or tastes for services. Indeed, assigning some local control over expenditure and tax decisions can be key to nation building. 18 Based on Bahl, Roy. 1990. “Implementation Rules for Fiscal Decentralization.” Working Paper 99 -1. International Studies Program, School of Policy Studies, Georgia State University, Atlanta, Georgia. 27 Attachment 3 Evaluation Methods Table 1. Evaluation Building Blocks Evaluation Question 1: Evaluation Question 2: Evaluation Question 3 a. Evaluation Question 3 b. What has been the nature and How relevant and coherent has How and to what extent has the How and to what extent has the volume of Bank and IFC the Bank and IFC support to Bank and IFC support to Bank and IFC support to support to strengthening strengthening SNGs been? subnational governments subnational governments SNGs? contributed to strengthening contributed to strengthening their ability to deliver services their ability to deliver services and infrastructure, through: and infrastructure, through strengthening their economic and strengthening their policies and financial management (EFM) institutions for good governance, policies and institutions? transparency and accountability (GTA)? Global Level Desk review of key corporate Structured literature review: to Structured literature review: to strategy documents: to map review existing evidence on review existing evidence on Bank and IFC SSNG strategies strengthening SNGs’ economic and strengthening SNGs’ policies and and approaches and assess their financial management (EFM) institutions for good governance, relevance. policies and institutions. transparency and accountability (GTA). Portfolio review analysis: Portfolio review analysis: overall Portfolio review analysis: overall overall mapping and description mapping and description of the mapping and description of the of the global portfolio. global EFM portfolio. global GTA portfolio. Semi-structured interviews: Bank and IFC management and key development agencies management. 28 Evaluation Question 1: Evaluation Question 2: Evaluation Question 3 a. Evaluation Question 3 b. What has been the nature and How relevant and coherent has How and to what extent has the How and to what extent has the volume of Bank and IFC the Bank and IFC support to Bank and IFC support to Bank and IFC support to support to strengthening strengthening SNGs been? subnational governments subnational governments SNGs? contributed to strengthening contributed to strengthening their ability to deliver services their ability to deliver services and infrastructure, through: and infrastructure, through strengthening their economic and strengthening their policies and financial management (EFM) institutions for good governance, policies and institutions? transparency and accountability (GTA)? Country Desk review of key country Review of key strategy and Review of key strategy and Level strategy documents: available project documents to assess project documents to assess Bank and IFC country strategy relevance of Bank and IFC relevance of Bank and IFC documents to assess relevance of strategic approach and program to strategic approach and program to the Bank and IFC strategic strengthening SNGs’ EFM policies strengthening SNGs’ GTA policies approaches and programs for and institutions in country-specific and institutions in country-specific SSNG in country-specific context. context. context. Portfolio review analysis: Portfolio review analysis: Portfolio review analysis: triangulation of initial global triangulation of initial portfolio triangulation of initial portfolio portfolio review to identify review to identify patterns of review to identify patterns of patterns of some activities and activities, drivers of change, and activities, drivers of change, and direct outcomes. direct outcomes. direct outcomes. Existing IEG evaluations: CPEs, Existing IEG evaluations: CPEs, CLRRs, to collect evidence on the CLRRs, to collect evidence on the Bank and IFC contribution to Bank and IFC contribution to strengthening SNGs’ EFM policies strengthening SNGs’ GTA policies and institutions. and institutions. Semi-structured interviews (pre- Semi-structured interviews (pre- determined set of open questions determined set of open questions with opportunity for the with opportunity for the interviewer to explore themes or interviewer to explore themes or responses further) with key country responses further) with key country counterparts, focusing on EFM. counterparts, focusing GTA. 29 Evaluation Question 1: Evaluation Question 2: Evaluation Question 3 a. Evaluation Question 3 b. What has been the nature and How relevant and coherent has How and to what extent has the How and to what extent has the volume of Bank and IFC the Bank and IFC support to Bank and IFC support to Bank and IFC support to support to strengthening strengthening SNGs been? subnational governments subnational governments SNGs? contributed to strengthening contributed to strengthening their ability to deliver services their ability to deliver services and infrastructure, through: and infrastructure, through strengthening their economic and strengthening their policies and financial management (EFM) institutions for good governance, policies and institutions? transparency and accountability (GTA)? SNG level Desk review of key SNG Review of key strategy and Review of key strategy and strategy documents to assess project documents to assess the project documents to assess relevance of the Bank and IFC relevance and effectiveness of relevance and effectiveness of strategic approaches and Bank and IFC support to Bank and IFC support to programs for SSNG in SNG- strengthening SNGs’ EFM policies strengthening SNGs’ GTA policies specific context. and institutions in SNG-specific and institutions in SNG-specific context context. Existing IEG evaluations: ICRRs Existing IEG evaluations: ICRRs and PPARs to collect evidence on and PPARs to collect evidence on the Bank and IFC contribution to the Bank and IFC contribution to strengthening SNGs’ EFM policies strengthening SNGs’ GTA policies and institutions. and institutions. Portfolio review analysis: Portfolio review analysis: Portfolio review analysis: triangulation of initial global and triangulation of initial global and triangulation of initial global and country portfolio reviews to country portfolio reviews to country portfolio reviews to identify patterns of some identify patterns of some activities identify patterns of some activities activities and direct outcomes. and direct outcomes. and direct outcomes. Semi-structured interviews (pre- Semi-structured interviews (pre- determined set of open questions determined set of open questions with opportunity for the with opportunity for the interviewer to explore themes or interviewer to explore themes or responses further). responses further). 30 Evaluation Question 1: Evaluation Question 2: Evaluation Question 3 a. Evaluation Question 3 b. What has been the nature and How relevant and coherent has How and to what extent has the How and to what extent has the volume of Bank and IFC the Bank and IFC support to Bank and IFC support to Bank and IFC support to support to strengthening strengthening SNGs been? subnational governments subnational governments SNGs? contributed to strengthening contributed to strengthening their ability to deliver services their ability to deliver services and infrastructure, through: and infrastructure, through strengthening their economic and strengthening their policies and financial management (EFM) institutions for good governance, policies and institutions? transparency and accountability (GTA)? Intervention Desk review of key strategy, Existing IEG evaluations: ICRRs Existing IEG evaluations: ICRRs Level project documents and ASA: and PPARs to collect evidence on and PPARs to collect evidence on to assess relevance of the Bank the WBG contribution to the WBG contribution to and IFC strategic approaches strengthening SNGs’ EFM policies strengthening SNGs’ GTA policies and programs for selected and institutions. and institutions. specific interventions Portfolio review analysis: Portfolio review analysis: identify patterns of activities, identify patterns of activities, drivers of change, and direct drivers of change, and direct outcomes for selected interventions outcomes for selected interventions in EFM. in GTA. Semi-structured interviews (pre- Semi-structured interviews (pre- determined set of open questions determined set of open questions with opportunity for the with opportunity for the interviewer to explore themes or interviewer to explore themes or responses further). responses further). 31 Sources of evidence: The evaluation team will use multiple sources of evidence that will be compared and aligned with the concepts within the intervention logic and evaluation questions. The planned data collection and analysis for every source of evidence will be driven by the evaluation questions and protocols (see Table 1). • Background and Literature Reviews. Two papers will be prepared to support the evaluation: (i) Overview of WBG engagement with national and subnational governments covering the evolution of thinking, engagement modalities, approaches, and instruments deployed by the Bank and IFC and (ii) Structured literature reviews that will survey existing evidence on strengthening SNGs’ policies and institutions in economic and financial management (EFM) and governance, transparency and accountability (GTA). • Review of key strategy documents. Global. Relevant key strategic and analytical documents at global level will be reviewed to assess the evolution of the Bank and IFC approach to strengthening SNGs and practical solutions implemented on the ground. The exercise will synthesize material to generate insights into the role of the Bank and IFC and the relevance of its response. Country. The evaluation team will review country strategy documents (SCD/CPF), available Country Learning and Results Reviews (CLRRs), Bank and IFC analytical reports and relevant documents (including existing evaluations) for each selected country. The inquiry will be based on a protocol detailing the main evaluation questions in relation to the Bank and IFC support. The findings across selected countries will be reviewed as comparative studies to distill similarities, differences, and patterns related to the Bank and IFC support that can, in turn, be used to inform lessons and recommendations. SNG. Review of key SNG strategy documents to assess relevance of the Bank and IFC strategic approaches and programs for SSNG in SNG-specific context. Review of key strategy and project documents to assess the relevance and effectiveness of Bank and IFC support to strengthening SNGs’ policies and institutions in EFM and GTA in SNG-specific context. Intervention. The evaluation team will review strategy documents, available ICRRs and PPARS, Bank and IFC analytical reports and relevant documents (including existing evaluations) for each selected intervention. The review will be based on a protocol detailing the main evaluation questions in relation to the Bank and IFC support. The findings across selected countries/SNGs will be reviewed as comparative studies to identify similarities, differences, and patterns related to the Bank and IFC support that can, in turn, be used to support findings, lessons and recommendations. • Portfolio Review Analysis (Bank lending and ASA including Non-Lending Technical assistance and IFC investment and Advisory Services). A Portfolio Review Analysis will be conducted at four levels. Global. The evaluation team is currently undertaking a global review of the Bank and IFC lending/investment portfolio and ASA/AS over FY2008-2017 to identify Bank and IFC operations from a series of search terms related to decentralization and subnational governments. Key source materials are the appraisal documents, completion reports (including IEG validations thereof), post-project studies, and all relevant ASA/AS. The review will also examine the type of assistance provided; the financing 32 mechanism used; the extent to which the projects and ASA/AS appeared to meet the objectives related to intergovernmental frameworks and strengthening SNGs core upstream policies and institutions for service delivery. A systematic review of activities and contextual parameters will be conducted to identify patterns. Country. The portfolio review analysis for selected countries will include a systematic review of activities, drivers for change and intermediary outcomes. SNGs. The portfolio review analysis at the SNG level will allow for triangulation of initial global and country portfolio reviews to identify patterns of some activities and outcomes. Intervention. The portfolio review analysis will identify patterns of, context and outcomes selected interventions across the three thematic areas (Intergovernmental frameworks, EFM and GTA). • Stakeholder Interviews. The evaluation team will collect data through semi- structured interviews and focus groups at the global level (Bank and IFC staff in various GPs (Governance, MTI, SURR, others), Regions, as well as global stakeholders), at country level (Bank and IFC country management units, central government, stakeholders in selected countries, and relevant stakeholders at the selected subnational level) and intervention level. Interviews will explore diverse perspectives on topics such as: Bank and IFC approach and instruments, collaboration, and constraints. Semi-structured interviews will combine a pre- determined set of open questions (questions that prompt discussion) with the opportunity for the interviewer to explore specific themes or responses further. • Consultations with stakeholders. The evaluation team is planning to continue active discussions with relevant counterparts and stakeholders within the Bank and IFC and externally throughout the evaluation process. These discussions will take the form of joint brainstorming sessions, methodological seminars, and discussions of early findings – aimed at mid-course corrections and capitalizing on the high learning potential of this evaluation. The multi-sector nature of the topic and its importance to multiple Global Practices and Regions of the WBG make this type of interactive approach particularly relevant and important. The evaluation intends to benefit from these consultations particularly at the initial stage of the evaluation, conducting structured literature review, global portfolio review, and identifying countries, subnational governments and interventions for field-based case work. 33 Attachment 4 List of Relevant IEG Evaluation Work 1. Peru: Country program Evaluation for the World Bank Group, 2003-2009 2. The World Bank in Indonesia 1999-2006, Country Assistance Evaluation 3. The World Bank in Nepal 2003-2008, Country Program Evaluation 4. Uganda Country Assistance Evaluation, 2001-2007, Joint IEG/OPEV Country Assistance Evaluation 5. Brazil Country Program Evaluation, FY2004-2011 6. Ethiopia Country Assistance Evaluation, 1998-2006 7. The Republic of Mozambique Public Sector Reform Project, PPAR 2014 8. Republic of Rwanda Decentralization and Community Development Project, PPAR 2015 9. The Republic of Uganda Second Local Government Development Project, PPAR 2014 10. Tanzania Local Government Support Project, PPAR 2016 34 Attachment 5 Main Criteria for Case Selection (Countries, SNGs, Interventions) Country Selection – First Step Income Level Low Income Country Lower Middle-Income Country Upper Middle-Income Country Country Status IBRD Blend IDA Regions AFR EAP ECA LCR MNA SAR Country Political Structure Federal Unitary FCV Status Yes or No WB SSNG Intensity SSNG in Total ASA SSNG Commitments IFC SSNG Intensity SSNG in Total AS SSNG in Total Investment Country Selection- Second Step Decentralization Stage Old Recent Degree of Fiscal Decentralization Share of SNG Expenditure in Total Public Expenditure Length of WB Support to SSNG Number of Decentralization Levels WB Thematic Areas Coverage Intergovernmental Frameworks Economic and Financial Management Governance, Transparency, and Accountability IFC SSNG Intensity SSNG in Total AS SSNG in Total Investment SNG Selection Tier of Government Tier 2 35 Tier 3 Tier 4 Socio-Economic Context Poverty Incidence GDP per capita Urban/Rural Intervention Selection IGF area To be determined EFM area To be determined GTA area To be determined 36