GPR 65336 Marrakech action Plan for StatiSticS, PartnerShiP in StatiSticS for DeveloPMent in the 21St century, anD truSt funD for StatiStical caPacity BuilDing GLOBAL PROGRAM REVIEW Volume 5 Issue 3 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions—the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their environment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP ImPROVING DEVELOPmENT REsuLTs ThROuGh ExcELLENcE IN EVALuATION The Independent Evaluation Group (IEG) is an independent unit within the World Bank Group. It reports directly to the Board of Executive Directors, which oversees IEG’s work through its Committee on Development Effectiveness. IEG is charged with evaluating the activities of the World Bank (the International Bank for Reconstruction and Development and the International Development Association), the work of the International Finance Corporation in private sector development, and the guarantee projects and services of the Multilateral Investment Guarantee Agency. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group’s work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. Global Program Review Marrakech Action Plan for Statistics Partnership in Statistics for Development in the 21st Century Trust Fund for Statistical Capacity Building June 30, 2011 Public Sector Evaluations http://www.globalevaluations.org ©2011 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: grpp@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. How to cite this report: IEG (Independent Evaluation Group). 2011. “Marrakech Action Plan for Statistics, Partnership in Statistics for Development in the 21st Century, and Trust Fund for Statistical Capacity Building.� Global Program Review Vol. 5, Issue 3. Cover image: ©iStockphoto.com/DNY59 ISBN-13: 978-1-60244-187-3 ISBN-10: 1-60244-187-1 Independent Evaluation Group Strategy, Communication, and Learning (IEGCS) E-mail: grpp@worldbank.org Telephone: 202-458-4497 Printed on Recycled Paper i IEG Mission: Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) of the World Bank annually reviews a number of global and regional partnership programs (GRPPs) in which the Bank is a partner, in accordance with a mandate from the Bank’s Executive Board in September 2004. The two main purposes are (a) to help improve the relevance and the effectiveness of the programs being reviewed, and (b) to identify and disseminate lessons of broader application to other programs. IEG does not, as a matter a policy, recommend the continuation or discontinuation of any programs being reviewed, because this is properly the jurisdiction of the governing body of each program. A global or regional program review (GPR) is a “review� and not a full-fledged “evaluation.� The preparation of a GPR is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. Each GPR assesses the independence and quality of that evaluation; provides a second opinion on the effectiveness of the program, based on the evaluation; assesses the performance of the World Bank as a partner in the program; and draws lessons for the Bank’s engagement in GRPPs more generally. The GPR does not formally rate the various attributes of the program. Assessing the independence and quality of GRPP evaluations is an important aspect of GPRs because one of the reasons why IEG initiated this new product in 2005 was to encourage high quality evaluation methodology and practice more uniformly across Bank-supported GRPPs. Providing a “second opinion� on the effectiveness of the program includes validating the findings of the GRPP evaluation with respect to the effectiveness of the program. Assessing the performance of the World Bank as a partner in the program provides accountability to the Bank’s Executive Board. In selecting programs for review, preference is given to (a) those that are innovative, large, or complex; (b) those in which the Bank is sufficiently engaged to warrant a GPR, (c) those that are relevant to upcoming IEG sector studies; (d) those for which the Executive Directors or Bank management have requested reviews; and (e) those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR seeks to add value to the program and to the World Bank beyond what is contained in the external evaluation, while also drawing upon IEG’s experience in reviewing a growing number of programs. It reports on key program developments since the evaluation was completed, including the progress in implementing the recommendations of the evaluation. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside the World Bank or Washington, DC. Key stakeholders include the Bank’s representative on the governing body of the program, the Bank’s task team leader (if separate from the Bank’s representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal and external peer review and IEG management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program being reviewed. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached as an annex to the final report. After the document has been distributed to the Bank’s Board of Executive Directors, it is disclosed to the public on IEG’s external Web site. ii Abbreviations and Acronyms AAA Accra Agenda for Action ADP Accelerated Data Program AfDB African Development Bank AFR Sub-Saharan Africa (World Bank region) CAS Country Assistance Strategy CG Consultative Group CSIDB Country Statistical Information Database (WB) CSO Central Statistics Office DECDG Development Data Group DFID Department for International Development (United Kingdom) DGF Development Grant Facility ECOSOC Economic and Social Council (United Nations) EU European Union Eurostat Statistical Office of the European Communities GDDS General Data Dissemination System (of the IMF) GPP Global Programs and Partnerships GPR Global Program Review GRM Grant Reporting and Monitoring System IDA International Development Association IEG Independent Evaluation Group IHSN International Household Survey Network IMC Internal Management Committee LIC Low-income Country Logframe Logical Framework LSMS Living Standards Measurement Study MAPS Marrakech Action Plan for Statistics MDG Millennium Development Goals NADA National Data Archive Application NSDS National Strategy for the Development of Statistics NSO National Statistical Office NSS National Statistical System OECD Organisation for Economic Co-operation and Development OPCS Operations Policy and Country Services PARIS21 Partnership in Statistics for Development in the 21st Century PRESS Partner Report on Support to Statistics PRSP Poverty Reduction Strategy Paper PREM Poverty Reduction and Economic Management SCB Statistical capacity building SCBC Statistical Capacity Building Committee (WB) SRF Statistics for Results Facility STATCAP World Bank lending program for investment in statistical capacity TF Trust Fund TFAU Trust Fund Administration Unit TFSCB Trust Fund for Statistical Capacity Building iii TTL Task Team Leader UNECA United Nations Economic Commission for Africa UNECE United Nations Economic Commission for Europe UNSC United Nations Statistical Commission UNSD United Nations Statistics Division` WB World Bank Fiscal Year of Programs MAPS July-June PARIS21 January-December TFSCB July-June Director-General, Evaluation Mr. Vinod Thomas Director, Independent Evaluation Group (Public Sector) Ms. Monika Huppi (Acting) Manager, Independent Evaluation Group (Public Sector) Mr. Mark Sundberg Global Programs Coordinator Mr. Chris Gerrard Consultant Mr. Brian Ngo v Contents Preface...................................................................................................................................... ix Program at a Glance: Marrakech Action Plan for Statistics (MAPS)...................................... xi Key Bank Staff Responsible during Period under Review (MAPS) ................................. xii Program at a Glance: Partnership in Statistics for Development in the 21st Century (PARIS21).............................................................................................................................. xiii Key Bank Staff Responsible during Period under Review (PARIS21) ........................... xiv Program at a Glance: Trust Fund for Statistical Capacity Building (TFSCB) ....................... xv Key Bank Staff Responsible during Period under Review (TFSCB)............................... xvi Glossary ................................................................................................................................ xvii Summary ................................................................................................................................ xix 1. Programs’ Objectives, Activities, and Costs......................................................................... 1 Programs’ Origins and Objectives ........................................................................................ 1 Activities ............................................................................................................................... 2 Governance and Organization............................................................................................... 6 Monitoring and Evaluation Frameworks .............................................................................. 8 2. External Evaluations of the Programs................................................................................... 9 Independence of the Three Evaluations ................................................................................ 9 Quality of the Three Evaluations ........................................................................................ 12 Evaluation Frameworks .................................................................................................. 12 Approach and Scope ....................................................................................................... 12 Evaluation Instruments ................................................................................................... 13 Overall Quality of the Evaluations ................................................................................. 13 Evaluation Findings, Recommendations, and Feedback .................................................... 15 3. The Effectiveness of the Programs ..................................................................................... 18 Relevance of the Programs ................................................................................................. 18 Efficacy of the Three Programs........................................................................................ 23 Approach to the Assessment of Efficacy ..................................................................... 23 Progress of Activities and Outputs of the Three Programs ........................................ 25 Overall Achievement of Core Objectives .................................................................... 28 Efficiency of the Programs ................................................................................................. 34 4. Governance, Management and Financial Sustainability..................................................... 38 Governance and Management............................................................................................. 38 vi Financial Sustainability....................................................................................................... 46 5. The World Bank’s Performance as a Partner .................................................................. 51 The WBG’s Contributions at the Global Level .................................................................. 51 The WBG’s Contributions at the Country Level ................................................................ 52 Oversight ............................................................................................................................. 54 Disengagement Strategy ..................................................................................................... 55 6. Findings and Lessons ...................................................................................................... 58 Main Findings ..................................................................................................................... 58 Lessons ................................................................................................................................ 59 References ............................................................................................................................... 61 Boxes Box 1. Monitoring and Evaluation Frameworks – MAPS, PARIS21, AND TFSCB .............. 8 Box 2. The Marrakech Action Plan for Statistics ................................................................... 24 Figures Figure 1. Organizational Chart – MAPS................................................................................... 5 Figure 2. Organization Chart – PARIS21 (current structure) ................................................... 6 Figure 3. Organization Chart – TFSCB .................................................................................... 7 Tables Table 1. Goals of the Three Programs ................................................................................... xix Table 2. Main Recommendations of the Three Evaluations .................................................. xxi Table 3. Programs’ Goals and Objectives................................................................................. 2 Table 4. Programs’ Activities ................................................................................................... 3 Table 5. Evaluation Framework of the Three Evaluations ..................................................... 11 Table 6. Details on the Scope of the Evaluations ................................................................... 12 Table 7. Details on Evaluation Instruments Used ................................................................... 14 Table 8. Results of a Review of Statistical Capacity Building in Recent CASs (Africa Region) ........................................................................................................... 20 Table 9. A Summary of the Programs’ Strategies .................................................................. 22 Table 10. NSDS Activities by Program .................................................................................. 26 Table 11. NSDS Status for IDA Countries ............................................................................. 29 Table 12. World Bank Statistical Capacity Indicator (on a scale of 0 to 100) ....................... 33 Table 13. MAPS’ Programs Supported by the World Bank DGF ($ millions) ...................... 34 Table 14. PARIS21: Expenditures (thousands of Euros)........................................................ 35 Table 15. TFSCB Financing ($ millions) ............................................................................... 36 Table 16. Governance and Management Arrangements for MAPS ....................................... 39 vii Table 17. Revised Governance and Management Arrangements for PARIS21 (as of June 2010) ......................................................................................................... 40 Table 18. Governance and Management Arrangements for TFSCB ...................................... 42 Table 19. DGF Grant Allocations for MAPS ($ millions)...................................................... 47 Table 20. DGF Funding Leverage (in $ millions) .................................................................. 48 Table 21. PARIS21 Core Program Contributions (in Euros) ................................................. 49 Table 22. Roles Played by the World Bank in the Three Programs ....................................... 51 Table 23. Statistical Capacity Building Projects Funded by or through the World Bank ...... 53 Table 24. Exit Plan for MAPS Program (FY11–15) ($ millions) ........................................... 56 Annexes Annex A. Evaluation Framework for Global Program Reviews ............................................ 65 Annex B. Program Goals, Objectives, and Activities ............................................................. 73 Annex C. External Evaluation: Major Findings and Recommendations and the Program Response ................................................................................................................................. 86 Annex D. Members of the Governing Bodies ........................................................................ 89 Annex E. Sources and Uses of Funds ..................................................................................... 94 Annex F. Persons Consulted ................................................................................................... 97 Annex G. Response of the Programs to IEG’s Global Program Review................................ 98 ix Preface This is the Global Program Review (GPR) of three related global partnership programs that aim to develop statistical capacity in developing countries — the Marrakech Action Plan for Statistics (MAPS), the Partnership in Statistics for Development in the 21st Century (PARIS21), and the Trust Fund for Statistical Capacity Building (TFSCB). The PARIS21 Secretariat is located in the Organization for Economic Cooperation and Development (OECD) in Paris and the other two secretariats are located in the Development Data Group (DECDG) in the World Bank. The three programs have similar objectives: • For MAPS, “to improve the data needed to monitor the Millennium Development Goals and to strengthen the statistical capacity of developing countries.� • For PARIS21, “to develop a culture of management for development results.� • For TFSCB, “to build up and strengthen the statistical systems of developing countries.� The three programs have been reviewed together in a single GPR because they have similar objectives, because the World Bank has been heavily involved in all three programs, and because of the potential to learn cross-cutting lessons of experience in relation to statistical capacity building (SCB). As a founding member of all three programs, the World Bank has played an important role in the international effort which began in the late 1990s to promote SCB in developing countries in support of evidence-based policy-making. Together with other partners, the Bank helped to establish PARIS21 in 1999 to build and strengthen national statistical systems in developing countries. To complement PARIS21 activities, the TFSCB was set up in 1999 as a World Bank-administered, multi-donor trust fund to provide financial resources to developing countries for SCB activities. MAPS emerged from the Second Roundtable on Managing for Development Results in 2004 to broaden the efforts at both national and international levels to help developing countries achieve stronger capacities in statistics. The Bank has provided financial support to this collective effort through the long-term financing window of the Development Grant Facility. This review follows IEG’s Evaluation Framework for Global Program Reviews (Annex A). It is based on recent evaluations of all three programs, as follows: • Richard Roberts and Claudine Voyadzis, Evaluation of PARIS21: Final Report, November 15, 2009 • Christopher Willoughby and Philip Crook, Marrakech Action Plan for Statistics: Report of an Independent Evaluation, December 8, 2008 • Hallgrímur Snorrason, Andrew J. Flatt, and Jette Jensen, Trust Fund for Statistical Capacity Building: Evaluation Report, January 2010. This GPR has also reviewed relevant internal materials (progress reports, results frameworks, minutes of governing body meetings, etc.) and other information available on the Web. In x addition, IEG has independently obtained opinions and views on the three programs by interviewing staff of the Bank and the PARIS21 Secretariat, and selected members of the PARIS21 Board at the 2010 Board meeting in Paris, France. IEG gratefully acknowledges all those who made their time available for interviews and provided useful information and insights into the program. It wishes to especially acknowledge the availability of GDN staff and their cooperation in providing all necessary information and documents. The complete list of people consulted can be found in Annex F. Following IEG’s normal procedures, copies of the draft GPR were sent to each of the three programs, to DECDG (which is responsible for the Bank’s engagement with the three programs), and to other World Bank units that have responsibility for the Bank’s involvement with global programs more generally. Their comments have been taken into account in finalizing the GPR. The formal responses received from PARIS21 and DECDG are attached in Annex G. xi Program at a Glance: Marrakech Action Plan for Statistics (MAPS) Start date 2004 Goal MAPS is an important element of an informal partnership whose main goal is to improve the development of statistics in developing countries. More particularly, it sets target dates regarding the preparation of national strategies for the development of statistics (NSDS) to improve the availability of key indicators, and when capacity should be in place in order to monitor progress towards the MDGs thus providing the required focus on NSDSs. Program objectives To help develop and strengthen national statistical systems through six specific actions: (i) prepare NSDSs; (ii) ensure full participation of developing countries in the 2010 census round; (iii) increase financing for statistical capacity building and international coordination; (iv) set up an International Household Survey Network; (v) undertake urgent improvements to monitor the Millennium Development Goals (MDGs) and other development goals; and (vi) increase accountability of the international statistical system. Activities Financial support for programs of activities designed to: (i) mainstream strategic planning for improving statistical systems in developing countries; (ii) introduce more efficient approaches to the conduct of household surveys in developing countries; (iii) help developing countries make best use of available survey data; and (iv) support International Development Association (IDA) countries in preparing for the 2010 Census Round. World Bank Group Under MAPS, the World Bank’s Development Grant Facility has provided contributions $29.7 million of financial support over FY2006–10. Other donor The African Development Bank provides support to statistical capacity contributions building in Africa using the MAPS framework and participates in the Accelerated Data Program (ADP). Location World Bank, Washington DC. Web site www.worldbank.org/data/action Governance and The (external) MAPS Advisory Board provides strategic direction, management promotes coordination, and monitors implementation. The World Bank (internal) Statistical Capacity Building Committee (SCBC) oversees the implementation of the Development Grant Facility (DGF) grant. A Coordination and Monitoring Unit in DECDG provides support to both. Latest program-level Christopher Willoughby and Philip Crook, Marrakech Action Plan for evaluation Statistics: Report of an Independent Evaluation, December 2008. xii Key Bank Staff Responsible during Period under Review (MAPS) Position Person Period Program Manager Misha V. Belkindas 2004 – December 2010 Grant J. Cameron December 2010 – present Global Program Task Team Neil J. Fantom 2004 – December 2010 Leader Barbro E. Hexeburg December 2010 – present World Bank Representative on Shaida Baidiee 2004 – present the Advisory Board Vice President Jim Adams 2004–08 Jeff Guttman 2008–10 Joachim von Amsburg 2010 – present Trust Fund Operations Not applicable Global Programs and Margret Thalwitz, Director May 2004 – 2008 Partnerships Global Partnership and Trust Junhui Wu, Director March 2009 – present Fund Operations xiii Program at a Glance: Partnership in Statistics for Development in the 21st Century (PARIS21) Start date November 1999 Goal To develop a culture of management for development results. Program objectives To encourage and support developing countries to design, implement, and monitor NSDSs including: (a) mobilizing resources for the implementation of NSDSs; (b) coordinating donor support to statistics; (c) coordinating all actors within the National Statistical System; (d) producing guidance and documentation; and (e) providing technical assistance (legislation, training, human resources, etc.). Activities PARIS21 focuses its efforts on: (i) encouraging all developing countries to design and implement NSDSs and to have nationally owned and produced data for all MDG indicators; (ii) through the preparation of advocacy materials, encouraging broader recognition by national and international policy makers of the role of statistics in development and poverty reduction; and (iii) promoting donor coordination for statistical capacity building efforts. World Bank Group Contributed $5.3 million to PARIS21 core activities until the end of FY2010 contributions and $10.3 million for two other programs. The Bank sits on the Board. Other donor Other donors contributed a total of $21.5 million to PARIS21 core activities. contributions Location OECD, Paris, France. Web site www.paris21.org Governance and The Board sets the strategic direction for PARIS21, reviews and advises on management the Secretariat’s medium-term work program, and reviews and provides inputs to the terms of reference and reports of the periodic evaluations of PARIS21. The Executive Committee provides an accountability mechanism and guidance to the ongoing work of the Secretariat, and reviews and approves annual work programs, budgets, and annual progress and financial reports. The Secretariat carries out the work program. Latest program-level Richard Roberts and Claudine Voyadzis, Evaluation of PARIS21, Final evaluation Report, November 15, 2009. xiv Key Bank Staff Responsible during Period under Review (PARIS21) Position Person Period Program Manager Antoine Simonpietri 2001–2009 Abadila Berrou 2009 – present Global Program Task Team Misha V. Belkindas 2004 – December 2010 Leader Grant J. Cameron December 2010 – present World Bank Representative on Shaida Baidiee 2004 – present the Steering Committee / Board of Directors Vice President Nicholas Stern 2000–2003 François Bourguignon 2003–2007 Justin Yifu Lin June 2008 – present Trust Fund Operations Not applicable Global Programs and Margret Thalwitz, Director May 2004 – 2008 Partnerships Global Partnership and Trust Junhui Wu, Director March 2009 – present Fund Operations xv Program at a Glance: Trust Fund for Statistical Capacity Building (TFSCB) Start date 1999 Goal To improve the capacity of developing countries to compile and use statistics with the overall objective of supporting the management of development results. From the beginning the TFSCB has been closely aligned with PARIS21. Since 2004, TFSCB has been an important instrument for implementing MAPS. Program objectives To support the preparation of NSDSs, in line with MAPS and working with PARIS21 to ensure that all low-income countries have an integrated and comprehensive plan for the strategic development of their national statistical systems and detailed capacity building programs for statistics. Activities TFSCB provides small grants of up to $500,000 over a period of two or three years to low-income countries and to appropriate regional or international organizations to implement specific capacity building projects. Over the past ten years, the Trust Fund has committed more than $35 million to projects covering more than 80 countries. World Bank Group Trust fund management; review and approve/reject project proposals for contributions trust fund funding; manage projects where country capacity is inadequate; monitor project implementation (focus on administrative aspects); and produce regular progress reports to trust fund donors. Other donor Six donors for three phases (TFSCB I, II and III) for a total of $46 million. contributions Location World Bank, Washington D.C. Web site www.worldbank.org/tfscb Governance and The TFSCB is governed by a 12-member Consultative Group. Day-to-day management management of the TFSCB is provided by the Trust Fund Administration Unit. The Internal Management Committee (IMC), which is chaired by a DECDG Manager, reviews project proposals and decides on funding. An external advisory panel of two experts reviews all IMC decisions on TFSCB proposals and provides advice on the future direction of the TFSCB. Latest program-level Hallgrímur Snorrason, Andrew J. Flatt, and Jette Jense, World Bank evaluation Trust Fund for Statistical Capacity Building: Evaluation Report, January 2010. xvi Key Bank Staff Responsible during Period under Review (TFSCB) Position Person Period Program Manager Misha V. Belkindas 2004 – December 2010 Grant J. Cameron December 2010 – present Global Program Task Team Mustafa Dinc – present Leader World Bank Representative on Shaida Badiee Since 2004 the Consultative Group Vice President Nicholas Stern 2000–2003 François Bourguignon 2003–2007 Justin Yifu Lin Since June 2008 Trust Fund Operations Arif Zulfiqar, Director June 1999 – present Global Programs and Margret Thalwitz, Director May 2004 – 2008 Partnerships Global Partnership and Trust Junhui Wu, Director March 2009 – present Fund Operations xvii Glossary Devolution or A proactive strategy to change the design of a program, to devolve some of its exit strategy implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. The term is also used as a broader, aggregate measure — encompassing relevance and efficiency as well — of the overall outcome of a development intervention such as a GRPP. Efficiency The extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs. Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or project, its design, implementation, and results. The aim is to determine the relevance and achievement of its objectives, and its developmental effectiveness, efficiency, impact, and sustainability. Governance The structures, functions, processes, and organizational traditions that have been put in place within the context of a program’s authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. It is the framework of accountability and responsibility to users, stakeholders and the wider community, within which organizations take decisions, and lead and control their functions, to achieve their objectives. Impacts Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Independent An evaluation that is carried out by entities and persons free from the control of evaluation those involved in policy-making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from interference, and avoidance of conflicts of interest. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Legitimacy As a criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program — including shareholders, other stakeholders, implementers, beneficiaries, and the community at large. Logical A management technique that is used to develop the overall design of a framework or program or project, to improve implementation monitoring, and to strengthen logframe evaluation by presenting the essential elements of the program or project clearly and succinctly throughout its cycle. It is a “cause and effect� model which aims to establish clear objectives and strategies based on a results chain, to build commitment and ownership among the stakeholders during the preparation of the program or project, and to relate the program’s or project’s interventions to their intended outcomes and impacts for beneficiaries. Management The day-to-day operation of a program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. xviii Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development intervention. Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Partners In most IEG Global Program Reviews, partners are understood as stakeholders who are involved in the governance or financing of the program (including the members of the governing, executive, or advisory bodies). Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular public good accrue across all or many countries, then the good is deemed a global or international public good. Relevance The extent to which the objectives and design of the program are consistent with (a) the current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. Shareholders The subset of donors that are involved in the governance of the program. Therefore, this does not include individual (particularly anonymous) donors who choose not to be so involved, or who are not entitled to be involved if their contribution does not meet the minimum requirement, say, for membership on the governing body. Stakeholders The parties who are interested in or affected, either positively or negatively, by the program. Stakeholders are often referred to as “principal� and “other�, or “direct� and “indirect�. While other or indirect stakeholders — such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries — may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. Sustainability When the term is applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When the term is applied to organizations or programs themselves, the extent to which the organization or program is likely to continue its operational activities over time. Transparency As a criterion for assessing governance and management, the extent to which a program’s decision-making, reporting, and evaluation processes are open and freely available to the general public. This is a metaphorical extension of the meaning used in physical sciences — a “transparent� objective being one that can be seen through. Source: For evaluation terms, the Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards, Independent Evaluation Group – World Bank, 2007. xix Summary Overview 1. The Partnership in Statistics for Development in the 21st Century (PARIS21), the Trust Fund for Statistical Capacity Building (TFSCB), and the Marrakech Action Plan for Statistics (MAPS) are three related programs with similar goals to improve statistical capacity, development statistics, and the use of development statistics in developing countries (Table 1). Despite the similarity in their goals, the programs continue to be separate because they arose at different times, in different contexts, and with different sources of funding. PARIS21 was founded in November 1999 in the context of the UN Conference on Development to develop a culture of evidence-based policy making. The TFSCB was established in 1999 as a World Bank-administered, multi-donor trust fund to provide grants to developing countries to improve their capacity to compile and use statistics. Created several years later, MAPS emerged from the Second Round Table for Managing for Development Results in 2004 as a global plan to improve development statistics based on an informal partnership involving developing countries, donors and statistical agencies worldwide. The World Bank’s Development Grant Facility has provided substantial funding to selected activities in this informal partnership since 2006. Table 1. Goals of the Three Programs MAPS PARIS21 TFSCB To improve development To develop a culture of To improve the capacity of statistics with two key target management for development developing countries to compile dates: 2010 when most results. Subsequent to the and use statistics with the overall countries would have prepared setting up of MAPS, objective of supporting the NSDSs, and 2015 when PARIS21’s work program has management of development capacity would be in place in been modified over time to results. From the beginning, order to monitor progress address MAPS’ objectives. TFSCB has been closely aligned towards the MDGs. with PARIS21. Since 2004, TFSCB has been an important instrument for implementing MAPS. 2. Like other Global Program Reviews, this GPR assesses the independence and quality of the three evaluations on which the review is based; provides a second opinion on the effectiveness of the three programs; assesses the performance of the Bank as a partner in the programs; and draws lessons for the Bank’s engagement in global and regional programs more generally. The GPR also seeks to add value to the programs and to the World Bank beyond what is contained in the external evaluations, while drawing upon IEG’s experience in reviewing a growing number of programs. It reports on key program developments since the evaluations were completed, including the progress in implementing the recommendations of the evaluations. 3. Since their inceptions in 1999, PARIS21and the TFSCB have been two parts of a common effort to build a culture of evidence-based policy making, with PARIS21 promoting dialogue and advocacy, and TFSCB providing small grants of up to $400,000 to developing xx countries to help strengthen their statistical systems. With the advent of MAPS in 2004, the focus of this collective effort has become more specific, namely, to monitor progress toward the MDGs and implementation of poverty reduction strategies. One key objective of MAPS has been to ensure that all developing countries have an integrated and comprehensive plan for the strategic development of their national statistical systems through an NSDS. MAPS has become, in effect, both an overall framework within which PARIS21 and TFSCB operate, and a major source of funding to PARIS21 and to other statistical capacity building activities such as to the United Nations Statistics Division (UNSD) to support developing countries in preparing for the 2010 census round. External Evaluations of the Programs 4. All three programs have had evaluations in recent years — in 2008 for MAPS, and in 2009 for PARIS21 and TFSCB. This IEG review is based on these evaluations, but also draws on, where appropriate, previous evaluations in 2003 and 2006 for PARIS21, and in 2003 and 2008 for TFSCB — the latter being an internal review by one of the principal donors, the United Kingdom’s Department for International Development (DFID). INDEPENDENCE OF THE THREE EVALUATIONS 5. DECDG commissioned the 2008 evaluation of MAPS on behalf of the MAPS Advisory Board and at the request of the DGF Council -- first, as a condition of the original approval for funding, and second, to inform the DGF Council’s decision whether to extend its funding for MAPS. The PARIS21 Secretariat commissioned its 2009 evaluation on behalf of the PARIS21 Steering Committee (now called the Board) with the primary purpose of evaluating progress towards the Partnership’s goal of developing a culture of evidence-based decision-making and using statistics to inform development policy. The TFSCB Consultative Group decided in early 2009 to carry out an evaluation of the TFSCB to coincide with the PARIS21 evaluation to review progress achieved so far and to help develop a coordinated strategy for the period 2010–2015. OVERALL QUALITY OF THE EVALUATIONS 6. This IEG review has identified a number of weaknesses in the external evaluations. First, while all three external evaluations clearly state that the assessment of program effectiveness was in their terms of reference, the focus was predominantly on processes and activities, with insufficient emphasis given to outputs and outcomes. While this may be justified by technical and conceptual challenges, the evaluations could have identified concrete ways in which the programs have contributed to the improvement in statistics and statistical capacity. Second, while all three evaluations share the common concern on the inadequate implementation of NSDSs, they did not provide useful insights on how or to what extent NSDSs have helped with the development of national visions for statistical development. Third, there could have been a sharper focus and more specific recommendations on the notable lack of progress in the use of statistics in sub-Saharan Africa. Lastly, it would have been useful to have more systematic cross-references to the results of the analyses in the three evaluations. xxi EVALUATION FINDINGS AND RECOMMENDATIONS 7. The evaluation report showed that the DGF-MAPS funding has yielded useful outputs and potentially useful tools (toolkits for household surveys, 2010 World Census Program of the UN, support for education, gender, and urban statistics with a strong potential for replicable results). The NSDS program, which has been managed by PARIS21, has demonstrated its utility and increasing country coverage. While the implementation of NSDSs remains a major challenge, NSDS processes have helped many governments develop a strategic vision for statistical development. The evaluation showed that projects financed by the TFSCB have been important for the SCB of developing countries and have had a positive impact on the generation and use of statistics. The main recommendations of the three external evaluations are summarized in Table 2. Table 2. Main Recommendations of the Three Evaluations Issue MAPS PARIS21 TFSCB Effectiveness • Keep NSDS • Improve the work on • Address the issue of implementation the top NSDSs and more the implementation of priority for MAPS. particularly on NSDSs and raise the • Work on accelerating enhancing their share of resources for the emergence of implementation. the NSDS process in country success cases. • Redirect government TFSCB to at least 60 percent. • Promote training in efforts on statistics to data use and focus more on interpretation among improving access to key government staff. policymakers. Governance • Expand the Advisory • Improve the • Enhance the and Board’s role in effectiveness of the transparency of management providing strategic Steering Committee by governance advice and direction. having more high level arrangements. • Widen the World Bank development and • Promote progress (internal) Statistical policy managers from reports on projects and Capacity Building donor institutions results-oriented Committee’s (SCBC) involved. completion reports. coordinating functions. Funding • DGF should extend • Secretariat should • Continue the funding levels for the continue working at the mobilization of donor main programs to at country level funding for the trust least 2012. fund. Role of the • Give fuller attention to • Intensify efforts to • Ensure that at least 60 Bank and statistical capacity and promote and support percent of the other donors the use of statistics in coordination of the TFSCB’s resources decision making in IDA national statistical should be allocated to Country Assistance systems. the NSDS process with Strategies. • Expand support in a strong emphasis on promoting coordination NSDS implementation; among donor partners increase ceiling of at the country level. grant to $500,000. xxii The Effectiveness of the Programs RELEVANCE OF THE PROGRAMS 8. International consensus. The IEG review confirms the assessments of the external evaluations that the objectives of all three programs were highly relevant at their inception and remain relevant notwithstanding the completion of important actions by all three programs. The broad consensus of the need for improved statistics in developing countries has been repeatedly reaffirmed, most recently at the 2010 MDG Summit which called for fostering a global partnership to support countries in generating reliable and timely data to assess progress on the MDGs and other country goals as prerequisites to improving aid effectiveness. 9. Beneficiary demand. The objectives and priorities of the three programs are well aligned with those of developing countries articulated in MDG reports, donors’ country assistance strategies, and the countries’ own poverty reduction strategy papers (PRSPs) — where the importance of statistical development receives much attention, particularly in relation to the need for monitoring and evaluation. At the Bank, the IDA Results Measurement System now requires that Country Assistance Strategies (CAS) should include a review of national statistical systems and an identification of what is needed to strengthen capacity both to generate and to use statistical information. But evidence from recent CAS documents in the Africa Region indicates that in spite of past efforts made to engage the attention of country teams, translating the need for SCB in countries to effective support for SCB will require much more effort. 10. Relevance of the design of the programs. Although the three programs were, as mentioned earlier in this report, designed separately, adjustment and refocusing of the objectives and activities have resulted in a package of strategies which reflects relatively well the evolving priorities of both developing countries and donors. The major activities of the three programs together provide: (a) a catalytic framework through MAPS and the DGF funding combined with knowledge networking with specialized agencies involved in survey methodology, and education and urban indicators; (b) technical assistance and advocacy at the level of countries and sub-regions through the various activities of PARIS21; and (c) extensive grant funding that has supported over 80 countries and initiatives in all developing sub-regions. 11. The IEG review confirms the strong relevance of the three programs designed to support SCB in developing countries. In terms of objectives, strategies, and activities, the three programs together reflect a coherent package. The initial designs of the programs, keeping in perspective that they were neither elaborated at the same time nor as a package, do, however, have some weaknesses that affect many global programs which are first, a strong focus on processes and activities, and second, insufficient attention to developing a rigorous conceptual framework linking activities and outputs to intended outcomes when these programs were set up. Since then, the programs have established logical frameworks that aim to provide these linkages and PARIS21 has recently revised its logical framework to strengthen the links between activities and outcomes. xxiii EFFICACY OF THE THREE PROGRAMS 12. The IEG review finds moderate progress on activities and outcomes from the three programs. The flexible approach toward redefining the programs’ intended outcome objectives with a primary focus on strengthening national strategies for the development of statistics has yielded important results. Progress toward the higher goal that aims to improve the capacity of developing countries to compile and use statistics in support of the management for development results has, however, been hampered by: (a) insufficient attention to implementation challenges facing the NSDS process; (b) the absence of a strategy to help stimulate demand for better data at the country level, not only among government users but also from other domestic stakeholders such as the civil society, NGOs, research institutes, and the media; and (c) inadequate albeit improving attention to the support for statistical capacity among the donor community. (a) Promoting NSDSs. Progress has been notable in relation to the design and implementation of NSDSs, a process which has taken hold in all regions. Of the 79 IDA-eligible borrower countries, all but 16 are designing or implementing an NSDS as of November 2010. Consultations undertaken for this IEG review show that the implementation of NSDSs remains an important challenge. To help countries meet the implementation challenge, the work program of PARIS21 in the 2010–14 period stresses: (a) translating the priorities into realistic, budgeted action plans; (b) bringing donors into the process, so that costs and financing plans can be prepared on a realistic basis and so that donor interest in and commitment to the NSDS can be followed through; and (c) reporting on progress in NSDS implementation against appropriate output and outcome indicators. (b) Promoting country participation in 2010 Survey Rounds. Compared to the 2000 Census Round, participation by developing countries has improved significantly for the 2010 round. Only nine countries or sub-regions have not yet scheduled a census. According to the most recent information, 140 million people will not be included in a census in the 2010 round, compared to 560 million in the 2000 round — a 75 percent reduction. Most significant in terms of outputs of this component is the large increase in the number of low-income countries, including 14 African countries, that either have held or are planning to hold a census during over 2005–14. (c) Investing in statistics. PARIS21’s annual exercise known as the Partner Report on Support to Statistics (PRESS), designed to learn what donors are doing and plan to do in statistical development, shows that financial commitments to statistical development for the period 2008–10 amounted to roughly $1.6 billion, with Africa receiving nearly half of the total. Since 2008, estimated global commitments to support statistics have risen by nearly 60 percent, although part of this increase is due to the greater response rate in the PRESS exercise. But the use of domestic resources by developing countries for statistical development remains very modest although more countries have reflected the rising importance of statistical development through their readiness to contract external loans and credits for that purpose instead of insisting on grant financing as was the case in the past. xxiv (d) Supporting surveys. More than 20 survey catalogues are now available on-line thanks to support from the Accelerated Data Program. There is a growing demand for support by countries but the lack of resources has prevented a scaling-up of this activity at the moment. Fifty-six countries have received or are receiving support, and 12 additional countries have requested assistance. (e) Strengthening MDG Indicators. Progress in the availability of data for 22 of the MDG official indicators has been noteworthy. Compared to 2003 when only 2 percent of the 163 countries had two data points for 16 or more of the 22 MDG indicators, 72 percent of the countries had at least two such data points in 2009, which allows for measuring trends over time. (f) Improving coordination in statistical support. PARIS21’s annual PRESS report has proven to be a useful tool for promoting effective collaboration among development partners both at the level of countries and for regional coordination. At the country level, a number of new approaches are being experimented with including donors’ groups and the setting up of “basket-of-funds�. EFFICIENCY OF THE PROGRAMS 13. The external evaluation teams found that the three programs are cost-efficient, with little evidence of resource wastage. For MAPS, the evaluators concluded that the amounts allocated were reasonable and that it would have cost more doing it through a profit-making body. For PARIS21, management costs have been held steady over the years at around 15–21 percent of total expenditures. For the TFSCB, an evaluation by DFID concluded that financial management of the fund was sound and that the application process was efficient. Consultations done for the review raise the legitimate concern that further scrutiny of the fund granting and monitoring processes may not be cost-effective given the relatively small size of TFSCB grants, with most grants amounting to less than $250,000. Governance and Management 14. Different but complementary, the governance and management arrangements of the three programs appear to be working moderately well in guiding and implementing a complex set of activities designed to build capacity and statistical development in developing countries. Overall, there do not appear to be important inconsistencies or anomalies in the governance or management structures although potential reputational risks for the World Bank should not be ignored due to the facts that the Bank is represented on all three boards and two of the three programs are located at the Bank. The complementarity of the three programs which has been highlighted throughout this report has important implications in terms of their financing. The phasing out of DGF funding to the MAPS program beginning in FY11 will have important implications for the PARIS21 Partnership. 15. On the whole, the governance structures of the three programs broadly comply with the generally accepted principles of good governance. On legitimacy, all three programs involve, besides their shareholders, a broader set of stakeholders. This is particularly the case for the PARIS21 Partnership whose Board consists of 40 members representing partners xxv from developing countries and developed countries in equal proportions. One area where participation was found to be inadequate concerns the weak representation in the governance structure of non-statistician data users, and especially of those with policy- making experience. This problem was highlighted by the external evaluations and had attracted renewed attention at the PARIS21 Board. 16. The governance and management structures ensure adequate command and control required for accountability. This is particularly true in the case of PARIS21 which has a strong accountability framework with clear chain of command and control. For MAPS, accountability is more diffuse with the DGF Council responsible for the overall scale of funding, the internal SCBC and external Advisory Board in charge of fund allocations among the different components of MAPS, the small MAPS Unit in charge of reporting, and individual task managers providing supervision of each of the grants disbursed by DGF through monitoring of progress reports and site visits as necessary. For the TFSCB, which is providing small grants for SCB, various external evaluations confirm good overall and financial management and efficient project selection. The use of Bank procurement rules, while considered cumbersome for the small grant-based projects, ensures good financial control. 17. Regarding transparency, public reporting of the progress of PARIS21 has been of high quality, and most particularly through its user-friendly Web site which also provides cross-references to key documents on the TFSCB. Reporting on MAPS and the TFSCB has significantly improved in the last year. 18. As regards efficiency of governance, the three external evaluations found that their managements are cost-conscious and supervising bodies help channel available resources to uses yielding higher development return. The external Boards themselves are not very costly since they normally cover travel costs for participants coming from developing countries only; all Board members contribute their time without any remuneration from MAPS or the other agencies served. 19. The fact that MAPS and TFSCB are located at the World Bank and the PARIS21 Partnership at the OECD raises a number of issues regarding program locations. For the TFSCB, the “Quality Assessment of Global and Regional Programs and Partnerships (GRPPs)� undertaken by the Bank’s Quality Assurance Group in 2009 highlighted the fact that the Bank has been too dominant in the design and management of the TFSCB. While the risk of an excessive involvement by the Bank remains, the risk has declined over time. For instance, the share of TFSCB projects executed by the Bank has declined significantly over the years from more than 40 percent in 2005 to less than 10 percent today (2011). Financial Sustainability 20. The complementarity between the three programs has important implications as far as financing of the programs is concerned. At the establishment of the PARIS21 Partnership and the World Bank-led TFSCB, it was envisaged that donor agencies would be solicited for jointly supporting these two programs which have distinct but closely related purposes. As things turned out, TFSCB grants have contributed to financing several PARIS21-initiated activities, especially for NSDSs. The establishment of MAPS in 2004 added another xxvi important source of significantly larger resources for statistical development, the World Bank’s DGF providing $5.7 million per year for an initial five-year period. The phasing out of DGF grants for MAPS will require a major effort by the PARIS21 Partnership to broaden the financing base beyond current donors. The World Bank’s Performance as a Partner 21. The World Bank plays many roles in the three programs and acts as financial contributor, trustee, convener, chair or member of the governing body, implementing agency, and host of the secretariat. As a founding member of all three programs under review, the World Bank played an important role in the international effort which began in the late 1990s to promote SCB in developing countries. Through its active involvement in the three programs, the Bank helped to build a broad network of partners involved in SCB, including statistical agencies, UN agencies, regional development banks, bilateral donors, and developing countries. The active participation of the Bank in all three programs has further enhanced their complementarity. 22. In Sub-Saharan Africa where the need for statistical support is greater compared to other developing regions, MAPS has served as the umbrella for the Reference Regional Strategic Framework for Statistical Capacity Building in Africa (RRSF) and for the African Development Bank (AfDB) — the leading provider of funds for SCB in Africa. 23. The external evaluations and other consultations undertaken in this review highlighted the Bank’s comparative advantage in SCB. First, the Bank was already heavily involved in country SCB primarily through the Living Standards Measurement Study (LSMS) which was established in the 1980s to foster increased use of household data as a basis for policy decision-making and through its active involvement in PRSP implementation. Second, the Bank’s various funding instruments have helped to support SCB projects in a large number of countries. Data made available on the World Bank Bulletin Board on Statistical Capacity show that cumulatively, the Bank through its own resources and the Trust Fund for Statistical Capacity Building, has funded a total of 239 projects. But in spite of this diversified lending portfolio for statistical development, the IDA Results Measurement System which aims to mainstream statistical capacity building efforts in country assistance strategies (CAS) has yet to have the desired impact. OVERSIGHT 24. Upon the recommendation of the DGF Council, the Bank’s Statistical Capacity Building Committee (SCBC) was established in 2005 to oversee the implementation of the MAPS’ DGF grant and to coordinate the preparation of future requests for DGF funding. But until very recently, the committee has played a more limited role than initially expected in terms of coordinating SCB activities within the Bank as a key component of the overall results agenda. This has undermined to some extent the impact that it could have had on advocacy for and promotion of SCB in developing countries. For the TFSCB, oversight is loosely provided through the Consultative Group which is supported by reports of the Advisory Panel, progress reports from DECDG, and external evaluations. As is the case for many GRPPs located in the World Bank, the Chair of the Consultative Group is also the Director overseeing the manager of the program. But in spite of the loose oversight arrangements, assessments by external evaluations tend to confirm that the existing xxvii arrangements seem to be working. But the lack of ex post evaluation of projects has not made it possible to draw on lessons from the past to improve the design of new projects. The Bank serves on both the Board and the Executive Committee of PARIS21 and thus provides strategic guidance and monitoring of progress of the activities undertaken by the Secretariat. DISENGAGEMENT STRATEGY 25. With the reorientation of DGF strategy which called for the exit of the programs under the long-term funding DGF window (known as Window 1), DGF funding will cease for most programs in FY13, leaving open the question of the sustainability or at least the scale of programs managed by PARIS21. At the Board meeting in June 2010, it was agreed that the PARIS21 Secretariat would start a fund-raising program that would focus on new sources of funding, including private foundations, as well as encourage governments in developing countries to increase funding from budgetary resources for NSDSs. The end of DGF funding will require a more in-depth review of options to ensure the sustainability of PARIS21’s core programs involving NSDSs as well as to the Accelerated Data Program. Designed as a pilot program with the overall objective to make existing survey data more widely and easily accessible, ADP has evolved very rapidly and the original target of 12 pilot countries in three years has been greatly exceeded; almost 60 countries are now participating. 26. The TFSCB continues to play an important role in implementing MAPS and more particularly, complementing PARIS21’s core program. Since its establishment, TFSCB has received over $46 million in donor contributions from six partners, namely, Canada, France, Germany, the Netherlands, Switzerland, and the United Kingdom. As of mid- 2010 about $35 million has been disbursed or allocated to ongoing projects. At the average annual commitment rate of $3 to $4 million, total available (unallocated) funds are expected to be exhausted by mid- or late-2011. An agreement in principle has been obtained from trust fund donors to continue funding of the trust fund by extending the operation of TFSCB III until 2015, using about $2 million of resources left over from TFSCB I and TFSCB II which have been closed. The prospects for additional funding from donors will be explored in the latter half of 2011, since there is no explicit disengagement strategy. Findings and Lessons MAIN FINDINGS 27. Significant progress has been achieved in the primary objective of encouraging and supporting developing countries to design National Strategies for the Development of Statistics (NSDSs), but only some progress has been made in NSDS implementation. The quality of statistics as measured by the World Bank Statistical Capacity Indicator has shown some improvement in the past 10 years, but the contribution of the three programs to the broader goals of developing a culture of evidence-based policy making and more specifically of reducing poverty and achieving the MDGs remains difficult to assess. 28. Statistical capacity building programs need to involve the users of statistics more actively. Through its advocacy work, PARIS21 has begun to broach this issue. Concerns xxviii about involving the users of statistics have yet to receive the same attention in MAPS and in the TFSCB. Providing more weight to the users of statistics in the design as well as the implementation of SCB programs would also imply the need to revisit the current governance arrangements of all three programs. 29. There appears to be a shift in the perception by developing countries toward statistical capacity building. Whereas low-income countries were reluctant to access resources other than grants to strengthen their statistical systems, statistical development has begun to receive the same priority as other sectors, instead of being perceived as a requirement imposed by external development partners for monitoring and evaluation purposes. 30. There is a need to strengthen the Bank’s commitment toward statistical capacity building activities. This conclusion is based first, on the continuing and rising relevance of the original objectives of the three programs as confirmed by the external evaluations and the renewed attention to aid effectiveness with the fast-approaching deadline to meet the MDGs by 2015. Second, the lack of or inadequate realization of the importance of SCB by the Bank’s operational departments, in spite of the need to monitor the implementation of the CAS, means that there is a need to redouble efforts on advocacy for statistical development among non-statisticians. 31. There is also a need to revisit the Bank’s engagement following the decision to phase out DGF funding for PARIS21’s core program. SCB funding at the country level faces important challenges of donor coordination that the Bank could help address, based on the experience it has developed over time. Bilateral funding for SCB will most likely decline as OECD governments are faced with fiscal consolidation, and current funding for SCB has been concentrated in a small number of countries, excessively fragmented within those countries, and weakly aligned with NSDSs. 32. More attention should be given to training and to the quality of statistics in order to more actively involve the users of statistics. The evaluation of the TFSCB highlighted concerns expressed by national statistical authorities for more effort on staff training to encourage the demand and use of data notwithstanding the fact that other bottlenecks will also need to be addressed. Consultations carried out for the IEG Review reveal that the shortage of local staff and the high demand on their time has led to their preference to attending meetings — where travel expenses are reimbursed — rather than doing their regular work. PARIS21’s work with the African Group on Statistical Training (AGROST) is an example of work to be promoted. LESSONS 33. Effectiveness requires explicit strategies for achieving outcome objectives. The focus on helping strengthening national statistical systems has provided the common ground for good collaboration between the three SCB programs. Good progress has been achieved in helping countries design NSDS which led to most IDA-eligible countries having prepared an NSDS. But without a clear road map and related operational strategies, implementing the NSDSs has proven to be a major challenge. xxix 34. Some selectivity may be needed to make progress. MAPS’ ambitious target that all countries should have prepared strategies for the development of their statistical systems by 2010 and have capacity in place in order to monitor progress towards the MDGs by 2015 has helped set a clear focus on NSDSs. Looking forward, a more selective approach with more intensive support to selected countries may be a more realistic approach, along the lines of the pilot countries under the new Statistics for Results Facility (SRF) — a multidonor trust fund established in the Bank in 2009 to increase the level of investment in statistical systems in developing countries. 35. Coordinated financial support across donors for statistical capacity building at the country level is important for moving the agenda forward and could benefit from documenting and sharing different approaches widely. A basket of funds approach such as what is currently being attempted in the case of Rwanda or a PRSC-type support for statistical capacity development as in the case of Lesotho are different ways to move away from a project-type approach to country-led capacity building. 36. The awareness gap on the need for statistical development between DECDG’s professional statisticians and the Bank’s operational staff may require stronger advocacy efforts inside the Bank. Consultations for the IEG review revealed that there continues to be a significant awareness gap between DECDG staff with more focused attention on broader statistical development based on NSDSs and the Bank’s operational staff with a more restricted view on monitoring of CAS implementation with little emphasis on the broader capacity building approach based on the NSDS. In addition, the monitoring of the MDGs by specific units in the Bank or in the UN system has, to some extent, relieved the country’s operational staff from a more holistic perspective on statistics. For reasons similar to those which justify the efforts by PARIS 21 on advocacy for SCB in developing countries, DECDG may consider playing a more active role on advocating for the importance of statistical development at the country level among the Bank’s operational staff. 1 1. Programs’ Objectives, Activities, and Costs Programs’ Origins and Objectives 1.1 Origins. The Partnership in Statistics for Development in the 21st Century (PARIS21) was founded in November 1999 in response to a call by the UN Economic and Social Council (ECOSOC) for better co-ordination in the area of statistical capacity building (SCB) in the context of meeting the goals of the UN Conference on Development. The PARIS21 Consortium was set up as a global partnership to act as a catalyst for promoting a culture of evidence-based policymaking and monitoring in all countries, and especially in developing countries. This new initiative was based on two key principles: (a) a partnership between developing and developed countries and multilateral organizations and between the providers and users of statistical data and (b) the process had to be driven by developing countries themselves if it was to be sustained. 1.2 The Trust Fund for Statistical Capacity Building (TFSCB) was established in 1999 to complement PARIS21 activities. Set up as a World Bank-administered, multi-donor trust fund, the TFSCB provides grants to developing countries for SCB activities. Created several years later, the Marrakech Action Plan for Statistics (MAPS) emerged from the Second Round Table for Managing for Development Results in 2004 as a global plan for improving development statistics based on an informal partnership involving developing countries, donors and statistical agencies worldwide. MAPS set two key target dates: 2010 by which countries should have prepared strategies for the development of their statistical systems and should have improved the availability of key indicators, and 2015 when capacity should be in place in order to monitor progress towards the MDGs. 1.3 Goals, objectives and linkages. Since their inceptions in 1999, PARIS21and the TFSCB have been two parts of a common effort to build a culture of evidence-based policy making, with PARIS21 promoting dialogue and advocacy, and TFSCB providing financial and technical resources. With the advent of MAPS in 2004, the focus of this collective effort has become more specific, namely, to monitor progress toward the MDGs and implementation of poverty reduction strategies. One key objective of MAPS has been to ensure that all low income countries have an integrated and comprehensive plan for the strategic development of the national statistical systems by 2010, thus allowing them to put in place systems to monitor progress towards the MDGs. MAPS provided the framework for the identification of such goals to guide collective efforts in statistical capacity building. With the broad endorsement of MAPS actions, PARIS21 has received substantial funding to facilitate achieving MAPS objectives. The TFSCB has also been an important instrument for implementing the MAPS. Table 3 highlights the complementarity in the objectives of the three programs. 1.4 Linkages to Other Programs. Building on the experience related to SCB in developing countries and more particularly on the recognition that the lack of resources is a major limiting factor to the implementation of national statistical strategies and that SCB needs a comprehensive approach have led to the establishment of the Statistics for Results Facility (SRF). As a multi-donor initiative designed to increase the level of investment in statistical systems in developing countries and to improve the effectiveness of financial and 2 technical assistance, the SRF has begun its pilot phase in five countries. While not part of this review, discussions of the SRF will provide useful insights as to issues of sustainability of the effort to strengthening national statistical systems in developing countries. Table 3. Programs’ Goals and Objectives PARIS21 TFSCB MAPS Higher-Order Goals To develop a culture of To improve the capacity of To improve development management for development developing countries to compile statistics with two key target results. Subsequent to the and use statistics with the dates: 2010 when most setting up of MAPS, PARIS21’s overall objective of supporting countries would have prepared work program has been the management of NSDSs, and 2015 when modified overtime to address development results. From the capacity would be in place in MAPS’ objectives. beginning, TFSCB has been order to monitor progress closely aligned with PARIS21. towards the MDGs. Since 2004, TFSCB has been an important instrument for implementing MAPS. Specific Objectives To encourage and support To support the preparation of To help develop and strengthen developing countries to design, NSDSs, in line with MAPS and national statistical systems implement, and monitor working with PARIS21 to through 6 specific actions: National Strategies for the ensure that all LICs have an (a) prepare national strategies Development of Statistics integrated and comprehensive for development of statistics for (NSDSs) including: plan for the strategic all LICs by 2006; (b) ensure full (a) mobilizing resources for the development of their national participation of developing implementation of NSDSs; statistical systems and detailed countries in the 2010 census (b) coordinating donor support capacity building programs for round; (c) increase financing for to statistics; (c) coordinating all statistics. SCB; (d) set up an International actors within the National Household Survey Network; Statistical System; (e) undertake urgent (d) producing guidance and improvements to monitor the documentation; and MDGs and other development (e) providing technical goals; and (f) increase assistance (legislation, training, accountability of the human resources, etc.). international statistical system. 1.5 In addition to hosting the TFSCB, the World Bank also provides financing to investments in statistical capacity. Under a multi-country funding mechanism for SCB — STATCAP — established in 2004, countries can access investment projects (supported by loans, credits, or grants) to finance the improvement of statistical capacity. Typically, a STATCAP project is designed to support a long-term approach to SCB using the NSDS as the essential framework which provides the rationale for the proposed investment. Thirteen STATCAP projects have so far been approved. Activities 1.6 The three programs under review cover a range of core activities, each intended to promote the overall objectives to varying degrees (Table 4). Under MAPS, the World Bank’s 3 Development Grant Facility (DGF) provides financial support for all programs of activities associated with PARIS21. The DGF also provides financial support to the UN Statistics Division (UNSD) for support to developing countries in preparing for the 2010 census round. The TFSCB provides small grants of up to $400,000 1 to developing countries to help the strengthening of statistical systems ranging from strategic planning focusing on NSDSs to sectoral statistics, survey programs, and regional/global seminars and workshops. Activities under the TFSCB are closely coordinated with the work of PARIS21 and have been one of the key instruments aimed at implementing some of the priority actions highlighted in the MAPS, that is, to mainstream strategic planning by countries and to increase financing for SCB. Table 4. Programs’ Activities MAPS PARIS21 TFSCB Under MAPS, the World Bank’s • NSDS country-based activities: TFSCB provides small Development Grant Facility has support design and grants of up to US$ provided financial support for implementation of, and funding 400,000 to developing programs of activities associated for NSDSs; provision of technical countries to implement with PARIS21: assistance and statistical tools capacity building • Strategic planning for improving and guidelines. projects. Funding is statistical systems in developing • Advocacy: help statisticians in provided through two countries (NSDS); developing countries with separate windows: advocacy work on strategic (a) preparation, • More efficient approaches to the statistical planning. updating, and revision of conduct of household surveys • Donor collaboration: support the NSDS; (b) promoting through the IHSN; and coordination of statistical implementation of NSDS • Improved practices of data and capacity building in collection, management and capacity development through annual Partner Reports on specific priority sectors. dissemination in developing countries through the ADP. Support to Statistics (PRESS). Through the DGF, MAPS has also • Support for survey data archiving provided financial support to: through the international household survey network • UNSD to enable it to better (IHSN). prepare IDA countries for the • Support for quality of survey data 2010 Census Round and gender (collection, assessment, statistics in informal employment. documentation, and • UN-Habitat’s program to improve dissemination of existing micro- urban indicators. data) through the Accelerated • UNESCO Institute of Statistics Data Program (ADP). for the Program for Education Statistics. • UNECE for the Gender Statistics Program in Southern and Eastern Europe. 1.7 The diverse activities undertaken by PARIS21 can best be presented under five categories covering: (a) NSDS country-based activities: (b) advocacy; (c) donor collaboration; (d) support for surveys; and (e) support for quality of survey data. 1. This amount has since been increased to $500,000. 4 (a) NSDS Activities. The PARIS21 Secretariat has provided technical assistance missions to about 40 countries, normally in the context of developing an NSDS. The Secretariat also supported the preparation of regional activities in Africa and Central America. 2 Presently, 72 of the 79 least developed countries classified as IDA-eligible countries are in the process of implementing, designing or awaiting adoption of an NSDS. In addition, PARIS21 also helped to improving statistical tools through the production of guidance and documentation on strategic statistical planning known as NSDS Guidelines. The Secretariat finances attendance of national statisticians in international meetings which also contributes to widening engagement of the Partnership. (b) Advocacy Activities. In addition to technical assistance provided for the preparation and implementation of NSDSs, PARIS21 also helps national statisticians in some 30 developing countries in Africa, Asia, and Latin America with their own advocacy to convince policy-makers, donors, and other domestic stakeholders of the importance of statistics and, in particular, of the necessity for developing countries to have a well- prepared, adequately funded and successfully implemented NSDS. This is done through the production of country-level, sub-regional, or sectoral advocacy materials. (c) Coordination of Support for Statistical Capacity Building. Started in 2008, the yearly PRESS reports (Partner Report on Support to Statistics), which are based on surveys, provide a snapshot of ongoing support for statistics, and help provide information on what development partners are doing (and plan to do) in the statistical field, identify countries or areas of statistics in need of more support, and raise the profile of statistics within the overall context of aid to developing countries. 3 According to the 2010 report, financial commitments from all sources for statistical development in developing countries for the period 2008–10 amounted to roughly $1.6 billion. This includes activities committed prior to, but still active during, the reporting period. Africa received nearly half of total support for statistics, equivalent to $710 million in commitments. (d) International Household Survey Network (IHSN). As one of the MAPS’ key recommendations, the IHSN was established in September 2004 with a small secretariat at the World Bank to help promote the coordination of international household survey programs, enhance international standards and best practices, harmonize data collection instruments, and foster better use of existing survey data. Since April 2006, the IHSN has been implemented as a PARIS21 program which focuses on two main activities namely, the development of tools and guidelines for data archiving and the harmonization of international survey methods and instruments. These include the Microdata Management Toolkit, the National Data Archive Application (NADA), and the Question Bank Application; and 2. Including the preparation of the Reference Regional Strategic Framework for Statistical Development in Africa (RRSF) and assistance provided to the Central American Integration System (SICA). 3. Although invaluable as a source of information, more efforts are needed to enhance the coverage of the survey. Out of 101 partners contacted by PARIS21 for the 2010 PRESS exercise, only 63 have replied. Among the non-respondents are bilateral agencies from non-OECD and new OECD member states, foundations, regional institutions (primarily in Africa), statistical training schools, and several UN agencies. 5 (e) The Accelerated Data Program (ADP) was launched in 2006 also as a recommendation of MAPS to undertake urgent improvements needed for monitoring the Millennium Development Goals (MDGs) by improving the use and value of survey data. With PARIS21 as the main implementing agency and the World Bank providing global coordination and the bulk of funding (through DGF), the ADP is currently supporting agencies in 56 developing countries and has received requests for support from a dozen other countries. PARIS21 provides support for the inventory, documentation, and dissemination of existing micro-data. But the lack of resources has prevented the implementation of ADP’s of other objectives, namely the assessment and analysis of existing data, the improvement of national survey programs, and data collection. 1.8 TFSCB finances two main kinds of projects. The first provides technical and financial support to the preparation of NSDS. The second type of project, also known as non-NSDS projects, supports capacity building in specific priority sectors. These projects may target the implementation of one or more critical components of NSDS. All non-NSDS projects must be based on a formal assessment of the statistical system and must demonstrate how they will address the main capacity weaknesses. Since 2004, the TFSCB has been the main mechanism for supporting the preparation of national strategies for the development of statistics, a key element of the MAPS. Over the past ten years, the Trust Fund has committed more than $35 million to projects covering more than 80 countries. Figure 1. Organizational Chart – MAPS Source. DECDG Web site. 6 Governance and Organization 1.9 MAPS. Because it was conceived as the plan, governance and management issues were not important considerations and the structures were kept at a minimum. DGF funding, which started in FY06, led to the need to create an international MAPS Advisory Board to provide strategic directions, promote coordination, and monitor implementation and the internal Statistical Capacity Building Committee (SCBC) to ensure better coordination among SCB initiatives within the Bank and to oversee the implementation of the DGF grant. A Coordination and Monitoring Unit was set up within the Development Data Group in the World Bank to support both the Advisory Board and the SCBC (Figure 1). 1.10 PARIS21. As a Partnership, PARIS21 has a well configured multi-tiered governance structure (Figure 2). The activities of the Partnership are carried out by a Secretariat located in the Organization for Economic Co-operation and Development in Paris. The strategic direction of the partnership and the general work program of the Secretariat are guided by the Steering Committee, an international group of stakeholders representing different developing regions, bilateral donors, and multilateral institutions. The Steering Committee is co-chaired by a member of the Bureau of the UN Statistical Commission (UNSC) representing developing countries and the chair of the OECD Development Assistance Committee (DAC) representing the donor community. The Steering Committee has a Bureau that provides guidance and support to the Secretariat between Steering Committee meetings. On behalf the Steering Committee, the Secretariat reports annually to the DAC and UNSC. Figure 2. Organization Chart – PARIS21 (current structure) Annual Progress UN Statistical Commission Reports OECD/DAC Partners - Developing countries Board Donors - Regional banks - Bilateral donors - Regional institutions - International agencies - Bilateral donors - Foundations - - Other contributors Multilateral organizations Executive Committee - Other institutions Secretariat Source. Constructed by IEG 1.11 At the recommendations of the special committee to review PARIS21’s governance and logical framework, a new governance structure was adopted in June 2010. First, the Steering Committee was strengthened to a Board, with the responsibility to guide the work of the Partnership between Consortium meetings 4 and hold its members accountable for the 4. The last PARIS21Consortium meeting that brought together all PARIS21 partners took place in November 2009 in Dakar, Senegal. The next Consortium meeting is scheduled for 2014. 7 results of the Partnership. To give a stronger voice to developing countries, the new Board which will have an extended membership. The Bureau became the Executive Committee with a strengthened role in providing an accountability mechanism and guidance to the work of the Secretariat, in addition to the responsibilities of the (former) Bureau. 1.12 Simultaneously with the reforms in its governance structure, the PARIS21 Partnership is also revising its logical framework, to be made more coherent with the Consortium’s 2009 Dakar Declaration on the Development of Statistics and is structured to be more results-oriented, with baselines, targets and milestones for all indicators related to its the goal, purpose and outputs. 1.13 TFSCB. The TFSCB is governed by a 12-member Consultative Group, which includes the five “founding fathers� of PARIS21. Given the strong complementarity between PARIS21 and the TFSCB, the yearly meetings of the Consultative Group take place jointly with the PARIS21 Steering Committee (now the Board) thus assuring good exchange of information and more importantly, the participation of representatives of developing countries. Day-to-day management of the TFSCB is provided by the Trust Fund Administration Unit. The Internal Management Committee (IMC), which is chaired by a DECDG Manager, reviews project proposals and decides on funding. An external Advisory Panel of two experts reviews all IMC decisions on TFSCB proposals and provides advice on the future direction of the TFSCB. The annual report by the Advisory Panel is submitted to the Consultative Group (Figure 3). Figure 3. Organization Chart – TFSCB Consultative Group Consultative Group Yearly technical review report Six-monthly reports Donors Advisory panel Trust Fund Administration Unit (TFAU) Funding - Window 1: NSDS projects Internal Management - Window 2 non- Internal Management Committee (IMC) NSDS projects Committee (IMC) Source. Constructed by IEG 8 Monitoring and Evaluation Frameworks 1.14 While the external evaluations of all three programs` tend to show that there has been progress toward outputs and outcomes in all three programs, the more meaningful monitoring and evaluation question is whether there has been progress toward the goals and objectives and, more specifically, whether efforts to strengthen statistical capacity are leading to better evidence-based policies in relation to poverty alleviation and other MDGs. Given the continuing challenges in measuring progress toward goals and objectives, Box 1 provides a synopsis of frameworks in all three programs for measuring progress toward outputs and outcomes. Box 1. Monitoring and Evaluation Frameworks – MAPS, PARIS21, AND TFSCB MAPS. The yearly DGF reports provide detail on fund allocations and activities made possible by DGF funding. As a fund provider without direct involvement in SCB activities, the DGF/MAPS relies on information originating from recipient agencies and organizations for results. The monitoring and evaluation framework of PARIS21, which is by far the largest DGF fund recipient, is discussed below. PARIS21. The yearly reports provide detailed information on progress with a focus on the achievement of specific actions and activities. The earlier log frame, which was revised in 2008 and has recently been replaced by a more results-based version, did not allow for an easy evaluation of the PARIS21 work program in terms of impacts due to a lack of verifiable indicators. The regular update on the status of NSDS processes — showing countries currently implementing their national strategies; designing or awaiting adoption of strategies; with strategies expired or planning an NSDS; and without an NSDS or not planning one — do not provide sufficient information to evaluate progress on this front given the continuing concern about weak implementation. TFSCB. The twice yearly reports provide updated information on allocation of funds, breakdowns of TFSCB projects by region and by project types (NSDS-related or other non-NSDS aspects of SCB and improvement). The yearly reports of the TFSCB Advisory Panel provide an external analysis of project proposals received by the TFSCB Administration Unit and decisions by the Internal Implementation Committee. At the project level, the task team leaders (TTLs) are required to submit a progress report every six months using the grant reporting and monitoring system (GRM). Currently, the GRM allows TTLs to rate progress toward project objectives as well as overall progress. In general, these reports show good performance but the fact that they reflect self- assessment must be kept in perspective. Sources: Compiled from various sources by IEG. 9 2. External Evaluations of the Programs 2.1 All three programs have been subject to independent evaluations in recent years — in 2008 for MAPS, and in 2009 for PARIS21 and TFSCB. This review is informed mainly by these recent evaluations but references will be made, where relevant, to previous evaluations in 2003 and 2006 for PARIS21 and in 2003 and in 2008 for TFSCB — the latter being an internal evaluation by one of the principal donors, DFID. In spite of the common objectives of the three programs under review, the evaluations were conducted for different purposes using different approaches and therefore require a separate discussion for each of the three evaluations. This review will also highlight complementarity among the three programs and will offer some useful comments about SCB, going beyond the scope of each of the three individual programs themselves. Independence of the Three Evaluations 2.2 MAPS. At the request of the DGF Council for an independent evaluation to inform its decision whether to extend MAPS funding, the Development Data Group (DECDG) commissioned the 2008 evaluation on behalf of the MAPS Advisory Board. The MAPS evaluation was conducted in two phases. Phase I consisted of a fact-finding review to assess the performance of the MAPS partnership — including the MAPS Advisory Board, the recipients of grants provided by the DGF (notably PARIS21, UNSD, UNESCO Institute for Statistics, UN Economic Commission for Europe, and UN Habitat) and other key instruments which support MAPS, particularly the Trust Fund for Statistical Capacity Building (TFSCB). Phase II concentrated on visits to four countries in Africa — Ethiopia, Malawi, Mali and Niger — to evaluate the formulation of NSDSs, country ownership, and obstacles to their implementation. In line with standard requirements of the DGF Council, independent consultants were selected competitively. The same consultants were awarded contracts for both phases of the evaluation. The terms of reference were reviewed and agreed by the World Bank (internal) Statistical Capacity Building Committee (SCBC), the GPP group, grant recipients, and the MAPS Advisory Board, and shared with IEG. 2.3 PARIS21. The evaluation was commissioned by the PARIS21 Secretariat on behalf of the Steering Committee as part of its monitoring and evaluation of the Partnership’s program. The primary purpose of the 2009 evaluation was to evaluate progress towards the Partnership’s goal and purpose on the use of statistics to inform development policy and to develop a culture of evidence-based decision-making. The 2009 evaluation built on previous evaluations, and provided the basis for recommendations on the future of the PARIS21 Partnership and its Secretariat following the 2007–2010 work programs. It also fed into discussions at the Partnership Consortium meeting in Dakar, Senegal, in November 2009. 2.4 A joint Reference Group — comprising the PARIS21 Bureau, representatives from the OECD/DAC Evaluation Department and the World Bank — was set up to oversee the evaluation. This includes agreeing on the terms of reference, overseeing the recruitment of the evaluation team — through a tender process managed by the OECD — and approving the evaluation team’s inception and final reports. 10 2.5 Since its establishment, PARIS21 has been evaluated three times. The Partnership and Secretariat were evaluated twice together, first in 2003 and again in 2009. The Secretariat was also reviewed in 2006. The 2003 independent evaluation assessed the progress of the partnership since its inception in 1999 and concluded that, based on progress achieved and the growing data needs from poverty reduction strategies and the monitoring of MDG achievements, PARIS21 should continue for an additional three years until 2006. In 2006, a sub-group of Steering Committee members — aided by an external facilitator — carried out an internal evaluation of the impact, relevance, effectiveness, and efficiency of PARIS21for the years 2004– 2005. And again, based on the success of the work and PARIS21’s adaptive approach to evolving needs, the Steering Committee authorized the extension of the partnership beyond 2006. 2.6 TFSCB. The Consultative Group of the trust fund decided in early 2009 that an independent evaluation of the TFSCB should be undertaken in the course of the year to coincide with a similar exercise for PARIS21. The evaluation was to review progress achieved so far and to help develop a coordinated strategy with PARIS21 for the period 2010–2015. 2.7 More specifically, the evaluation aimed to: (a) Assess the impact of the TFSCB in supporting the preparation of NSDSs and in helping to strengthen the capacity of national statistical systems; (b) Identify what changes might be needed in the design of the TFSCB or in the ways in which it operates to improve its effectiveness and impact in the future; and (c) Recommend what changes might be needed to enable the TFSCB to better assist countries in the preparation of NSDSs but also their implementation. 2.8 As was the case for PARIS21, the TFSCB has been subject to several reviews. The 2003 independent evaluation undertaken in conjunction with the 2003 evaluation of PARIS21covered topics such as the relevance of the trust fund to SCB needs and the emerging procedures of the Fund. Given that no projects were completed in 2003, the evaluation only attempted very tentative assessments of the impact of the TFSCB’s portfolio of projects and the sustainability of individual projects. In 2008, the Department for International Development (DFID) undertook a light review exercise of its own support to the TFSCB, primarily to meet internal DFID reporting requirements focusing on strategic issues and management and funding mechanisms. 2.9 In summary, given that the three programs under review are funded out of program funds held in trust by the host organizations — the OECD for PARIS21 and the World Bank for MAPS and the TFSCB — tendering and contracting with consultants followed well established procedures by the two host organizations. But the fact that the respective secretariats commissioned the evaluations rather than acting simply as agents of the governing bodies and that they had control over funds used for the evaluations ran the risk of compromising the organizational independence of the evaluations. 5 Consultations for this 5. To preserve the organizational independence of the evaluation, IEG recommends that the programs’ governing bodies should commission the evaluation, approve the evaluation terms of reference, select the evaluation team (ideally using competitive methods), and receive the final evaluation report — or establish an oversight subcommittee for these purposes. If the governing body does not have the institutional capacity to undertake all these things independently of the management (secretariat) of the program, then the secretariat 11 Table 5. Evaluation Framework of the Three Evaluations Character- istics MAPS PARIS21 TFSCB Objectives and The evaluation focused The evaluation reviewed The evaluation assessed strategies on assessing the overall progress towards the impact of the Trust impact of the various PARIS21’s goal Fund in helping to deliverables funded by (developing an evidence- strengthen the capacity of the DGF on the based management) and national statistical systems development purpose (improved and to identify changes management of recipient capacity of national needed to improve its countries. statistical systems in effectiveness and impact. developing countries). Use of results The evaluation assessed The evaluation closely The TFSCB’s logframe framework the extent to which MAPS followed the results was used to design the partnership activities framework set out in the evaluation framework, and have effective monitoring PARIS21 2007–2010 particularly, the and evaluation of Logical Framework questionnaires. But unlike activities funded by the (logframe) that was the PARIS21 evaluation DGF. revised in June 2008. exercise, there was no attempt to assess results using the means of verification. Use of Trend analysis of World Systematic use of the The assessment is measurable Bank Country Statistical means of verification of primarily qualitative with indicators Information Database the logframe. little or no references to (CSIDB) in IDA countries measurable indicators. since the DGF funding started in 2005. Data collection MAPS activities in IDA Evidence relating Response on structured countries. Improvements PARIS21 activities from questionnaires from 29 in MDG monitoring. various documents such implementing authorities of as MDG Reports, projects, 16 task team PRSPs, and other UN leaders and members of documents. the Internal Management Committee, Trust Fund donors, and other Consultative Group members. Coordination The original terms of The two evaluation teams met in Durban, agreed on across reference called for the purpose and expected outcomes of coordination, evaluations including review of and exchanged documentation and relevant TFSCB evaluation — information, including draft reports. which was unfortunately completed too late for the MAPS evaluation exercise. should act purely as an agent of the board in these respects. Even if the organizational independence is somewhat compromised, the evaluation may still be independent if behavioral independence is sound, that is, if the evaluators behave independently of management notwithstanding the challenges of doing so. 12 review, however, show that the selection of evaluation teams and the approaches followed for the three separate evaluations indicated that issues of compromised organizational independence did not arise or were not significant. Furthermore, the reviews were not hindered by factors such as access to information, inadequate budget, or any other restrictions. Quality of the Three Evaluations EVALUATION FRAMEWORKS 2.10 With MAPS being the overall framework program and a major funding mechanism for SCB through the DGF, its evaluation rightly focused not on program details but on the program’s overall contribution to the impact on development, and especially to the quality of decisions by governments and others about broader development policies and programs. A list of specific questions was included in the terms of reference including specific references to monitoring and evaluation frameworks of the various programs that receive DGF funding support such as the extent to which they have effective monitoring and evaluation mechanisms, including measurable performance indicators, and whether these indicators have improved since 2005 when the DGF MAPS program started (Table 5). 2.11 The evaluation frameworks for PARIS21 and the TFSCB were more specific. In the case for the PARIS21 evaluation, the framework followed closely the Partnership’s logical framework that was revised in 2008. The PARIS21’s evaluation also undertook separate assessments of the Partnership’s programs and of the Secretariat’s activities and outputs. For the Trust Fund, besides outcomes and impact, the evaluation report also focused on TFSCB project management and governance. APPROACH AND SCOPE 2.12 In terms of scope, the evaluations of the three programs were broad. For MAPS, the evaluation covered all the relevant issues including relevance, effectiveness, cost efficiency, resource mobilization, sustainability, and governance and management. Resource mobilization issues were not covered in the evaluations of the two other programs. Also for the TFSCB, the evaluation did not assess issues of cost efficiency, given the small size of the TFSCB grants which were limited to $400,000 until the recent decision to raise the ceiling to $500,000. The trust fund evaluation also assessed the links between TFSCB grants and commitments by the recipient governments’ use of their own domestic resources to continue Table 6. Details on the Scope of the Evaluations Scope of the evaluation MAPS PARIS21 TFSCB Relevance    Efficacy    Efficiency or cost-effectiveness   n/a Governance and management    Resource mobilization and financial management  n/a Sustainability    13 implementing the actions prescribed by the NSDS; these links were found to be weak. Details on the scope of the evaluations are summarized in Table 6. EVALUATION INSTRUMENTS 2.13 The evaluations of the three programs used a combination of qualitative and quantitative evaluation instruments including document reviews, consultations and interviews, structured surveys and questionnaires, field visits, and case studies. A total of 16 countries were visited by the three evaluation teams in addition to face-to-face meetings with representatives from 9 other developing countries and the Russian Federation at the 57th session of the International Statistical Institute (ISI) in Durban, South Africa. Eighteen desk studies were developed, 12 for developing countries and 6 for donor countries (Table 7). OVERALL QUALITY OF THE EVALUATIONS 2.14 As shown in the previous discussion, the evaluations’ frameworks, scope and approach, and use of the evaluation instruments appear to have taken into account all relevant information available at the time of the evaluations. This IEG review has identified a number of weaknesses in the evaluations which are summarized below: (a) More systematic cross-references of results of the analyses covered in the 2008 MAPS evaluation in the assessments of the PARIS21 Partnership and the TFSCB (done in 2009) could have been useful. For instance, the survey questionnaire used in the TFSCB evaluation sent to national implementing authorities could have been more specific on the linkages between the NSDS process and the use of statistical information in the country; (b) While all three external evaluations clearly state effectiveness of the programs in their terms of reference, the focus was predominantly on processes and activities and insufficient emphasis was given to outputs and outcomes. As a result, the evaluations have not done much stakeholder analysis -- policymakers, line ministries, parliaments etc. -- on the demand for statistics. This issue is now being looked at more systematically, particularly in the context of PARIS21; (c) While the focus on processes and activities may be justified by technical and conceptual challenges, the evaluations could have identified concrete ways in which the programs have contributed to successful change and improvement in statistics and statistical capacity. Possible options include learning from ‘what has worked or not worked’ from the case studies, and cross-regional comparisons of practices and outcomes; (d) While all three evaluations share the common concern on the inadequate implementation of NSDSs, they did not provide operationally useful insights on how or to what extent NSDSs have helped with the development of national visions for statistical development, and on the role and impact of advocacy for more focus on statistics. Lastly, a sharper focus on the notable lack of progress in the use of statistics in sub-Saharan Africa could have led to more specific recommendations for this group of countries; and 14 Table 7. Details on Evaluation Instruments Used Instruments MAPS PARIS21 TFSCB Desk and MAPS key documents. In addition to key documents, PRSPs Key documentation on the document and MDG Country Reports, follow- TFSCB and its projects. reviews up reports to Roundtables on Managing for Development Results and the Paris Declaration on Aid Effectiveness and reports of the UNSC. Consultations Visits to all major Consultations with 12 international Structured consultations with DGF/MAPS recipients organizations including the 3 largest members of the Internal (PARIS21, UN Statistics contributors to PARIS21 (World Bank, Management Committee, TF Division, and UNESCO European Commission, and donors and other Consultative Institute for Statistics). Eurostat); country offices of donors in Group members. Interviews and/or survey 6 recipient countries; members of Meetings with representatives questionnaires of MAPS Steering Committee; and major users of FAO, UNECA, UNECE and Advisory Board members. of developing country statistics. SADC. Structured Mail questionnaire to several national The evaluation team designed surveys statistical offices (NSOs). four questionnaires targeting implementing authorities of projects (primarily NSOs), task team leaders, members of the IMC, and Trust Fund donors and other CG members. Country visits The evaluation team The evaluation team visited 7 of the The evaluation team visited visited Ethiopia, Malawi, 12 countries with desk studies (see five countries with TFSCB Mali, and Niger as part of below) plus two others. It also projects and, as part of the Phase II of the evaluation attended the 57th session of the collaboration with the PARIS21 to seek the views of International Statistical Institute (ISI) evaluation team, also attended developing countries to in Durban, South Africa where it met the Durban event. the MAPS Initiative. In with representatives from 10 addition, the visits allow countries. for a better feel by the Visited five donor countries including evaluators on the degree two of the six covered by desk studies of ownership of the (see below). NSDSs and obstacles to their implementation. Case studies Desk studies for 12 countries analyzing references to statistics, improvements in MDG indicators, and coverage of statistical issues in government Web sites. Desk studies for 6 donor partners assessing use of policy-relevant statistics in country assistance strategies; and coordination at the (recipient) country’s level. (e) The external evaluations could have made more emphasis on the fact that the programs did not set out clearly defined and measurable baselines and targets. While MAPS may be in a different category, since it is a financing and coordinating 15 mechanism, PARIS21 and the TFSCB programs implemented specific programs, projects, and activities, and baselines and targets at least around outputs and intermediate outcomes would have been useful. Evaluation Findings, Recommendations, and Feedback 2.15 MAPS. The evaluation report showed that the DGF-MAPS funding has yielded useful outputs and potentially useful statistical tools (toolkits for household surveys, 2010 World Census Program of the UN, support for education, gender, and urban statistics with a strong potential for replicable results). The NSDS program managed by PARIS21 has demonstrated its utility, and there has been increasing country coverage. While the implementation of NSDSs remains a major challenge, one can safely say that the NSDS processes have helped many governments develop a strategic vision about statistical development. But building statistical capacity faces the same constraints, such as weak state institutions and the lack of personnel and resources, as building capacity elsewhere and will require time and sustained effort. 2.16 The evaluation report’s main recommendations covered: • Funding: extend current funding levels for the main programs to at least 2012. The evaluation team did not, however, assess compliance with the DGF’s 15 percent funding leverage guidelines; 6 • Evaluation: improve the specification of development objectives and the means of verification in DGF-supported programs; • Effectiveness: NSDS implementation must remain the top priority for MAPS; work on accelerating the emergence of country success cases; promote training in data use and interpretation among key government staff; • Governance and management: expand the Advisory Board’s role in providing strategic advice and direction; widen the World Bank (internal) Statistical Capacity Building Committee’s (SCBC) coordinating functions; and • For the World Bank: give fuller attention to statistical capacity and the use of statistics in decision making in IDA Country Assistance Strategies. 2.17 The MAPS Advisory Board reviewed the recommendations of the evaluation in its 2008 meeting and requested that the World Bank review the actions of MAPS and make proposals for expanding the role and scope of the Bank’s internal SCBC, and for enhancing the role of the Advisory Board. The roles and responsibilities of these bodies should take into account governance arrangements for other partnerships, such as PARIS21 and the new Statistics for Results Facility. 2.18 Internally to the World Bank, a decision has been made to widen the role of the SCBC. First, the membership of the committee has been expanded to cover more regions and networks. Second, the committee has broadened its coordinating role to cover besides DGF/MAPS which was its main focus until recently, the SRF. On DGF financing, support for MAPS would continue but given the considerable demand (and hence competition) for funds, support in future years would likely be more limited and might end after 2013. A more 6. One of DGF’s eight eligibility criteria states that DGF grants should not generally exceed 15 percent of expected funding over the life of Bank funding to the program. 16 detailed discussion of MAPS DGF financing will be covered under Section 5 of this report on the role of the World Bank. 2.19 PARIS21. Progress has been made over the years in achieving the Partnership outputs and outcomes, namely on the design and preparation of NSDSs, the coordination across national statistical systems (NSSs) and between governments and donors, and assisting in the provision of better survey data. But the implementation of NSDSs remains a major challenge. The overall movement of the country statistical capacity indicators suggested an improvement in statistical capacity, although improvements in Africa have been much more modest compared to other developing regions. 2.20 The evaluation also confirmed that the PARIS21 activities and outputs have, overall, been effective in contributing to the progress towards achieving the Partnership’s goal, purpose, outcomes, and outputs. Interviews with national statistical offices (NSOs) done for the evaluation indicated that the PARIS21 Secretariat has provided critical and effective assistance to the launching, preparation and development of NSDSs, through technical assistance and advocacy efforts. But, as mentioned in the evaluations of MAPS and TFSCB, the implementation of NSDSs has revealed to be an important challenge due to a variety of factors including the lack of resources and weak technical capacity. 2.21 The recommendations of the report addressed two groups of issues. The first group covered strategic issues including: (a) redirecting government efforts on statistics to focus more on improving access to policymakers; (b) intensifying the Partnership’s efforts to promote and support coordination of the national statistical systems (NSSs); (c) expanding support in promoting coordination among donor partners at the country level; (d) making a more selective approach to country coverage to avoid spreading too thinly across countries. 2.22 The recommendations on technical issues focused particularly on: (a) improving the work on NSDSs and more particularly on guidance on the developing of realistic statistical development plans to facilitate and enhance their implementation; and (b) Encouraging partner countries to include in their country assistance strategies specific statements of intent and to work within the NSDS of recipient countries. 2.23 The majority of the suggestions by the evaluation team has been more or is being taken into account in the revision of the PARIS21 Logical Framework for 2010–2014 which is being finalized. Using 2008–09 as the baseline, targets will be set for various quantitative indicators for 2014. Some of the indicators that which directly address the evaluators’ recommendations are summarized below: • Percentage of countries that have developed sound frameworks for monitoring development results; 17 • Increase in the average World Bank Statistical Capacity Indicator score of IDA eligible countries (with a population of 1 million or more) with a score of less than 65 in 2009; • Number of countries having a Statistical Committee led by a Minister or similar ranking and number of sectors covered by NSDSs across countries; • Percentage of countries that are implementing an NSDS and of countries having a NSDS that scores more than 75 percent of the total score defined by the NSDS Quality Assessment Framework; and • Share of aid to statistics in total aid over a 3-year period starting from 2007–2009 (0.19 percent for 2006–08); and • Percentage of countries having country-donor coordination mechanisms in the field of the development of statistics perceived as satisfactory or better. 2.24 TFSCB. The evaluation showed that projects financed by the TFSCB have been important for building the statistical capacity of developing countries and had a positive impact on both the generation and the use of statistics. The advent of new instruments to support SCB in developing countries, such as the new Statistics for Results Facility (SRF), did not reduce the need for the Fund’s continued operation at the present time. The TFSCB and new SRF will be complementary: TFSCB will provide resources on a limited scale to help countries implement priority actions and develop good-quality implementation plans, and the SRF will help mobilize large-scale investment resources needed to put those plans into action. The evaluation of the TFSCB also confirmed the concerns about the weak implementation of NSDSs and highlighted the risk that countries will not be able to maintain the momentum created during the design and elaboration of their strategies. The lack of funds was mentioned as the main obstacle and although the report made reference to other obstacles, these were not made specific in the findings. 2.25 The evaluation made several recommendations on the future operations of the TFSCB: (a) On funding, continue the mobilization of donor funding for the trust fund; (b) On priorities, address the issue of the implementation of NSDSs and raise the share of resources for the NSDS process in TFSCB to at least 60 percent; increase the maximum amount of the grants; (c) On TFSCB’s governance, enhance the transparency of governance arrangements; and (d) On project management and reporting, promote progress reports on projects and results-oriented completion reports; and increase awareness about the TFSCB among its potential clientele. 2.26 The evaluation has led to a number of decisions on the TFSCB including: (a) the increase in the size of the grants from $400,000 to $500,000; (b) further effort on strengthening linkages of new project proposals to the NSDS process therefore ensuring that well over 60 percent of projects will support the NSDS process; (c) new guidelines and a new prospectus on the TFSCB detailing the governance process are now available on the TFSCB external Web site; and (d) a new brochure has been produced to enhance the TFSCB’s visibility. 2.27 There is, however, no plan for the moment to make changes to how reporting by task team leaders is done. Instead, DECDG is in the process of initiating the evaluation of a sample of NSDSs to provide better guidance for future projects. 18 3. The Effectiveness of the Programs 3.1 This chapter presents the review of the three programs covering: (a) the relevance of the programs’ objectives and design for meeting the broadly recognized need to support SCB in developing countries and to strengthen evidence-based economic and social management; (b) the extent to which activities and outputs generated by the three programs have helped to advance their broad objectives; and (c) the extent to which the programs have been managed efficiently in terms of costs. Overall, the review finds that the relevance of the three programs is substantial, their record of outputs is high, their achievements of outcome objectives modest with some shortcomings, and that the programs are managed efficiently in terms of costs. Relevance of the Programs 3.2 The IEG review confirms the assessments of the external evaluations that the objectives of all three programs are highly relevant. Moreover, the relevance of the programs’ objectives has been sustained in spite of the completion of important actions by all three programs. 3.3 International consensus. PARIS21and the TFSCB were founded in 1999 as important contributions to the efforts to build a culture of evidence-based policy making in developing countries. These objectives were further sharpened in the aftermath of the launching of MAPS in 2004 with a focus on helping developing countries prepare and implement the NSDSs. In turn, PARIS21 and the TFSCB have also been important instruments for implementing the MAPS. 3.4 Other initiatives most notably the UN MDG Indicators Expert Group also contributed to the broad consensus of the need for improving statistics in developing countries. This consensus has been reaffirmed at various international forums including the Third High Level Forum on Managing for Development Results in Hanoi in 2007, the 2008 Third High Level Forum on Aid Effectiveness — also known as the Accra Agenda for Action (AAA) — and most recently the 2010 MDG Summit which called for fostering a global partnership to support countries in generating reliable and timely data so that so that they can assess progress on the MDGs and other country goals. 3.5 In Africa, the Reference Regional Strategic Framework for Statistical Development in Africa (RRSF), 7 which provided the overall framework for SCB including the African Development Bank’s (AfDB) support to the NSDS processes, is broadly perceived as an application of the MAPS framework to Africa. More recently, a new UN Working Group on 7. The RRSF was adopted at the second Forum on African Statistical Development (FASDEV) in 2004. FASDEV is a United Nations Economic Commission for Africa (UNECA) initiative co-sponsored by the AfDB, PARIS21, and the World Bank with the objectives to: share information about and create more synergy among the various statistical programmes for African countries; take stock of the resolutions and initiatives designed to scale-up the investment in National Statistical Systems (NSSs); and explore new financing sources for SCB in Africa. 19 Statistics in Africa 8 has been established with the aim of supporting the implementation of the RRSF in close relation with the newly created Africa Statistical Coordination Committee, which is composed of the AfDB, the Africa Capacity Building Foundation, the Africa Center for Statistics of UNECA, 9 and the African Union. It was also agreed that works by the UN Working Group on Statistics in Africa and by the MAPS be linked through the sharing of reports. 3.6 Beneficiary demand. The objectives and priorities of the three programs are well aligned with those of developing countries articulated in the countries’ PRSPs — where the importance of statistical development receives much attention, particularly in relation to the need for monitoring and evaluation — and with MDG reports and donor country assistance strategies. PARIS21 and the TFSCB have helped to initiate the need to improve the capacity of developing countries to compile and use statistics in the fight against poverty and in achieving the MDGs. The MAPS strategy, with its central focus on country-owned development of a national integrated plan for production of official statistics, resonates well with both developing country governments and their partner agencies alike. MAPS is also aligned with broader world trends that have gradually prompted more support for the importance of the needs of the developing countries in developing their statistical capacities and of the recognition of the challenges faced by statisticians in developing countries. For instance, the annual sessions of the UN Statistical Commission, the central official body of the world’s statisticians, also give more explicit attention to topics of interest to the developing countries. 3.7 At the World Bank, to help improve government statistical capacity in low-income countries, work is underway in the context of the IDA Results Measurement System aimed at requiring that Country Assistance Strategies (CASs) or Country Partnership Strategies 10 include a review of national statistical systems and an identification of what is needed to strengthen capacity both to generate and to use statistical information. A Statistical Capacity Building Guidance Note has been produced to help country teams on how to approach the assessment of national statistical capacity and how to address its shortages in a CAS or CPS program with a focus on: (a) the supply of and demand for official statistics in the country; (b) the adequacy of national data for monitoring indicators of country outcomes and IDA’s contribution; (c) the main constraints to developing the capacity of national statistical systems and the use of statistics; and (d) the extent to which the Government is committed to addressing these constraints and the support countries may require from the Bank and/or from other donors. But whereas guidelines and guidance are in place, progress has been much more modest in practice. 3.8 Taking the cue from the 2009 external evaluation of MAPS which recommended that the Bank should give fuller attention to SCB in CAS documents, we reviewed 21 CAS 8. Presentation by Mr. Ben Kiregyera, Director of the Africa Center for Statistics. 9. At the launching of the RRSF, the UNECA also created the Africa Center for Statistics (ACS) which aims to enhancing UNECA’s statistical coordination function and helping with the development of statistical capacities in Africa. 10. Country support strategies for non-IDA countries. 20 documents and progress reports which have been completed in the last three years in the Africa region (Table 8). The Africa Region was chosen for this exercise primarily because African countries tend to have lower statistical capacity scores 11 compared to other developing regions. Fifteen of these 21 countries 12 have benefited from various SCB support from the TFSCB, STATCAP, and other projects including as pilot countries under the SRF. A few of the countries have benefited from more than one project. Table 8. Results of a Review of Statistical Capacity Building in Recent CASs (Africa Region) Detailed Mention of Mention of joint assessment of statistics as No mention activities/studies Total the need for input to PRSP of statistics related to SCB SCB monitoring Number of 3 7 4 7 21 country CAS Countries with SCB support by the Bank (TFSCB, 3 7 0 5 15 STATCAP, SRF or others) 3.9 Only three of the 21 CASs systematically followed the guidelines spelled out in the previous paragraph 3.7 above. Of the remaining countries, seven of the CASs mentioned Bank activities or studies related to statistics but failed to provide an assessment of the need to strengthen capacity for statistics. In seven other CASs, there was no specific mention of statistics although five of these seven countries have received support from the TFSCB. In the remaining CASs, the section on SCB remains somewhat descriptive and the information provided appears to be more driven by the need for indicators to monitor the implementation of the CAS or the PRSP. To sum up, evidence from recent CAS documents for 21 African countries indicates that actual support by country teams for SCB remains spotty and inadequate in spite of efforts made to draw more attention to statistical development. 13 11. The Bank compiles an annual composite Statistical Capacity Indicator to provide an overview of the statistical capacity of each developing country based on three aspects: statistical methodology, source data, and data periodicity. See also the Bank’s Web-based database tool, the Bulletin Board on Statistical Capacity, which helps IDA countries identify weaknesses and plan remedial action, This is available at http://www.worldbank.org/data/bbsc. 12. Globally, about 70 percent of countries in Africa have received support for SCB from the World Bank through the various instruments. The percentage is roughly the same for the 21 countries under review with no specific patterns why the Bank has not provided support to some countries. 13. The above results are corroborated by the findings cited in a recent IEG review of the current regional strategy or Africa Action Plan. The new Africa Regional Strategy issued in March 2011—Africa’s Future and the World Bank’s Support to It— recognizes this inadequate attention to SCB in the past and makes a stronger case for building statistical capacity by focusing on initiatives which require the need for and use of statistics. They include enhancing social accountability -- by increasing citizens’ access to information --, impact evaluations and other evidence on performance to provide information with which citizens can hold governments accountable as well as a greater engagement with the media during the implementation of the strategy. 21 3.10 Vertical relevance. While the three programs were established at different dates, the review highlights the vertical relevance of the three programs with MAPS activities taking a more global perspective while PARIS21 and the TFSCB focus more on country and regional activities therefore ensuring complementarity in activities and outcomes. It is also worth noting that the PARIS21 business strategy has consistently focused on emerging issues and adjusted the work program and activities accordingly. 14 3.11 Horizontal relevance. Official statistics needed for decision-making in economic and social management for most countries and more particularly for developing countries are without exception public goods which are financed from government budgets with financial and technical support by external partners. The option of alternative sources for economic and social statistics is non-existent. 3.12 A key rationale underlying MAPS was to raise the profile of SCB with both donors and recipient countries and to help attract resources to fill unmet needs. The comparative advantage of MAPS lies in tapping the expertise of different international agencies involved in the statistical development to create a more holistic framework for dealing with SCB. The outputs and outcomes on SCB arising from the three programs also benefit from the work done by various UN agencies such as the United Nations system on MDG indicators, the UN Statistical Commission (UNSC), UN Statistical Division (UNSD),15 as well as regional institutions such as the AfDB and UNECA that play an instrumental role in the development of SCB in Africa. 3.13 Relevance of the design of the programs. Although the three programs were designed separately, the adjustment and refocusing of their objectives and activities have resulted in strategies that reflect relatively well the evolving priorities of both developing countries and donors (Table 9). The major activities of the three programs together provide: (a) a catalytic framework through MAPS and the DGF funding combined with knowledge networking with specialized agencies involved in survey methodology, and education and urban indicators; (b) technical assistance and advocacy at the level of countries and sub-regions through the various activities of PARIS21; and (c) extensive grant funding that has supported over 80 countries and regional initiatives in Africa, East Asia, Europe, and Latin America. 14. For instance, the PARIS21 Board has recently adopted a four-year program (2010–14) drawing on the vision of the 2009 Dakar Declaration on the Development of Statistics (DDDS) and on the recommendations of the 2009 PARIS21 evaluation. To implement this revised agenda, the logical framework, which was last revised in 2008, is being substantially modified to reflect the four main axes for PARIS21 work: co-ordination, advocacy, NSDS, and knowledge for statistics. 15. UNSC, which is part of the UN Economic and Social Council (ECOSOC), is the highest decision-making body for international statistical activities and assumes an important role in promoting the development of national statistics. UNSD provides secretarial support to the Commission and is in charge of (a) collection, processing and dissemination of statistical information; (b) standardization of statistical methods, classifications and definitions; (c) technical cooperation; and (d) coordination of international statistical programs and activities. 22 Table 9. A Summary of the Programs’ Strategies MAPS PARIS21 TFSCB • MAPS has assembled an • NSDS methodology: Helping Following the launch of informal partnership of countries design, seek funding for, MAPS, the TFSCB has international agencies with and implement NSDS. been used increasingly specialized experience that, • Advocacy: Helping statisticians in for laying the with the aid of grant finance, developing countries with their groundwork for SCB in were able to develop programs own advocacy work. developing countries specifically focused on helping through two financing • Donor collaboration: producing the developing countries achieve windows: (a) an NSDS annual Partner Report on Support stronger capacities in statistics. window and (b) a more to Statistics (PRESS) and piloting • It is expected that DGF funding general SCB window for with country expenditures on will stop in 2013 at which time various development statistical development. the World Bank’s engagement projects. • Support for surveys: Helping in MAPS will be through the countries improve their survey Statistics for Results Facility programs and make best use of (SRF) established in 2009 and available data. other funding facilities such as the TFSCB and STATCAP. 3.14 The various aspects of relevance as discussed above were fundamentally confirmed by the three external evaluations with little questioning or suggestions for changes. According to the external evaluations. • MAPS focused on topics that remain of highest priority in the development of statistics in the developing countries. Stressing the supply-side relevance, the evaluation praised the fact that thanks to DGF funding, MAPS was successful in drawing together different disciplines and expertise — IHSN/ADP for surveys and UN- Habitat for urban indicators — to create a strong framework for dealing with the multifaceted problems of statistical development in developing countries. The evaluation also suggested that the relevance of MAPS to country needs only required minor adjustments and refinement so that no significant change of direction was needed. • According to the PARIS21 external evaluation, the Secretariat activities and outputs are highly relevant to achieving the Partnership’s goal, purpose, and outcomes and outputs. Based on the results of interviews with both recipient countries and donor partners, the evaluation also confirmed that the activities and outputs meet user requests, are consistent with recipient country priorities, and are responsive to changed circumstances • The Evaluation Team is of the opinion that the TFSCB funding of NSDSs and other national capacity building projects has been highly relevant for the developing countries. Furthermore, from the perspective of country priorities — reflecting demand-side relevance — country-based consultations have confirmed that countries have been free to set their own priorities. 3.15 Summary. The IEG review confirms the strong relevance of the three programs designed to support SCB in developing countries. In terms of objectives, strategies, and 23 activities, the three programs together yield a coherent package. The initial designs of the programs, keeping in mind the fact that they were elaborated separately, do show weaknesses that affect many global programs such as, first, a strong focus on processes and activities, and second, insufficient attention to developing a rigorous conceptual framework linking activities and outputs to intended outcomes. The revision of the logical framework used by the PARIS21 partnership will certainly help to strengthen the linkages between activities and outcomes through the use of specific quantitative indicators, and a target date (2014) using 2012 as a milestone. 3.16 In particular, given the importance of NSDS processes in the work program of PARIS21, it is worth noting that the new logical framework will attempt to monitor the percentage of countries having a NSDS that scores more than 75 percent of the total score defined by the NSDS Quality Assessment Framework. This should go some way in helping identify obstacles to the implementation of NSDS which, as will be discussed in the next section on efficacy, remains a major challenge to the programs being reviewed in this report. 3.17 Along the same lines, the IEG review agrees with the recommendation of the TFSCB external evaluation that a more concerted effort should be expended to provide a more systematic assessment of the outcomes of TFSCB activities. Current World Bank guidelines only require the filing of a report detailing project implementation from an administrative perspective. Efficacy of the Three Programs 3.18 IEG’s review of the three programs’ progress in achieving their objectives draws on the evaluations of the three programs undertaken over 2003–2010 and interviews conducted in Paris and Washington. Overall, the evidence on efficacy from these sources is modest as the evaluations focus more on activities and outputs than outcomes. Based on the available evidence, this IEG review finds that the programs have a strong record of outputs in each of the six areas identified as priority actions under MAPS which provides the overall framework for SCB in developing countries. The review shows evidence of moderate achievements in advancing the following objectives: (a) preparation of national strategies for the NSDSs; (b) participation of developing countries in the 2010 Census Round; (c) investing in statistical development; and (d) improvement in MDG indicators. Implementation of NSDS remains an important challenge. More importantly, compared to other developing regions, the capacity of African countries as a group to compile and use economic and social statistics has shown more limited progress and will require renewed and more concerted efforts. APPROACH TO THE ASSESSMENT OF EFFICACY 3.19 The three programs under review in this IEG report together constitute the core components of an action plan to support the development of statistics through the mobilization of resources for investment and capacity building. In particular, MAPS has proved to be an important catalyst for improving the availability of statistics and for building the institutional capacity to improve statistics in the future. MAPS has also proved to be a good framework for increasing coordination among agencies, both for capacity building at 24 the country level and for actions by the international community to improve the availability of data, especially for the MDG indicators. 3.20 Although MAPS, PARIS21, and the TFSCB are three separate programs with distinct funding, and governance and management frameworks, they have strong linkages and complementarity in terms of goals, objectives, and activities. Most noteworthy, all three programs share the common objective of encouraging and supporting developing countries to prepare and implement NSDSs. It is therefore unrealistic to attempt to attribute progress on NSDSs to the individual programs although it is generally recognized among statisticians16 that in terms of activities, PARIS21 may have provided the larger contribution through technical support, advocacy, and coordination of donor support. It is, however, important to note that many other actors and sources of financing (e.g., the African Development Bank’s RRSF program, the UN agencies’ work etc.) have also made contributions to this overall effort. But progress achieved so far on the NSDS process would not have been possible without grant funding to individual countries by the TFSCB and the catalytic role of MAPS including sustained MAPS/DGF funding of MAPS activities and grant funding to individual countries by the TFSCB. A complicating factor of the assessment is that the outcomes supported by the three programs also benefit from the work done by other statistical agencies such as the United Nations system on MDG indicators, UNSC, UNSD, and regional institutions. 3.21 As a result of the conceptual challenges discussed in the previous paragraph, the discussion on efficacy — the extent to which the three programs have achieved their objectives — will take a different approach than what has been followed so far in this report. Instead of presenting the three programs separately and highlighting their linkages where appropriate as was the case until now, the discussion on efficacy will use the MAPS’ core program objectives as the broad framework under which PARIS21 and the TFSCB operate. Using the six specific actions of MAPS shown in Box 2 as a basis, this section will first review the progress in activities and outputs for each of the 6 objectives showing contributions by the three programs but also those of agencies which receive GDF funding. The assessment of efficacy of all relevant contributions to the six objectives will be presented in the following section. Box 2. The Marrakech Action Plan for Statistics 1. Promote strategic planning for developing national statistical systems. 2. Ensure full participation of developing countries in the 2010 Census Round. 3. Increase investment in SCB. 4. Set up an international network to better coordinate support for household surveys. 5. Undertake urgent improvements to monitor the MDGs and other development goals. 6. Improve coordination and accountability of the international statistical system. 16. For instance, PARIS21 has been asked to present the status on SCB at the UNSC annual meetings. 25 PROGRESS OF ACTIVITIES AND OUTPUTS OF THE THREE PROGRAMS 3.22 The first action, which aimed to help all developing countries to prepare and implement NSDSs, has been the main focus of the three programs under review and therefore will receive more attention in this report than the other activities. For some of the MAPS objectives such as for instance “to undertake urgent improvements to monitor the MDGs and other development goals,� references will be made to contributions by other institutions and agencies which have received funding through MAPS-DGF such as the UNESCO Institute of Statistics (UIS) for data on education, UN-HABITAT for indicators of urban development, and UNSD for assistance with the 2010 Census Round. 3.23 Promote strategic planning for developing national statistical system through National Strategies for the Development of Statistics. PARIS21 and the TFSCB which were established five years earlier than MAPS had originally broader goals aimed at developing a culture of evidence-based policymaking for PARIS21 and of providing grant funding to support developing countries to compile and use statistics for results- based management. By setting two key target dates on NSDSs — that countries should have prepared strategies by 2010, and have capacity in place to monitor progress towards the MDGs by 2015 — and recognizing that any effort to improve statistics cannot be managed from the outside but be based on a nationally owned and managed process, MAPS has helped to refocus SCB on helping developing countries prepare and implement their statistical development strategies. This has led PARIS21 to take the lead in developing guidelines and good practices for the preparation of NSDSs and TFSCB to give more funding support to the preparation of NSDSs. A summary of NSDS activities of the three programs is shown in Table 10. 3.24 Consultations carried out by the external evaluations of the three programs done in 2008–09 confirmed progress in the activities aimed at supporting the design of NSDSs. But they also highlighted a number of weaknesses. According to the PARIS21 evaluation, there is limited formal knowledge on the use of NSDS guidance as a checklist to ensure the quality of national strategies for statistical development. The MAPS evaluation raised the dilemma faced by PARIS21 Secretariat staffs who have desisted from making assessments of the quality of NSDS on the principle that since MAPS was built around country ownership, it is the responsibility of countries to report on NSDS development including implementation and its quality. The TFSCB evaluation’s field visits confirmed that countries have generally assumed ownership of the strategies and plans. Also the cooperation of national statistical offices with other government institutions and local stakeholders varies considerably across countries. 3.25 To address the somewhat conflicting challenges faced by staffs in respecting country ownership of the NSDS process while wanting to help countries identify weaknesses and take remedial actions, PARIS21 is promoting south-south learning in statistical development through a peer review mechanism to ensure that good practice passes from country to country, based on the first hand experience of peers, to help accelerate the change processes in reforming statistical systems. The focus of the peer reviews is on governance of the national statistical system (NSS), its organization, strategic planning, service to users, 26 Table 10. NSDS Activities by Program MAPS PARIS21 TFSCB • Indirect assistance to • Country/Regional programs • Since 1999, TFSCB countries through o Directly assisted countries in national has allocated about DGF funding of strategies for the development of $35 million to projects PARIS21’s programs statistics. aimed at improving of activities. o Organized regional/national strategy country statistical seminars. systems including • Advocacy and development of statistical NSDS projects in 61 advocacy tools countries (of which 36 are IDA countries). o Produced a statistical advocacy resource toolkit. o Helped countries in preparation of advocacy material. o Delivered advocacy messages at international events. • Promotion of donor collaboration o Conducted annually the Partner Report on Support to Statistics (PRESS) providing information on donors’ support to statistical development • Development of methodology for NSDS o Guide to NSDS design o Guide to dissemination strategy and dissemination guideline o Guide to financing national statistical activities funding, and sustainability. PARIS21 has facilitated peer reviews in eight African countries in collaboration with AFRISTAT 17 for francophone African countries. 3.26 Ensure full participation of developing countries in the 2010 Census Round. 18 Population censuses are the backbone of any national statistical system and provide essential data for monitoring progress toward the MDGs. The complexity of censuses and funding constraints seriously affected the 2000 round. The aim of this component of MAPS was to raise awareness of the importance of censuses, and to encourage effort to limit costs and improve the efficiency of data collection. Work under this component of MAPS is being coordinated by the UNSD through a World Program for the 2010 Census Round. Activities include a training and guidance program including country exchange of experiences covering survey methods and advice on the costs and benefits of various technologies to keep down the cost of census-taking. 3.27 Increase investment in statistical capacity building. The third MAPS action, recognizing that the cost of data collection is substantial — particularly in small, low income developing countries where communications are difficult and administrative 17. AFRISTAT was created in 1993 as a regional organization to help develop economic, social, and environmental statistics and SCB in francophone Africa. 18. Covering all censuses taken during the 2005-2014 decade. 27 systems are not well developed — calls for donors to increase their funding for statistics and for countries to provide increased domestic budget allocations. PARIS21 conducts an annual exercise known as the Partner Report on Support to Statistics (PRESS) with objectives: (a) to learn what donors (technical and financial partners) are doing and plan to do in the statistical field and identify countries or areas of statistics in need of more support; (b) to make this information available to both donors and recipient countries; and (c) to help raise the profile of statistics within the overall context of aid to developing countries. The data is drawn partly from the OECD’s Creditor Reporting System (CRS) for OECD/DAC members complemented by a questionnaire for all other partners. 3.28 Set up an international network to better coordinate support for household surveys. The fourth MAPS action translated into two complementary subcomponents, the International Household Survey Network (IHSN) and the Accelerated Data Program (ADP). The IHSN is a partnership of international organizations seeking to improve the availability, quality and use of survey data in developing countries. Both of these two programs are hosted by the PARIS21 Secretariat and implemented in collaboration with the World Bank and other partners. Activities undertaken under the IHSN partnership involve the development of: (a) the Microdata Management Toolkit to document surveys and censuses in compliance with international standards; (b) the National Data Archive (NADA) to help disseminate survey and census data in a searchable on-line catalog; (c) information on on- going and planned surveys and censuses in developing countries; (d) the Question Bank, a central repository of survey guidelines; and (e) tools and guidelines for measuring and reducing disclosure risk in survey microdata. 3.29 As a complement to the tool-development approach of IHSN, the activities of the Accelerated Data Program (ADP) conducted at the country level were designed to: (a) assist countries that do not have a coherent long-term survey program in strategizing their data collection activities to make them more widely and easily accessible to secondary users and to foster better analysis and use (also known as Task 1 in PARIS21 terminology); (b) build national capacity in micro-data preservation, analysis, protection of anonymity of the sources, and dissemination (Task 2); and (c) work with national data producers and secondary users on the production of updated estimates of key indicators, by further exploiting existing datasets and collecting new data development of an improved survey program, and data collection (Task 3). Originally designed as a pilot initiative for 12 countries, the ADP has met with a larger-than-anticipated success and is currently supporting agencies in more than 50 countries in Africa, Asia and the Pacific, Latin America and the Caribbean. 3.30 Undertake urgent improvements to monitor the MDGs and other development goals. Besides a number of other initiatives 19 designed to address immediate and pressing data gaps, specific works covered by MAPS with funding from DGF aimed at developing guidelines and improving the coverage and quality of data held in international databases on 19. They include the work of the UN Inter-Agency and Expert Group on the MDGs, which now includes discussion of statistical capacity issues. UNSD is managing a statistical capacity improvement program in southern Africa. UNICEF and UNDP have produced software (MDGInfo and DevInfo) to help countries make indicator data more accessible. UNDP has developed a statistical literacy program for MDG indicators; recently published materials are available online. 28 education, and urban development. In education, the World Bank is working with the UNESCO Institute of Statistics (UIS) to improve data on education, focusing on improving particularly the timeliness, integrity, and coverage of internationally comparable indicators to ensure a complete and informative picture of the state of education. This is complemented by a SCB program. In urban indicators, it is working with UN-HABITAT to strengthen global, national, and local capacities to collect and analyze national urban indicators and city-level indicators. 3.31 Improve coordination and accountability of the international statistical system. In recent years improvements in the availability and quality of data for monitoring the MDGs have demonstrated the value of cooperation between international agencies and the importance of a coordinated approach to supporting the work of national statistical authorities. Coordination of the statistical activities of the UN and its specialized agencies takes place in a number of forums, with the UNSC as the highest governing body. The Committee for the Coordination of Statistical Activities and the Inter-Agency and Expert Group on the MDGs have proven to be useful forums for exchanging information. But while much progress has been made, further improvements require a better system of accountability to clarify responsibilities for setting standards, disseminating information, providing technical assistance, and mobilizing resources to support national efforts. OVERALL ACHIEVEMENT OF CORE OBJECTIVES 3.32 In comparison to the programs’ relative success in implementing a rich and diversified menu of activities, the record on advancing the key program objectives is more varied. These are briefly reviewed and assessed using the MAPS’ six priority actions as a framework. 3.33 Promoting NSDSs. Progress has been notable in relation to the design and implementation of NSDSs, a process which has taken hold in all regions. As of November 2010, all but 16 of the 79 IDA-eligible countries are either designing or implementing a NSDS (Table 11). There are 28 countries currently designing or awaiting adoption of their NSDS (representing 36 percent of the total); of these 16 are in Africa, the region where PARIS21 has been most active. Of the 79 IDA borrowers, 35 are currently implementing their NSDS (44 percent), 20 of whom are African countries (50 percent). 3.34 Consultations undertaken for this IEG review show that the implementation of NSDS remains an important challenge. Several factors have been mentioned as possible obstacles including inadequate funding 20 and capacity building support, the lack of integration of the NSDSs to national development plans, weak links between the producers and users of data, fragmented/uncoordinated support and approaches, and the 20. The new World Bank-led, multi-donor effort known as the Statistics for Results Facility (SRF) mentioned in chapter 1 of this report is a significant attempt to bridge the design and implementation of national statistical strategies. 29 Table 11. NSDS Status for IDA Countries Without or with NSDS under expired NSDS Without an NSDS under preparation or All IDA and currently NSDS nor implementation awaiting countries planning an planning one adoption NSDS No. % No. % No. % No. % No. Africa 20 50.0 16 40.0 3 7.5 1 2.5 40 Asia & Pacific 11 40.7 8 29.6 6 22.2 2 7.4 27 Latin America & Caribbean 2 22.2 3 33.3 0 0.0 4 44.4 9 Europe 2 66.7 1 33.3 0 0.0 0 0.0 3 All Countries 35 44.3 28 35.4 9 11.4 7 6.9 79 Source: PARIS21. NSDS Status in IDA and Middle-Income Countries - Progress Report as of November 2010. lack of specific guidance to countries on available options. At the PARIS21 Consortium 21 meeting, held in Dakar, Senegal in 2009, the participants recognized that much has been achieved since 2000, but reaffirmed that much still remains to be done to deliver the vision of MAPS. “The need for countries to set their own development priorities for their statistical systems through the preparation of national strategies has been widely accepted…The Consortium called on all partners to take action so that by 2014, the focus of attention should have moved from preparing strategic plans to implementing them with sustainable funding and technical capacity.� 3.35 To help countries meet the implementation challenge, the work program of PARIS21 in the 2010–14 period stresses: (a) translating the priorities into realistic, budgeted action plans; (b) bringing donors into the process, so that costs and financing plans can be prepared on a realistic basis and so that donor interest in and commitment to the NSDS can be followed through; and (c) reporting on progress in NSDS implementation against appropriate output and outcome indicators. 3.36 Promoting country participation in 2010 Survey Rounds. Compared to the 2000 Census Round, participation by developing countries has improved significantly for the 2010 round. Only nine countries or sub-regions have not yet scheduled a census, seven of which did not participate in the last census round. Three of the nine countries without a scheduled census were preparing for a census but delayed the enumeration sine die due to political reasons or unrest. According to the current situation, around 140 million people will not be included in a census in the 2010 round, compared to 560 million in the 2000 round — a 75 percent reduction. Most significant in terms of outputs of this component is the large increase in the number of low-income countries, including 14 African countries that either have held or are planning to hold a census during 2005–14. 21. The PARIS21 Consortium statement can be found on the following website http://www.oecd.org/dataoecd/57/0/44088255.pdf. 30 3.37 Investing in statistics. According to the 2010 PRESS Report, financial commitments to statistical development for the period 2008–10 amounted to roughly $1.6 billion. Africa received nearly half of total statistical support, equivalent to $716 million in commitments. A further $151 million were committed to global, non-country-specific projects and programs. Approximately 87 percent of country-specific commitments went to low-income. Fifteen countries, most of them in Africa accounted for almost half of total estimated commitments. Fragile states have received significant donor attention support with the sum of commitments to all fragile states representing one-third of total commitments according to PRESS 2010. International organizations — which include by order of importance the European Commission and Eurostat, the World Bank, the AfDB, UNICEF, and the IMF — provided 55 percent of all commitments, bilateral donors 37 percent and regional organizations the remaining 8 percent. The European Commission, the World Bank, and the United Kingdom are the three largest donors providing 62 percent of total commitments. Drawing on the sample of the top 15 PRESS recipients, the figures show that more than one-third of donor– recipient relationships were “non-significant� indicating continuing fragmentation of support. 3.38 The third round of the PRESS exercise points toward some emerging trends. Since 2008, global commitments to support statistics have risen by nearly 60 percent. Although Africa continues to account for the bulk of support to statistics, the share of Africa has declined rather significantly — 59 percent of total commitments in 2008, 52 percent in 2009 and 45 percent in 2010 — while the share of Asia doubled to reach 31 percent in 2010. With a modification in the questionnaire, PRESS 2010 was able to show that 71 percent of commitments confirmed their alignment with NSDSs priorities. Across regions, Asia and LAC enjoyed the highest rate of alignment of support to the NSDS process. 3.39 An issue that warrants more attention is the weak financial effort by developing countries using their own domestic resources to sustain the development of their statistical services.22 But whereas low-income countries were reluctant to borrow funds for statistical development, which in part explains the low startup of the World Bank’s STATCAP lending program, 23 the recent decision by Senegal, Rwanda, and Mozambique to borrow for statistical development points toward the rising importance of statistics in development planning and management. 3.40 Supporting surveys. The Accelerated Data Program, which was until recently the purview of national statistical agencies, has increasingly involved line ministries therefore ensuring broad-based in-country support. Fifty-six countries have received or are receiving support, with 12 additional countries have requested assistance. More than 20 survey catalogues are now available on-line thanks to ADP support. There is a growing demand for support by countries but the lack of resources has prevented a scaling-up of this activity at the moment. 22. According to data available on the programming of expenditures linked to the NSDSs of 26 countries, expenditures (excluding those related to censuses) are estimated to average around 0.26 percent of public expenditures. 23. STATCAP is a new lending program launched in 2004 designed to make it easier for clients to access regular World Bank financing for improving statistical capacity with flexible financing including meeting recurrent costs. Only 13 countries, including 4 in Africa, have availed themselves of this source of funding. 31 3.41 Strengthening MDG Indicators. The adoption of the Millennium Declaration and of the MDGs has led to a focused effort across all UN and other international agencies to measure progress toward the MDGs and to improve the data with which to do so. This work is led by the UN Statistics Division. Progress in the availability of data for 22 of the MDG official indicators has been noteworthy. Compared to 2003 when only 2 percent of the 163 countries had two data points for 16 or more of the 22 MDG indicators, 72 percent of the countries had at least two such data points in 2009, which allows for measuring trend over time. Although this is not solely due to actual progress taking place in countries, it does reflect, at least in part, an increased availability of data in national sources and a stronger capacity of national statistical systems in addressing the monitoring requirements. Other factors play a role, including improvement in the reporting mechanisms from countries to international agencies and increased access and understanding by agencies of existing national sources. A key challenge has been the need to reconcile statistics from national and international sources led by the Inter-Agency and Expert Group on MDG Indicators (IAEG). 3.42 Improving coordination in SCB. At the international level, there is a wide support for PRESS as a useful tool for promoting effective collaboration among development partners. In some countries PRESS had been well received and was serving as a basis for donor dialogue. From the regional perspective, PRESS has also been seen as a good starting point for discussions on issues relating to regional coordination. The Committee for the Coordination of Statistical Activities of the UNSC 24 which is the apex agency on matters related to statistics, has discussed the use of PRESS not only to inform on past support but also as a tool for planning and coordination purposes, and making online access to PRESS available to help with project identification of projects. 3.43 But there remains the challenge posed by the low rate of response to the PRESS questionnaire including by some important donors in the area of SCB and several United Nations agencies. Based on the above, PARIS21 has made plans to refine the PRESS exercise for the future including modifications to the online questionnaire for the 2011 Round to allow donors more options to verify the information and to begin addressing the reliability of data in a practical way through a pilot country to determine gaps or differences between the level of support between that reported by donors and what countries actually receive. This country-based exercise will also make it possible to report on the domestic budgetary resources allocated to statistics. The results of this exercise — to be called Country Report on Support to Statistics or CRESS 25 — could provide a more comprehensive picture of funding for SCB including national budgetary resources which, as discussed above, appear to be rising in importance in a small number of low-income countries. 3.44 At the country level, the PARIS21 Secretariat carries out specific activities in support of donor coordination for SCB. Interventions by the PARIS21 Secretariat were recognized in some cases to having provided a much needed impetus to coordination of donor partners. 24. United Nations Economic and Social Council, Statistical Commission, Work of the Committee for the Coordination of Statistical Activities, Report of the Secretary-General, New York, February 2010. 25. PARIS21, “Partner Report on Support to Statistics – PRESS, 2010 Round�, Paris, November 2010. 32 However, based on country reviews done in the context of the external evaluation, progress has been limited and donor coordination remains generally not well developed. 3.45 Progress on objectives: an overall assessment. The previous paragraphs attempted to assess the roles of the three SCB programs under review in promoting progress in statistics using a mixture of performance indicators that include activities and outcomes. Assessing whether support for SCB in statistics has achieved the primary objective of helping developing countries improve the capacity to compile and use statistics for evidence-based policymaking is a more challenging task given the multifaceted and long-term nature of any capacity building processes. 3.46 Furthermore, the evidence of progress must originate from the countries themselves to be meaningful and representative. The external evaluations of the three programs based on country studies and field visits show a slightly positive, albeit mixed, picture. These are briefly summarized below: • The MAPS evaluation of 2008 concluded, based on in country consultations in Ethiopia, Malawi, Mali, and Niger in 2008 and using the MDG indicators as an assessment criterion, that while there is improvement in basic education data, and the effect of the introduction all reintroduction of surveys, there are no consistent pictures of capacity improvement. 26 • The 2009 evaluation of PARIS21 which involves field visits to nine countries (Cote d’Ivoire, Djibouti, El Salvador, Ethiopia, Lao PDR, Malawi, Mali, Mozambique, and Tunisia) has a slightly more positive assessment. Overall, the capacity to compile, analyze, and use statistics has improved. Using the PRSPs as the basis of the assessment, the evaluation reported that the use of statistics and statistical analysis has a moderate impact on PRSP design but appears to have played an increasingly more important role in PRSP monitoring and evaluation. 27 • The 2010 evaluation of the TFSCB, based on evidence from 23 countries concluded that the NSDS projects have, in several countries, highlighted to national governments “the importance of having regular and trustworthy statistics for decision-making in economic and social policies. The NSDS may be a catalyst for increased respect for and use of statistics for public policies, monitoring, and evaluation.� 28 3.47 The Statistical Capacity Indicator which the World Bank has begun to compile since 1999 provides an objective approach to assess the statistical capacity of some 145 developing countries based on a set of criteria consistent with international recommendations. Based on scale of 0 to 100, this composite indicator covers three aspects of statistical capacity: (a) statistical methodology which measures the country’s capacity 26. MAPS Report of an Independent Evaluation (2008), page 48. 27. Evaluation of PARIS21 (2009), pp. 47-48. 28. Evaluation Report of the World Bank Trust Fund for Statistical Capacity Building (2010), pp. 21-22. 33 to meet internationally recommended standards, methods and data reporting practices in economic and social statistics; (b) source data which assesses the ability to collect relevant data at recommended intervals (censuses and surveys); and (c) data periodicity which measures the capacity for making data accessible to international databases. 3.48 Table 12 shows the aggregate scores of the Statistical Capacity Indicator 29 for 111 middle and low-income IBRD/IDA countries with a population of a million or more. On average, the scores improved from all three aspects of the indicator showing that over the 1999–2009 period, all groups of countries have made progress in the way statistics are collected, compiled, disseminated, and reported. Data periodicity has experienced the sharpest improvement, due in part to more regular health surveys. But progress made in statistical capacity has been very uneven across regions. While IDA-eligible countries from outside of Africa have made remarkable improvements in all three criteria, improvements in African IDA countries as a group were significantly lower, especially regarding methodology and source data. Table 12. World Bank Statistical Capacity Indicator (on a scale of 0 to 100) of which: of which: All countries IDA Sub- IDA Non-Sub- Saharan Africa Saharan Africa 1999 2009 1999 2009 1999 2009 Overall 52 65 47 53 48 68 Methodology 45 56 35 37 40 58 Source Data 53 63 46 47 50 67 Periodicity 59 77 61 76 54 78 Source: Development Data Group, World Bank. Details on methodology can be found at http://go.worldbank.org/VD2BR27RN0 3.49 Summary. Based on the review of recent evaluations complemented by additional analyses, this IEG review finds that moderate progress on activities and outcomes from the three programs. The redefining of the programs’ outcomes with a primary focus on strengthening national strategies for the development of statistics has yielded important results. The higher goal that aims to improve the capacity of developing countries to compile and use statistics in support of the management for development results has, however, been hampered by: (a) insufficient attention to implementation challenges facing the NSDS process; (b) the absence of a strategy to help stimulate demand for better data at the country level, not only among government users but also from other domestic stakeholders such as the civil society, NGOs, research institutes, and the media; and (c) improving but still inadequate attention to the support for statistical capacity among the donor community. 3.50 Some of the measures that would seem essential to accelerate progress in statistical systems in developing countries including: (a) making the NSDSs more relevant, realistic 29. A “score� for each aspect is derived, and an overall score is calculated by combining the scores for the three aspects, giving equal weight to each. The score is scaled to provide a value between 0 and 100; a score of 100 indicates that the country all the benchmarks in all three aspects of statistical capacity. 34 and sustainable; (b) reinforcing the NSDSs as a continuous process with regular feedback on implementation — similar to the experience of PRSPs; (c) engaging in more active consultations between the producers and users of statistics; and (d) continuing the effort covered under the three programs to increase the volume and improve the efficiency and delivery of financial and technical resources to strengthen statistical systems. 3.51 An important issue concerns ways to increase the demand for statistics by policymakers, their advisers, and other analysts. First, MAPS actions need to encompass the use of statistics in addition to the generation of data. But statistical data producers also need to better understand policy environments and the need for data. For instance best practice NSDSs link strongly with PRSP implementation and monitoring and evaluation. This has led to the suggestion that a share of ADP resources be allocated to assessing gaps in the supply of data compared to demand. 3.52 Looking ahead, the support for SCB will be affected by several trends. First, the contributions by bilateral donors will most likely be constrained by budget austerity. Second, there appears to be a shift in donor support toward regions that show results as reported in the 2010 PRESS report. Among international partners, the role of the Bank is generally perceived as important through advocacy and technical support but even more importantly as a major provider of funding for SCB including, in addition to the TFSCB and MAPS-DGF, STATCAP, and the new Statistical for Results Facility. But whereas the importance of statistical development has figured more prominently in recent discussions and reviews of IDA, the analysis shown earlier in this chapter, which confirms the assessment made by the MAPS evaluation team, draws attention to the necessity for country teams in the Africa Region of the Bank to be much more pro-active and knowledgeable about addressing in-country SCB needs. Efficiency of the Programs EXPENDITURES 3.53 MAPS. The World Bank’s DGF has provided financial support to various SCB programs since FY06 (Table 13). Programs covered by MAPS have received an average of $5.7 million per fiscal year, of which 70–75 percent has represented contributions to PARIS21. Table 13. MAPS’ Programs Supported by the World Bank DGF ($ millions) Recipient Supported program FY06 FY07 FY08 FY09 FY10 PARIS21 Support for NSDS processes 1.50 1.30 1.30 1.00 0.70 PARIS21 ADP and IHSN 2.90 2.75 2.75 3.20 3.20 Census 2010, urban indicators, Others 1.50 1.65 1.65 1.50 1.80 gender and education statistics Total for MAPS 5.90 5.70 5.70 5.70 5.70 Source: DGF Council reports. 3.54 PARIS21. Expenditures under the PARIS21 work program are summarized in Table 14 with a distinction between the Partnership’s core programs related to NSDS processes — 35 country and regional programs, advocacy, and donor coordination — and the ‘so-called’ satellite programs 30 connected to surveys and survey data management which are hosted by PARIS21 and principally financed with DGF resources. Contributions to the core programs come from the World Bank, the European Commission, Eurostat, and 15 bilateral donors. 31 The United Kingdom is the largest contributor followed by the World Bank. Table 14. PARIS21: Expenditures (thousands of Euros) 2010 Expenditure categories 2006 2007 2008 2009 (estimate) Core Programs 1,930 2,068 2,579 3,550 3,440 Satellite programs (ADP- IHSN) 499 2,291 2,355 2,139 3,044 Source: PARIS21 Progress Report (June 2010). 3.55 TFSCB. Since its establishment, TFSCB has received over $46 million in donor contributions from six partners — Canada, France, Germany, the Netherlands, Switzerland, and the United Kingdom. About $35 million has already been disbursed or committed to ongoing projects. In the past five years, annual disbursements average about $4 million. Total available funds, calculated by combining the unallocated funds from TFSCB I (1999–2005), II (2004–2010), and III (2007–2012) amount to $6 million as of July 2010. It is expected that at the current rate of demand and project approvals, this amount is expected to be sufficient to fund future projects under the NSDS and the SCB windows until mid-2011 (Table 15). FINANCIAL MANAGEMENT 3.56 MAPS. The external evaluation team did not find evidence that the DGF grant amounts were decided as a result of competitive tendering or detailed budgeting but also that there were no obvious cases of inefficiency or wasted resources. The evaluators concluded that based on what MAPS was able to achieve through its partner institutions with relatively small amounts of money, that the amounts allocated were reasonable and that it would have cost much more doing it through a profit-making body. Examples of significant cost saving made possible through MAPS and cited by the evaluation team for illustrative purposes included: (a) the support provided by UNSD to minimize the risk of census failure is potentially important given the large waste of money associated with badly done censuses; (b) the collaborative effort with specialized agencies such as UNESCO/UIS, UNSD, and UN- Habitat has allowed surveys to be developed more quickly and in line with international norms thereby generating important cost savings; (c) the strong links between some of the MAPS partners has led to significant cost sharing for specific activities. 30. According to its governance, the PARIS21 Secretariat may take on additional functions which help to achieve the aims of the Partnership. Since 2006, the ADP and IHSN, both funded by the DGF-MAPS have been hosted by the Secretariat. Each of the satellite programmes, which are not part of PARIS21’s main objective, has separate funding and staff who report to the PARIS21 Secretariat Manager. 31. Include Austria, Belgium, Canada, Finland, France, Greece, Ireland, Italy, Japan, the Netherlands, Norway, Spain, Sweden, Switzerland, and the United Kingdom. 36 Table 15. TFSCB Financing ($ millions) Financial Items $ million TFSCB I [1999–2005] A. Donor contributions including investment income 13.57 B. T otal allocation 11.01 Allocation to projects 9.81 TFSCB oversight, proposal review, reporting and monitoring (9%) 1.20 C. Unallocated funds (C = A - B) 2.55 TFSCB II [2004–2010]] D. Donor contributions including investment income 13.26 E. Total allocation 10.40 Allocation to projects 9.81 TFSCB oversight, proposal review, reporting and monitoring (5%) 0.59 F. Unallocated funds (F = D - E) 2.86 TFSCB III [2007–2012] G. Donor contributions including investment income 19.26 H. T otal allocation 18.50 Allocation to projects 15.52 TFSCB oversight, proposal review, reporting and monitoring (2007–13) 2.00 TFSCB project supervision (2007–12) 0.60 Administration fee (2%) 0.38 I. Unallocated funds (I = G - H) 0.76 J. Total available funds (J = C + F + I) 6.17 Note: Allocation to projects is the disbursed amount for closed projects and the allocated amount for active projects. In the previous progress reports, committed amounts were used for all projects. TFSCB oversight and project supervision costs are estimated for the lifetime of TFSCB II and III. Source: World Bank, Development Data Group, The Trust Fund for Statistical Capacity Building at 10 – Investing in Statistical Capacity, October 2010. 3.57 PARIS21. The external evaluation team found that the Secretariat was well managed and that the outputs were commensurate with staff levels and expenditures. Management costs have been held steady over the years at around 15–21 percent of total expenditures; except for 2009 when costs rose above the trend line due to the costs of the Consortium meeting. 32 Interviews in recipient countries undertaken by the external evaluation team indicated that (a) PARIS21 staff and consultants were effective during missions to the countries; (b) regional workshops were well organized and conducted; (c) peer reviews were well organized and efficiently facilitated; (d) the regional network of advisers played multiple roles thus saving money; and (e) building on easy-to-use technology reduced costs for training and other development activities. 32. The next Consortium is scheduled to meet in 2014. 37 3.58 TFSCB. The Trust Fund Administration Unit (TFAU) monitors the aggregate financial status of the fund and produces financial updates published annually 33 in the trust fund’s progress reports showing funds committed by donors and allocations towards approved projects. These are posted on the Web site. The DfID evaluation 34 concluded that financial management of the fund was sound and that the application process was efficient. The introduction of the Bank’s electronic Grant Reporting and Monitoring System (GRM), which includes an automated system of reminders on grant approval and disbursement, has helped to limit inefficiencies due to potential Bank’s internal bottlenecks. Consultations done for the review raised the legitimate concern that further scrutiny of the fund granting and monitoring processes may not be cost-effective given the relatively small size of TFSCB grants, with most grants amounting to less than $250,000. 33 Until 2009 progress reports were produced twice a year. 34. Agulhas Consulting, Evaluation of DFID Support to the World Bank Trust Fund for Statistical Capacity Building Strategic and Management Overview, February 2008. 38 4. Governance, Management and Financial Sustainability 4.1 Different but complementary, the governance and management arrangements of the three programs appear to be working moderately well in guiding and implementing a complex set of activities designed to build capacity and statistical development in developing countries. Overall, there do not appear to be important inconsistencies or anomalies in the governance or management structures although potential reputational risks to the World Bank should not be ignored due to the facts that the Bank is represented on all three boards and two of the three programs are located at the Bank. 35 To give more substance to the broadly accepted principle that developing countries should take the lead in developing their own statistical capacity would mean giving stronger voice to developing countries representatives in the governing structures of the programs under review. Changes decided recently by the PARIS21 Partnership to give wider stakeholder representation while maintaining equal presentation between developed and developing countries in the newly created Board could serve as a good model for the other programs to emulate. 4.2 The complementarity of the three programs which has been highlighted throughout this report has important implications in terms of their financing. The phasing out of DGF funding beginning in FY11 to the MAPS program will have important implications for the PARIS21 Partnership. Together with pressures from fiscal consolidation among current bilateral donors for SCB, resources availability for statistical development in low-income countries will most probably become scarcer at a time when reliable statistics will become increasingly more important, more particularly to monitor progress on the MDGs as the 2015 deadline looms closer. This trend will require renewed efforts for increasing resources and for broadening their base including through a more active role played by developing country governments themselves. Governance and Management 4.3 Having been established as complimentary programs forming parts of a coordinated effort to support statistical development in developing countries, the three programs under review present a number of overlapping features in their governance structures. The annual meetings of the Consultative Group, which is the highest governing body of the TFSCB, are organized back-to-back with the annual PARIS21 Board meetings so as to have access to a much broader audience including representatives from all developing regional groupings. Internally to the Bank, the same individuals and more particularly the Director of DECDG and one of its managers play a very active role in all three programs. 4.4 MAPS. Created as a non-bureaucratic mechanism for encouraging partnership and cooperation among a small number of international agencies involved in SCB, MAPS was designed with the purported intention to keep governance arrangements light. The program started out by relying solely on the DGF Council for guidance on the scale of the overall program. But soon afterward, the need to bring about a more effective framework for communications and partnership among the stakeholders led to the creation at the end of 35. 40 percent of global and regional partnership programs in which the Bank is involved are hosted by the Bank. 39 2005 of two new bodies: (a) the external MAPS Advisory Board to provide strategic direction, promote coordination, and monitor implementation of the program and (b) and the internal Statistical Capacity Building Committee (SCBC) which was proposed by the DGF Council to ensure better coordination among SCB initiatives from different parts of the World Bank, and to oversee implementation of the DGF grant. Day-to-day work is handled by a small MAPS Implementation Unit led by the Development Data Group (Table 16). 4.5 The Advisory Board (AB) met once a year. Based on the progress reports, the review endorses the conclusion of the 2009 evaluation that the Board has provided valuable advice on components of the MAPS programs but has not given time for in-depth discussion of strategic directions. Participation of non-statistician policymakers has also been weak. The internal SCBC has also played a useful role on helping to monitor progress on MAPS implementation. But it has met annually which far less frequently than planned. Also, it has confined its attention almost exclusively to the DGF and has not broached the wider within- Bank coordination role it was originally designed to play. This has begun to change since the establishment of the Statistical for Result Facility (SRF). Beginning in 2010, the SCBC has broadened its membership as a first step of an effort to look at Bank-wide coordination and has, for instance, initiated actions for the MAPS Unit to contact the MDG Indicators Inter- Agency and Expert Group with a view to assessing specific areas where support from the MAPS partnership may be needed. Table 16. Governance and Management Arrangements for MAPS Governing (Internal) Statistical Capacity DGF Council External Advisory Board Bodies Building Committee (SCBC) Governance The Council reviews 8 senior representatives from 20 senior policymakers and structures and endorses the Bank’s networks and regions statisticians from proposals for grant involved in the results agenda international agencies, allocations under the and SCB. The OPCS Director OECD and developing development grant in charge of country services countries. The OPCS Vice facility (DGF) which serves as Chairperson. President serves as was established in Chairperson. 1998. Functions Provides guidance Provides guidance on DGF Provides strategic direction, and overall program grant allocations among the promote coordination, and scale. constituent programs, and monitor implementation. oversees implementation and the grant renewal process. Organizational Meets twice a year. Although Convenes once a year at traditions the SCBC is assigned the role a location to which many to promote the coordination of members are expected to SCB activities within the Bank, be coming for other it has confined its attention to gatherings. Sessions the DGF grant and not generally lasted half a day. broached the wider Bank coordination role. 4.6 PARIS21. The partnership has a two-tiered governance structure which has experienced important changes at the last annual meeting in June 2010. Until then, the work 40 of the PARIS21 Partnership was guided by the Steering Committee, an international group of stakeholders with representatives from the five founding members — the European Commission, IMF, OECD, the United Nations (represented by the UN Development Group and the UN Statistics Division), and World Bank — developing countries representing different regions of the developing world, bilateral donors, and other multilateral institutions including regional development banks. A sub-group, the Bureau, provides guidance and support to the Secretariat and its work program between the yearly Steering Committee meetings. The Steering Committee, which meets twice a year, is responsible for the oversight of the PARIS21 Secretariat’s work program. More specifically, it sets strategic directions; prioritizes activities and the appropriate level of funding; oversees the budget; ensures co-ordination within the partnership; reviews progress reports; and reviews and provides inputs into the terms of reference and reports of the periodic evaluations of PARIS21 (Table 17 outlines the arrangements revised in June 2010). Table 17. Revised Governance and Management Arrangements for PARIS21 (as of June 2010) Governing/ Management PARIS21 Board Executive Committee Secretariat Bodies Governance/ The 5 founding members Consisting of 9 Members Hosted by the OECD, the Management hold permanent seats. appointed by the Board Secretariat carries out its structures Membership is open to and representing 2 functions in accordance all interested bilateral developing countries, 2 with the rules and donors who either bilateral donors, practices of the OECD, in finance or participate in representatives of the 5 particular those with PARIS21 activities. The founding institutions. The regard to staff and Board has two co-Chairs: Secretariat Manager is financial administration. the chair of the an ex-officio member OECD/DAC and a without voting rights. developing country representative. Functions Sets strategic direction Provides an Co-ordinates the and priorities of future accountability Partnership activities and action, oversees the mechanism and manages and implements design and guidance to the work of the day-to-day work implementation of the Secretariat. In program as approved by PARIS21 work, particular, it provides a the Board and the advocates for and policy direction to Executive Committee. publicizes PARIS21 activities and acts as a activities within Board decision-making body members’ when required. constituencies. Organizational Meets once a year. The Meets at least four times The Secretariat may take traditions annual meetings of the a year, where possible in on additional work if they Board usually include the margins of suitable help to achieve the broad seminars or other in- international meetings. aims of the Partnership. depth discussion of Other business is The so-called “satellite current important issues conducted through programs� have separate relating to the consultation by funding and staff. development or use of telephone and e-mail. statistics. 41 4.7 Following the Dakar Declaration on the Development of Statistics (DDDS) 36 adopted in November 2009 by the PARIS21 Consortium (all partnership members), the Steering Committee approved an enhanced governance structure at its June 2010 meeting. The Steering Committee was transformed into a Board with an expanded membership to ensure wider representation of developing countries. The Board, which meets once a year, sets the strategic direction for PARIS21, reviews and advises on the Secretariat’s medium-term work program, and reviews and provides inputs to the terms of reference and reports of the periodic evaluations of PARIS21. The Board nominates a nine-member Executive Committee whose role is to provide an accountability mechanism and guidance to the ongoing work of the Secretariat. The Executive Committee has taken over some of the responsibilities of the former Steering Committee including the monitoring of progress of the work program of the Secretariat; the review and approval of annual work programs, budgets, and annual progress and financial reports; and the review of the budget situation on a regular basis including fundraising efforts for the Secretariat. 4.8 PARIS21 is hosted by the OECD, 37 functions in accordance with the rules and practices of the OECD with regard to staff and financial administration and audit, and reports to the OECD Development Assistance Committee (OECD/DAC) whose chair is also one of MAPS’ 2 co-chairs. On behalf of the Board, the PARIS21 Secretariat also reports to the UN Statistical Commission — the highest body in charge of statistical development — in the form of the annual progress report on SCB at the UNSC’s annual meetings. PARIS21’s other co-chair is a developing country representative. In comparison to MAPS and TFSCB which are located in and led by DECDG, PARIS21’s work program is run practically independently of the rest of the OECD, its host organization. 4.9 TFSCB. The governance of TFSCB is closely linked to that of PARIS21. The TFSCB’s highest governing body is the Consultative Group (CG) which consists of contributing members — Canada, France, Germany, the Netherlands, Switzerland, and the United Kingdom — and non-contributing members (IMF, OECD/DAC, EU, UN, and the PARIS21 Secretariat). An Advisory Panel (AP) was created to conduct a yearly technical review of the Trust Fund activities and report its findings and recommendations to the CG during its Annual Meeting. The TFSCB is assisted by an Internal Management Committee (IMC). Day-to-day business is administered by the TFSCB Administration Unit (Table 18). 4.10 Assessment of governance and management. On the whole, the governance structures of the three programs broadly comply with the generally accepted principles of good governance. On legitimacy, all three programs involve, besides their shareholders, a broader set of stakeholders. This is particularly the case for the PARIS21 Partnership whose 36. The DDDS sets the year 2014 as a target when all countries that are committed to improving their statistical systems have been able to put their NSDSs into effect by strengthening co-ordination at all levels including establishing national partnerships for statistics, where such partnerships do not already exist, and improving consultation between statistical producers and key user groups. The PARIS21 Secretariat, in collaboration with other partners, will monitor the implementation of the DDDS and report on progress. 37. The Secretariat is part of the OECD’s Development Co-operation Directorate, and the PARIS21 Secretariat Manager is a member of the DCD Management Team. 42 Table 18. Governance and Management Arrangements for TFSCB Governing/ Internal Trust Fund Consultative Management Advisory Panel Management Administration Group Bodies Committee (IMC) Unit Governance/ Members are the 6 The Advisory Chaired by The Unit consists Management contributing Panel (AP) is the DECDG of the TFSCB structures bilateral donors technical review Manager, the 9- Program Manager and key 38 arm of the TFSCB member and Administrator, international Governing Committee is who are DECDG bodies (Eurostat, Structure. made up of staff. OECD, United DECDG staff and Nations Statistics representatives Division, IMF), from Regions and PARIS21 and the Networks. World Bank. Chaired by the DECDG Director and the developing country co-chair of PARIS21. Functions Reviews and Conducts yearly Meets formally Administers the discuss project technical review of twice annually, day to day performance and TFSCB activities but conducts business of the policy implications and reports its some of the work TFSCB. The Unit for the Fund; it findings and via email. is also involved in takes decisions recommendations Provides assisting TTLs in based on to the Consultative oversight of the project preparation consensus Group (CG) during Fund and reviews and its Annual and approves implementation. Meeting. project proposals. Organizational The CG meets The AP plays a de Decisions on The TFAU traditions once a year in a facto quality control projects proposed prepares a TFSCB joint meeting with role and, in recent under the so- annual progress the PARIS21 years, schedules called “NSDS report for Board. This the review window� are submission to the enables broader meetings close to reviewed through CG. participation by the CG annual virtual meetings PARIS21 donors meetings to with therefore and, more provide insights to rapid turnaround importantly, by the latest time. representatives of developments. developing countries. Board consists of 40 members representing in equal proportions partners representing developing countries 39 and developed countries. The new PARIS21 Board that came into 38. The number and the composition of the IMC members change, as necessary. 39. While efforts have been made to promote a greater involvement of developing regions in the PARIS21 Board, it is less clear whether and how discussions and decisions taken by the Board are shared with individual countries by the Board attendees representing various sub-regions. This raises concerns about weak ‘second-round’ legitimacy which is partly alleviated by the posting of detailed minutes of Board meetings on the PARIS21 Web site. 43 existence in June 2010 has therefore fully addressed one of the issues raised by the 2009 external evaluation namely that developing countries perceived to be underrepresented in the (former) PARIS21 Steering Committee. The PARIS21 Secretariat produces an annual report on SCB for the UN Statistical Commission (UNSC) which as mentioned earlier in the report is part of the UN Economic and Social Council (ECOSOC) and the highest decision-making body for international statistical activities. By combining the annual meetings of its Consultative Group with the PARIS21 Board meeting, the TFSCB has also enhanced its legitimacy most particularly vis-à-vis developing countries. 4.11 One area where participation was found to be inadequate concerns the weak representation in the governance structure of non-statistician data users, and especially of those with policy-making experience. This problem was highlighted by the external evaluations and had attracted renewed attention at the PARIS21 Board. Related to the challenge of getting a stronger involvement of users of statistics in the governance of SCB are issues related to training and to the transparency, integrity and credibility of official data. On training, PARIS21 has organized regional workshops related to NSDSs and the TFSCB has funded training events and learning networks on various topics both at country and regional levels. In spite of this effort on training, the external evaluation of MAPS recommended that MAPS should find a more satisfactory and lasting solution to meet the urgent training needs of junior staffs in ministers’ offices and policy divisions for practice in the interpretation of statistics. According to the MAPS evaluation, such training could also help to strengthen both the understanding and demand for statistics from more senior staff. 4.12 To encourage the use of statistics, timeliness and reliability of data also need more attention. In the context of the recent global economic and financial crisis and the problems caused by the lack or non-comprehensiveness of data, such as for instance on sovereign debts, have drawn the attention of policy makers, analysts and the general public to the transparency, integrity and credibility of official data. While the three programs under review do not directly address these particular aspects, more can be learned from other experiences such as the General Data Dissemination System (GDDS) project, a statistical capacity building initiative of the Statistics Department of the IMF, which was designed to: (a) encourage IMF member countries to improve data quality; (b) provide a framework for evaluating needs for data improvement and setting priorities in this respect; and (c) guide member countries in the dissemination to the public of comprehensive, timely, accessible, and reliable economic, financial, and socio-demographic statistics. From the perspective of the users, the GDDS approach can provide a valuable body of experiences and learning. 4.13 The governance and management structures as described above were designed to ensure adequate command and control required for accountability. 40 This is particularly true in the case of PARIS21 which has a strong accountability framework with clear chain of command and control within a program starting with the annual Steering Committee/Board meetings and trickling down to the Bureau/Executive Committee members and on to the Management Unit. The minimal intervention of the OECD as host organization also helps 40. The fact that all three programs are located in host organizations -- the World Bank for MAPS and TFSCB and the OECD for PARIS21-- have raised a number of issues particularly regarding the management of the TFSCB. This will be addressed in a subsequent section in this chapter. 44 minimize interference to the governing and managing of PARIS21. For MAPS, accountability is more diffuse with the DGF Council responsible for the overall scale of funding, the internal SCBC and external Advisory Board in charge of fund allocations among the different components of MAPS, the small MAPS Unit in charge of reporting, and individual task managers providing supervision of each of the grants disbursed by DGF through monitoring of progress reports and site visits as necessary. For the TFSCB, which is providing small grants for SCB, various external evaluations confirm good financial management and efficient project selection and overall management. The use of Bank procurement rules, while considered cumbersome for the small grant-based projects, ensures good financial control. 4.14 Regarding transparency, public reporting of the progress of PARIS21 has been of high quality, and most particularly through its user-friendly Web site which also provides cross-references to key documents on the TFSCB. The knowledge base on SCB is extensive and regularly updated and responds to a wide range of user needs. This has helped the Partnership to achieve widespread recognition. Reporting on the TFSCB has been significantly improved in the last year in part as a reaction to a recommendation of the 2009 external evaluation. The TFSCB external Web site is much more comprehensive, provides cross-references to relevant material from PARIS21 and, thanks to the Bank’s new Access to Information Policy, allows access to project documentation financed by the Trust Fund and also by other World Bank sources such as STATCAP. 4.15 The relatively low profile of the MAPS governance/management structure makes for a very efficient organization. In the view of the external evaluators and endorsed by this IEG review, the MAPS’ governance is probably appropriate since it needs simply to be a non-bureaucratic mechanism for encouraging partnership and cooperation among the small number of international agencies involved in SCB. Nonetheless, compared to the other two programs, MAPS does appear to have suffered some neglect in terms of reporting. According to the 2009 external evaluation, there is no evidence of any intentional purposes but rather the result of shortage of time and resources. In the same vein, the MAPS Unit has dropped the original plan, as shown in the MAPS brochure, of sending an annual report on MAPS activities to the UN Statistical Commission and the World Bank’s Board of Executive Directors. The agencies benefiting from the DGF funding do appear to give good publicity on their Web sites to the additional activities being undertaken. 4.16 As regards efficiency of governance, indications are that the normal operating procedures and disciplines of the Bank and the other agencies, as well as the dedication of staff attracted to this kind of work and the quality of leadership appointed, together ensure attainment of high standards. The three external evaluations found that managements are cost-conscious and supervising bodies help channel available resources to uses yielding higher development return. The external Boards themselves are not very costly since they normally cover travel costs for participants coming from developing countries only; all Board members contribute their time without any remuneration from MAPS or the other agencies served. 4.17 The functioning of the governance structures of PARIS21 has shown evidence of becoming more effective over time. Recent changes including the creation of a Board with a 45 broader representation by both developing and developed countries have helped to address earlier concerns of legitimacy of the former Steering Committee. In addition, besides the traditional roles of setting strategic direction and priorities of future actions, and overseeing the design and implementation of PARIS21 work, Board members will in addition, be playing the role of “champions� for PARIS21 activities within their respective constituencies. It is hoped that these changes will first, promote the goal of developing a culture of evidence-based policy making leading to stronger demand for statistics in developing countries and second, help to mobilize more financial and institutional support for the development of statistics from developed countries. 4.18 Similarly, considerations under discussion by the TFSCB — such as to increase support for the implementation of national statistical development strategy (NSDS) compared to the earlier focus on the design and preparation of NSDSs, and to raise the share of total trust fund resources to NSDSs — reflect a conscious effort to address shortcomings identified by the external evaluation team and trust fund Advisory Panel. Progress has been less noteworthy regarding MAPS. The World Bank internal Statistical Capacity Building Committee (SCBC) — which was created in 2005 to provide guidance on DGF grant allocations among the constituent programs, and to oversee the implementation and the renewal process of these grants and to promote the coordination of SCB activity within the Bank — has met far less frequently than planned and confined its attention to the DGF grant. With the implementation of the new Statistics for Results Facility (SRF), the decision has been taken for the SCBC to play a more active role in the wider within-Bank coordination as called for when the Committee was created. 4.19 Finally, the strong complementarity between PARIS21 and TFSCB raises the question of possible efficiency gains through the merger of the two programs. Consultations for the IEG review support the rationale for two complementary but separate programs as designed by the founders of PARIS21 and TFSCB. As an example of how they work together, part of PARIS21’s mandate is to advocate, set guidelines, and provide technical support for the NSDS while TFSCB provides small grants to countries to establish their NSDS. As the host for PARIS21, the OECD is known for its expertise in advocacy and the sharing of experiences but with no experience or capacity for lending. As the host for the TFSCB, the Bank is known for its strong project-funding expertise. Also, once the NSDS is established, it can form the foundation for additional larger-scale Bank project financing for SCB. 4.20 Issues regarding program locations. As mentioned earlier, the three SCB programs being reviewed are, as is the case for most of the global and regional partnership programs, located in host organizations, with the MAPS and TFSCB located at the World Bank and the PARIS21 Secretariat at the OECD. This fact raises a number of issues including: (a) how to achieve a good working relationship between host organizations and programs; (b) roles, responsibilities, and accountabilities of each or in short, the “two masters� issue; and (c) what are the procedures for resolving issues that arise, such as different strategies for achieving results in the sector? 4.21 A brief review of the concerns mentioned above and how they apply to the three programs in relation to the locations do not reveal any conflicts as far as MAPS and PARIS21 are concerned. For the TFSCB, the QAG “Quality Assessment of Global and Regional 46 Programs and Partnerships (GRPPs)� undertaken in 2009 have highlighted the fact that the Bank has been too dominant in the design and management of the TFSCB. The Bank houses and manages the Trust Fund, administers the program, co-chairs the Consultative Group, 41 reviews and accepts or rejects project proposals, and prepares progress reports on the Trust Fund. The QAG has noted that such concentration increased the Bank’s reputational risk and prevents the program from benefiting fully from greater involvement of the other donors. 4.22 Finding the right balance for the role of the Bank in multi-donor, global initiatives such as the TFSCB is inherently difficult. One suggestion, which the present IEG review supports, would be a rotating chairmanship of the Consultative Group. The newly reformed governing structure of PARIS21 with a larger and more representative Board could provide an ideal opportunity to implement this recommendation. While the risk of an excessive involvement by the Bank remains, the risk has declined over time. For instance, the share of TFSCB projects executed by the Bank has declined significantly over the years for more than 40 percent around 2005 to less than 10 percent this is according to a recent report by the Advisory Panel. Considering the fact that there will still be the need for Bank staff’s involvement in the execution of projects particularly in small countries with very limited institutional capacity, this IEG Review does not recommend changing current procedures. Financial Sustainability 4.23 The complementarity between the three programs which has been highlighted throughout this report has important implications for the financing of the programs. At the establishment of the PARIS21 Partnership and the World Bank-led TFSCB, it was envisaged that donor agencies would be solicited for jointly supporting these two programs that have distinct but closely related purposes. As things turned out, TFSCB grants have contributed to financing several PARIS21-initiated activities and a number of projects, especially in NSDS strategic planning. This highlights the synergy of the two programs: activities stimulated in part by PARIS21 have led to the need for funding as well as providing the vehicle to help disseminate to countries information about the availability of TFSCB funds and application procedures. Common priorities and more particularly the design and preparation of national strategies for NSDSs have helped to facilitate the synergy between these two programs. 4.24 The establishment of MAPS in 2004 as a new global partnership for statistical development in developing countries added another important source of resources for statistical development, reflecting the serious underfunding of SCB as well as the fragmentation of efforts in providing statistical support in the past. The World Bank’s Development Grant Facility allocated about $5.7 million per year to MAPS for an initial five-year period to help mobilize additional investments to develop and strengthen national and international statistical systems (Table 19). This support was intended to improve the efficiency and effectiveness of Bank instruments and investments by other partners. Unlike the PARIS21 Partnership and the TFSCB which provide in-kind support and grant financing, respectively, to the final beneficiaries — country-based or regional agencies — the DGF has 41. The other co-Chair also serves as the co-Chair of the PARIS21 Board although he/she has played a more secondary role to the DECDG Director at CG meetings. 47 transferred resources to other agencies such as PARIS21 and the UN Statistics Division involved in the implementation of statistical development activities. Table 19. DGF Grant Allocations for MAPS ($ millions) Supported Supported Program FY06 FY07 FY08 FY09 FY10 Agency PARIS21 1. Support for NSDS processes 1.50 1.30 1.30 1.00 0.70 PARIS21 2. Accelerated Data Program 2.00 2.00 2.00 2.20 2.20 PARIS21 3. International Household Survey Network 0.90 0.75 0.75 1.00 1.00 UN Statistics 4. Census 2010 1.00 1.00 1.00 1.00 1.00 Division UN Statistics 5. Gender statistics - - 0.10 0.20 0.15 Division HABITAT 6. Urban indicators 0.20 0.20 0.20 - - UNECE 7. Gender statistics (Central and South East - 0.35 0.20 0.15 0.25 Europe) UIS 8. Education Statistics Capacity Building 0.18 CIS 9. 2011 International Comparisons Program - - - - 0.25 MAPS Unit 10. Advisory Board, monitoring and 0.12 0.10 0.15 0.15 0.15 evaluation Total MAPS 5.90 5.70 5.70 5.70 5.70 Source: World Bank, FY11 Development Grant Facility Budget and Reorientation of DGF Strategy, June 10, 2010. 4.25 As part of the DGF exit strategy for MAPS, agreed with the DGF Council in 2009, funding decreased from $5.7 million in FY10 to $4.2 million in FY11. Beyond 2011, it is envisaged that those components of MAPS that will continue will be increasingly incorporated into regular work programs and budgets. The Statistics for Results Facility (SRF), which was established in 2009 in response to the Third International Roundtable on Managing for Development Results (Hanoi), will also enable the continuation and scaling up of the investment in statistical development. The AfDB is also expected to scale up the implementation of MAPS actions in Africa. 4.26 One of the eight eligibility criteria for DGF grants concerns financial leverage — that “any single grant to a recipient should generally not exceed 15 percent of expected funding over the life of Bank funding to a given program.� The 2008 evaluation of MAPS did not address this issue. The 2009 MAPS Progress Report to the DGF projected that DGF funding of $22.8 million from FY2009–12 would help to generate almost half a billion dollars of other support for statistical capacity building, for an average leverage ratio of about 20 times (Table 20). IEG does not find either these projections or this leverage ratio to be legitimate without additional evidence that the DGF grants actually led other agencies to make their contributions, or that there was some kind of collective decision making, like a pledging conference, among the agencies listed in support of statistical capacity building. These projections also beg the question as to what are the boundaries of the MAPS program for the purposes of making this calculation. The calculation seems to suggest that the MAPS 48 Table 20. DGF Funding Leverage (in $ millions) FY2009 FY2010 FY2011 FY2012 Total DGF Funding 5.7 5.7 5.7 5.7 22.8 TFSCB + internal budget 4.3 3.3 3.3 3.3 14.2 Other sources of funding (including in kind) African Development 6.0 12.0 12.0 12.0 42.0 Bank (AfDB) Canadian Int'l Dev. 4.0 4.0 0 0 8.0 Agency (CIDA) DANIDA 1.0 1.0 1.0 1.0 4.0 European Commission 10.0 10.0 10.0 10.0 40.0 France 2.5 2.5 2.5 2.5 10.0 Germany 1.5 1.5 1.5 1.5 6.0 Italy 2.0 2.0 2.0 2.0 8.0 International Development Association 10.0 10.0 10.0 10.0 40.0 (IDA) International Labor 0.2 0.2 0.2 0.2 0.8 Organization (ILO) International Monetary 2.0 2.0 2.0 2.0 8.0 Fund (IMF) Norway 4.0 4.0 4.0 4.0 16.0 Netherlands Ministry of 2.0 2.0 2.0 2.0 8.0 Foreign Affairs OECD (PARIS21) 4.0 4.0 4.0 4.0 16.0 Portugal 1.0 1.0 1.0 1.0 4.0 Sweden 2.5 2.5 2.5 2.5 10.0 UK DfID 30.0 60.0 50.0 50.0 190.0 UN Economic 1.2 1.2 1.2 1.2 4.8 Commission for Europe UNECA 0.7 0.7 0.7 0.6 2.7 UNFPA 1.0 1.0 1.0 1.0 4.0 UNESCO Institute for 3.8 3.8 3.8 3.8 15.2 Statistics United Nations Statistical 2.0 2.0 2.0 2.0 8.0 Division Total 101.4 136.4 120.4 120.4 478.6 DGF Leverage Ratio 5.6% 4.2% 4.7% 4.7% 4.7% (% of Total) Source: DGF Progress Report, Marrakech Action Plan For Statistics (MAPS), P093877, 01/05/2009 program effectively includes all support for statistical capacity by all official agencies, whether this support takes the form of knowledge networking, technical assistance, or 49 investments. That the DGF Council would accept these projections as evidence of leverage suggests that this eligibility criterion has little practical import. 42 4.27 Over the years, 15 bilateral partner countries and three international agencies have contributed funding to the PARIS21 Partnership (Table 21). It is worth noting that, in addition to contributing to PARIS21’s core program which consists of statistical development activities in country or regional levels, advocacy, partnerships, studies and knowledge development, the World Bank’s DGF grants have provided the bulk of financing for two so-called “satellite programs� of PARIS21, namely, the Accelerated Data Program (ADP) and International Household Survey Network (IHSN) (Table 19). Table 21. PARIS21 Core Program Contributions (in Euros) Country/ 1999–2007 2008 2009 2010 2011 2012 2013 Institution Austria 200,000 Belgium 1,000,000 200,000 Canada 966,370 254,300 EFTA 80,000 European 16,226 121,185 150,000 Commission Finland 200,000 100,000 France 1,424,000 45,000 Greece 0 200,000 Ireland 762,499 100,000 Italy 0 200,000 Japan 68,602 Netherlands 1,050,000 Norway 579,262 198,879 Spain 0 250,000 250,000 250,000 250,000 Sweden 660,519 Switzerland 597,695 99,445 United Kingdom 3,904,596 833,155 555,436 555,436 277,718 World Bank 3,956,992 780,470 481,729 Other income 25,050 Total Income 15,491,812 1,651,655 1,579,353 1,283,155 805,436 555,436 277,718 Source: PARIS21, Progress Report for the Partnership in Statistics for Development in the 21st Century (Reporting Period: November 2009–May 2010) 4.28 Since its establishment in 2004, TFSCB has received over $46 million in donor contributions from six partners, namely, Canada, France, Germany, the Netherlands, 42. See also IEG 2011, pages 87–88. The DGF Council has generally been very lax in allowing programs to include expenditures in the denominator that do not relate closely to the activities being supported by the DGF grant. 50 Switzerland, and the United Kingdom. Although the donor composition has been different for each of the three TFSCB funding rounds (I, II, and III), the contribution level has been stable over time. TFSCB III, funded by Canada, the Netherlands, and the United Kingdom, received the highest contribution. About $35 million has already been disbursed or allocated to ongoing projects. In the past five years, the average annual disbursement/commitment amount for projects was about $4 million. Total available funds, calculated by combining the unallocated funds from TFSCB I, II, and III, amount to $6 million as of July 2010. At the current rate of demand and project approvals, this amount is expected to be sufficient to fund future projects under the NSDS and the SCB windows until mid-2011. DECDG has initiated discussions with current contributing donors to the Trust Fund and there is broad agreement that they are willing to extend the validity of TFSCB III to 2015, using about $2 million of unspent resources from TFSCB I and TFSCB II, which have been closed. 4.29 Because of the complementarity in funding between the three programs, the phasing out of DGF grants for MAPS will require a major effort by the PARIS21 Partnership to broaden the financing base beyond current donors to include the participation of private foundations, for instance. This will also require developing at the PARIS21 Secretariat, in- house expertise in fund raising, including the development of a fund-raising database and materials, and identifying SCB “champions� to support a resource mobilization campaign, which will be launched soon. The recent modification of the PARIS21 governance structures was driven in part by the need to search for new funding sources. 51 5. The World Bank’s Performance as a Partner 5.1 The World Bank plays many roles in the three programs and acts as financial contributor, trustee, convener, chair or member of the governing body, implementing agency, and host of the secretariat (Table 22). Through these multiple roles, the Bank has contributed significantly to an international partnership in support of SCB in developing countries. More importantly, given the strong complementarity between the three programs as discussed earlier in this report, the active involvement of the Bank in all three programs has helped to enhance the role of the Bank in the SCB partnership. Table 22. Roles Played by the World Bank in the Three Programs MAPS PARIS21 TFSCB Member of the governing body X X X Chair or Co-Chair X X Convener X X Trustee X Financial contributor X X Implementing agency X Host of the secretariat X X The WBG’s Contributions at the Global Level 5.2 Initiator. As a founding member of all three programs under review, the World Bank played an important role in the international effort which began in the late 1990s to promote SCB in developing countries in support of evidence-based policy-making. In the context of the Millennium Declaration, the Bank, together with other partners, helped to establish PARIS21 in 1999 to build and strengthen national statistical systems in developing countries. To complement PARIS21 activities, the TFSCB was set up in 1999 as a World Bank-administered, multi-donor trust fund to provide financial resources to developing countries for SCB activities. Spearheaded by the World Bank, MAPS was agreed at the Second Roundtable on Managing for Development Results in Marrakech In 2004, to broaden the efforts at both national and international levels to help developing countries achieve stronger capacities in four specific areas namely strategic planning for statistics, the 2010 Census Round, management of statistical surveys, and MDG monitoring. 5.3 Through its active involvement in the three programs, the Bank helped to build a broad network of partners involved in SCB, including statistical agencies, UN agencies, regional development banks, bilateral donors, and developing countries. The active participation of the Bank in all three programs further enhances their complementarity. Most noteworthy, MAPS, which provided the overall action plan for statistics, was purposefully designed to provide support through funding international statistical agencies involved in the development of statistical capacity. The allocation of DGF funds to several agencies including the UN Statistics Division, UN-Habitat, UNESCO Institute of Statistics and UNECE helped to develop broad buy-in from the statistics community to the development effort. 52 5.4 Catalyst. In Sub-Saharan Africa where the need for statistical support is more important relative to other developing regions, consultations carried out for this IEG review reveal an important contribution. Shortly following the establishment of MAPS, several key African agencies adopted the Reference Regional Strategic Framework for Statistical Capacity Building in Africa (RRSF) 43 which was designed to play a role in the regional context for Africa similar to that of MAPS in the global context. Using the RRSF as a framework, the African Development Bank (AfDB), the leading provider of funds for SCB, was aiming to double its financial support in this area. The AfDB’s Statistics Department has expanded its mandate from a more narrow focus on generating purchasing power parity (PPP) estimates for cross-country comparisons as part of the International Comparison Program to leading the effort in supporting African countries, financially and technically, to develop or update their NSDSs. The AfDB also contributes to the MAPS- financed Accelerated Data Program. The WBG’s Contributions at the Country Level 5.5 Country level knowledge. The review of the external evaluations and other consultations undertaken in this review highlighted some of the Bank’s comparative advantage. First, the Bank was already heavily involved in country SCB primarily through the Living Standards Measurement Study (LSMS) which was established in the 1980s to foster increased use of household data as a basis for policy decision-making and, in the process, also facilitate communications between statisticians and policy makers and analysts. Second, the introduction of PRSPs in 1999 and regular reviews by staff of the World Bank and International Monetary Fund of PRSP implementation has helped to raise the demand for statistics for monitoring and evaluation purposes. 5.6 In some countries such as Mali, the reviews and updates of the PRSP have led to the demand for a stronger integration of NSDSs in the PRSPs. Experiences drawn from PRSP financing has led to the adoption of new approaches to enhance the effectiveness of the support for capacity building in statistics. These take various forms including (a) the formation of donors’ groups with a lead donor such as for instance the United Kingdom for Ghana and UNDP for Cape Verde, (b) new funding mechanisms such as the setting up of a “basket-of-funds� approach to finance SCB in Burkina Faso, and (c) funding for statistical capacity support as part of the PRSP as in the case of Uganda. 5.7 Lending activities. Given the importance of statistical monitoring and the limitations on the availability and quality of data on country-level development results, the International Development Association (IDA), the World Bank’s concessional lending facility for low-income countries, has recommended support for investment in this area. Some countries have used IDA resources to support SCB projects and the implementation of NSDSs, either through the World Bank Statistical Capacity Financing Program (STATCAP)44 program or as part of larger operations. The Bank’s insistence that 43. See footnote 7 for more detail. 44. STATCAP is a horizontal adaptable program loan, approved in 2004, designed to simplify procedures for developing and implementing SCB projects. 53 countries must have an NSDS as a prerequisite to a STATCAP loan has further strengthened support for NSDSs. To encourage further investment at the country level, a new trust fund, the Statistics for Results Facility Catalytic Fund (SRF-CF),45 has been established and is providing support to five pilot IDA-eligible countries that have weak statistical capacity. 5.8 Data made available on the World Bank Bulletin Board on Statistical Capacity show that cumulatively, the Bank through its own resources and the Trust Fund for Statistical Capacity Building has funded a total of 239 projects. These include all projects which are either stand-alone operations or with significant components aimed at statistical development financed by the World Bank loans, credits and grants as well as projects which are financed by the TFSCB. The majority of these projects are country-specific but regional or multi-country projects are also significant, particularly in Africa where they account for about one-third of the total number of projects (Table 23). Table 23. Statistical Capacity Building Projects Funded by or through the World Bank TFSCB Other funding All funding Sub-Saharan Africa 64 34 98 East Asia and Pacific 23 4 27 Europe and Central Asia 22 18 40 Latin America and Caribbean 27 12 39 Middle East and North Africa 14 4 18 South Asia 7 10 17 All Regions 157 82 239 Source: World Bank, Bulletin Board on Statistical Capacity (http://go.worldbank.org/QVSQM1R6V0 5.9 But in spite of this diversified lending portfolio for statistical development, the IDA Results Measurement System which aims to mainstream statistical capacity building efforts in country assistance strategies (CAS) has yet to have the desired impact. As summarized in Table 8 in Chapter 3, only three of the 21 most recent CAS Reports followed the guidelines spelled out in a Statistical Capacity Building Guidance Note produced to help country teams on how to approach the assessment of national statistical capacity and how to address its shortages in a CAS or CPS program. A significant effort to give more importance to statistical capacity building particularly from the perspective of the host country is still required. 45. A key focus of the SRF-CF is greater attention to aid effectiveness principles, including the use of a system wide approach. The TFSCB and new SRF-CF will be complementary: TFSCB will provide resources on a limited scale to help countries implement priority actions and develop good-quality implementation plans, and the SRF-CF will help mobilize large-scale investment resources needed to put those plans into action. The five pilot countries are Afghanistan, Democratic Republic of Congo, Ghana, Nigeria, and Rwanda. 54 Oversight 5.10 MAPS. Upon the recommendation of the DGF Council, the Bank’s Statistical Capacity Building Committee (SCBC) was established in 2005 to oversee the implementation of the MAPS’ DGF grant and to coordinate the preparation of future requests for DGF funding. The committee has played that role since 2005, meeting once a year except for 2007 when it met twice. As expressed in its original mandate, the SCBC was, however, supposed to play a more far-reaching role of ensuring better coordination of SCB activities within the World Bank as a key component of the overall results agenda. The committee was also supposed to meet on two or three occasions each year and to include representatives from eight of the Bank’s networks and regions involved in the results agenda and SCB. Until recently, the SCBC has been made up only of representatives from OPCS, HDN, AFR and DEC meeting under the chairmanship of the Director of OPCS Country Services. The more restricted role played by the SCBC and its more limited membership have, to some extent, undermined the impact it could have had within the Bank on advocacy for and promotion of SCB in developing countries as a prerequisite for the overall results agenda. 5.11 For MAPS, day-to-day decisions are made by the managers in the recipient agencies. The Bank’s oversight varies with each partnership. The partnership with the UN recognizes the need to collaborate and synergize with other major players in the international statistical capacity building landscape. The UN Statistics Division and the World Bank’s Development Data Dept are in constant contact to ensure work programs are harmonized. 5.12 TFSCB. The Bank plays a much more dominant role in the TFSCB compared to that for MAPS. Besides hosting and managing the trust fund, the Bank led the design the program, raises funding, reviews project proposals and selects those for funding and co- chairs the Consultative Group which is the governing body for TFSCB. This has led the 2006 IEG review of the TFSCB — as a pilot exercise under the newly launched review of global and regional partnership programs (GRPPs) — to raise concerns about the potential for a lack of objectivity and conflicts of interest that compromise the Bank’s conduct of these roles. The QAG’s quality assessment of the TFSCB in 2009 flagged the fact that the dominance of the Bank’s role increases its reputational risk; furthermore, such concentration of roles runs the risk of reducing the benefits from a greater involvement of the other donors. 5.13 According to the governance and management structures described in the previous chapter, there is no provision for Bank’s oversight independent of the management of the program. Instead, oversight is loosely provided through the Consultative Group which is supported by reports of the Advisory Panel, progress reports from DECDG, and external evaluations. But in spite of the loose oversight arrangements, assessments by external evaluations tend to confirm that the existing arrangements tend to be working. For instance, the 2008 DfID evaluation 46 concluded that “financial management of the trust 46. Evaluation of DFID Support to the World Bank Trust Fund for Statistical Capacity Building - Strategic and Management Overview, February 2008. 55 fund was sound and that the application process was efficient and that the Internal Management Committee (IMC) communicates well and approves country proposals as swiftly as can be expected although communication of IMC’s decisions has not always been effective.� 5.14 According to the 2009 external evaluation, “project formulation and their review/approval run efficiently based on well-established practices. The origin of project ideas is evenly divided among client agencies, PARIS21 and World Bank staff and does not seem to be supply-driven. The use of the World Bank Grant Funding Request system and guidance from the Trust Fund Administration Unit (TFAU) in using it, have helped speed up project formulation and approval. The IMC’s decision process works well and decisions are well documented.� Project monitoring is primarily done by the TTL who must submit a progress report every six months, but there are concerns over low compliance. The lack of ex-post evaluation of projects has not made it possible to draw on lessons from the past to improve the design of new projects. 5.15 A careful chronological review by the IEG Review of the Advisory Panel annual reports’ assessments and recommendations on the basis of reviews of projects submitted for funding (by type, by region etc.) and subsequent decisions made by the Consultative Group (CG) tend to support the fact that there is a reasonable degree of feedback that have led to management improvement over the years. Among the more significant management changes, one can mention the revised guidelines that led to the shift from the dominance of Bank-executed projects to a dramatic increase in country-executed activities, and the coordination (back-to-back) of the annual meetings of the PARIS21 Steering Committee/Board and the TFSCB Consultative Group which made it possible for the greater participation of developing countries in the meetings of the CG. 5.16 PARIS21. The Bank serves on both the Board and the Executive Committee of PARIS21 and thus provides strategic guidance and monitoring of progress of the activities undertaken by the Secretariat. Disengagement Strategy 5.17 The calls at the national and international levels for effective donor support and coordination for the development of national statistical systems will likely be sustained by the focus on meeting the MDGs by 2015, by efforts toward managing for development results by bilateral and multilateral development agencies, by the monitoring and evaluation of PRSP implementation, and by new demand for data to address the challenges posed by climate change and environmental sustainability in general. This is true for most developing countries, but even more relevant to the poorer countries including most African countries. 5.18 MAPS. When funding for MAPS began in FY06, there was a broad understanding that funding could be expected to be renewed annually through FY2010 provided the individual programs performed up to expectation. The 2008 external evaluation recommended that funding be maintained at the levels of FY09 to at least 2012. With the reorientation of DGF strategy which called for the exit of the programs under the long-term 56 funding DGF window (also known as Window 2), the DGF Council discussed and DEC and OPCS agreed on a gradual reduction in funding with specific exit strategies for each component funded under MAPS. As shown in Table 24, DGF funding will cease for all other activities except for programs managed by the PARIS21 Secretariat, justified on the basis that these activities have been or will shortly be completed. Table 24. Exit Plan for MAPS Program (FY11–15) ($ millions) Supported Supported Program FY09 FY10 FY11 FY12 FY13 FY14 FY15 Agency PARIS21 1. Support for NSDS processes 1.00 0.70 0.50 0.30 0.30 0.30 0.30 PARIS21 2. International Household Survey Network 1.00 1.00 1.00 1.00 0.70 0.50 0.50 PARIS21 3. Accelerated Data Program 2.20 2.20 2.00 2.00 1.30 1.00 0.70 UNSD 4. Census 2010 1.00 1.00 0.50 Exit – – UNSD 5a. Gender statistics (informal 0.20 – Exit – – – employment) UNSD 5b. Gender statistics – 0.15 0.20 0.20 0.20 0.20 Exit UNECE 6. Gender statistics (Central and South 0.15 0.25 East Europe) HABITAT 7. Urban indicators Exited – – – – – UIS 8. Education statistics capacity building Exited – – – – – MAPS Unit 9. Advisory Board, monitoring and 0.15 0.15 0.05 – – – evaluation ICP-CIS* 10. International Comparison Program – 0.25 0.15 Exit – – (CIS countries) Total MAPS 5.70 5.70 4.20 3.50 2.50 2.00 1.50 Note: * New program, only for start-up phase of the 2011 program. Source: DGF Progress Report, Marrakech Action Plan for Statistics (MAPS), March 2010 and DECDG, Proposed FY12 DGF Allocations Supporting the Marrakech Action Plan for Statistics (2011). 5.19 PARIS21. For the DGF-financed programs managed by PARIS21, the exit by the Bank by FY13 leaves open the question of the sustainability or at least the scale of these programs. As confirmed by the full partnership (also known as the Consortium) meeting in Dakar in November 2009, PARIS21 is the main forum and network for development partners and countries to support global SCB efforts in developing countries. Whereas PARIS21 receives funding from a large number of bilateral donors, with few exceptions, these contributions tend to be small and unlikely to offset the reduced funding by the Bank which has been the largest provider of funding for PARIS21’s programs, if the satellite programs are included. 5.20 At the Board meeting in June 2010, it was agreed that the PARIS21 Secretariat will start a fund raising program that would focus on new sources of funding including private foundations as well as encourage governments in developing countries to increase funding from budgetary resources for NSDSs. The new Statistics for Results Facility (SRF) which was established to help developing countries implement the national statistical strategies can be expected to help partly alleviate the impact of the gradual reduction of DGF funding in 57 general. The experience to date with the SRF shows that it may take more time for a full launch of the facility in the five pilot countries 47 although there are indications that other countries have expressed a strong interest in accessing the SRF to support SCB. It is therefore necessary for the Bank to continue to look at various delivery mechanisms including a stronger reliance on policy-based lending and other PRSP-related financing to help support SCB in developing countries. 5.21 The reduction in and eventually the end of DGF funding for the Accelerated Data Program (ADP) and the International Household Survey Network (IHSN) will likewise require a more in-depth review of options to ensure their sustainability after the Bank’s exit. The Accelerated Data Program has been one of the most successful MAPS programs. Designed as a pilot program with the overall objective to make existing survey data more widely and easily accessible, ADP has evolved very rapidly and the original target of 12 pilot countries in three years has been greatly exceeded; almost 60 countries are now participating. The IHSN provides a network for survey practitioners working in developing countries, and provides important tools and guidance material. In spite of their relevance and effectiveness, these programs have been almost exclusively funded by DGF resources. There is therefore the urgent need to search for ways to diversify funding sources as well as to review the Bank’s long-term role and engagement in these two important programs. 5.22 TFSCB. The Bank-administered TFSCB continues to play an important role in implementing MAPS and more particularly, complement PARIS21’s core program. Since its establishment, TFSCB has received over $46 million in donor contributions from six partners, namely, Canada, France, Germany, the Netherlands, Switzerland, and the United Kingdom. As of mid-2010 about $35 million have been disbursed or allocated to ongoing projects. At the average annual commitment rate of $3 to $4 million, total available (unallocated) funds are expected to be exhausted by mid- or late-2011. An agreement in principle has been obtained from trust fund donors to continue funding of the trust fund by extending the operation of TFSCB III until 2015, using about $2 million of resources left over from TFSCB I and TFSCB II, which have been closed. The prospects for additional funding from donors will be explored in the latter half of 2011, since there is no explicit disengagement strategy. 47. The five pilot countries are Afghanistan, Democratic Republic of Congo, Ghana, Nigeria, and Rwanda. There are, however, indications that other countries have shown interest in having access to the SRF. 58 6. Findings and Lessons Main Findings 6.1 Significant progress has been achieved in the primary objective of encouraging and supporting developing countries to design National Strategies for the Development of Statistics (NSDSs), but only some progress has been made in NSDS implementation. The main finding of this review is that the joint program of activities implemented through MAPS, PARIS21 and the TFSCB has partly achieved the primary objective by giving prominence to the need for SCB both at the level of countries and among a small group of donors. The quality of statistics as measured by the World Bank Statistical Capacity Indicator has shown some improvement in the past 10 years, but the contribution of the three programs to the broader goals of developing a culture of evidence-based policy making and more specifically of reducing poverty and achieving the MDGs remains difficult to evaluate. 6.2 Statistical capacity building programs need to involve the users of statistics more actively. Through its advocacy work, PARIS21 has begun to broach this issue. It was furthermore agreed at its last board meeting in June 2010 that the PARIS21 Secretariat should give more attention and provide more support to involving the users of statistics, and more particularly policymakers and advisers in the SCB process. Concerns about involving the users of statistics have yet to receive the same attention in MAPS and in the TFSCB. Providing more weight to the users of statistics in the design as well as the implementation of SCB programs would also imply the need to revisit the current governance arrangements of all three programs. 6.3 There appears to be a shift in the perception by developing countries toward statistical capacity building. Whereas low-income countries were reluctant to access resources other than grants to strengthen their statistical systems, recent experiences appear to show that this is changing. For instance, several countries in Africa either have already borrowed external resources or are in the process of doing so to invest in statistical capacity building. As a result, statistical development has begun to receive the same priority as other sectors instead of being perceived as a requirement imposed by external development partners for monitoring and evaluation purposes. 6.4 There is a need to strengthen the Bank’s commitment toward statistical capacity building activities. This conclusion is based first, on the continuing and rising relevance of the original objectives of the three programs as confirmed by the external evaluations and the renewed attention to aid effectiveness with the fast-approaching deadline to meet the MDGs by 2015. Second, the lack of or inadequate realization of the importance of SCB by the Bank's operational departments as discussed in Section 3 in spite of the need to monitor the implementation of the CAS (and the PRSP) mean that there is a need to redouble efforts on advocacy for statistical development among non-statisticians. 6.5 There is also the need to revisit the Bank’s engagement following the decision to phase out DGF funding for PARIS21’s core program. A meeting organized by the PARIS21 Secretariat in February 2011 to discuss SCB funding at the country level 59 highlighted three important challenges that the Bank could help address. The meeting confirmed (a) that bilateral funding for SCB will most likely decline as OECD governments are experiencing severe fiscal consolidation; (b) that current funding practices for SCB at the country level are characterized by heavy aid concentration — 45 percent of total aid commitments for SCB worldwide are destined to 15 countries; and (c) that current funding is excessively fragmented — 38 percent of aid relationships in SCB to the 15 countries being insignificant — and weakly aligned with NSDSs. Donor coordination therefore represents a major challenge and the Bank could play a critical role in helping to improve this, based on the experience it has developed over time. 6.6 More attention should be given to training and to the quality of statistics in order to more actively involve the users of statistics. The evaluation of the TFSCB pointed out concerns expressed by national statistical authorities for more effort on staff training to encourage the demand and use of data notwithstanding the fact that other bottlenecks will also need to be addressed. The evaluation of PARIS21 highlighted the limited capacity for training of statisticians. Furthermore low public service salaries have made it difficult to keep good statisticians in the public services. Consultations carried out for the IEG Review reveal that this situation has led statistical staff to prefer attending meetings where there are paid fees rather than doing their regular work. PARIS21’s work with the African Group on Statistical Training (AGROST) is an example of work to be promoted. Lessons 6.7 The review of SCB’s effectiveness highlights four main lessons, of which three are relevant to the design and implementation of global programs and one is relevant to the Bank’s support of such programs. 6.8 Effectiveness requires explicit strategies for achieving outcome objectives. The focus on helping strengthening national statistical systems has provided the common ground for good collaboration between the three SCB programs. Good progress has been achieved in helping countries design NSDS which saw most IDA-eligible countries having prepared an NSDS. But without a clear road map and related operational strategies, implementing the NSDSs has proven to be a major challenge. The situation is made even more complicated in that there is no commonly agreed definition as to what implementation means for an NSDS which, by definition should be defined by the country according to its own specific priorities. In the last two years, there have been efforts, using the experience of PRSP implementation, to help countries design more realistic and fully costed NSDS, but the experience is too recent to assess its impact. PARIS21 has also been experimenting with applying the peer review approach of the OECD whereby country strategies for statistical development are reviewed by their peers. This practice has proven its worth where there are good practices to share. This remains to be seen for statistical development where lessons of good practices are not well publicized. 6.9 Some selectivity may be needed to make progress. By setting a rather ambitious target — all countries should have prepared strategies for the development of their 60 statistical systems by 2010 and have capacity in place in order to monitor progress towards the MDGs by 2015 — MAPS, which financed the larger share of the work by PARIS21, had set the tone for a clear focus on speed and comprehensiveness regarding NSDSs. The experience in other capacity building exercises including experimentation by PARIS21 with peer review shows that not all countries can proceed at the same pace particularly given the shortage of funding and of staff capacity. Looking forward, a more selective approach with more intensive support to selected countries, somewhat along the lines of the pilot countries under the new Statistics for Results Facility (SRF) may be a more realistic approach. 6.10 Coordinated financial support across donors for statistical capacity building at the country level is important for moving the agenda forward and could benefit from documenting and sharing different approaches widely. A basket-of-funds approach such as in the case of Rwanda or PRSC-type support for statistical capacity development as in the case of Lesotho are various initiatives being experimented with to help move support for statistical capacity building away from a narrow project-type approach to a broader country-led capacity building exercise. 6.11 Awareness gap on the need for statistical development between DECDG’s professional statisticians and the Bank’s operational staff may require stronger advocacy efforts inside the Bank. Consultations for the IEG review reveal that there continues to be a significant awareness gap between DECDG staff with a more focused attention on broader statistical development based on NSDSs and the Bank’s operational staff with a more restricted view on monitoring of CAS implementation with little emphasis on broader capacity building approach based on the NSDS. In addition, the monitoring of the MDGs by specific units in the Bank or in the UN system has, to some extent, relieved the country’s operational staff from a more holistic perspective on statistics. For reasons similar to those which justify the efforts by PARIS 21 on advocacy48 for SCB in developing countries, DECDG may consider playing a more active role on advocating for the importance of statistical development at country level among the Bank’s operational staff. 48. “Advocacy is pleading for, defending or recommending an idea before key people� in order to obtain change. 61 References General Independent Evaluation Group. 2011. The World Bank’s Involvement in Global and Regional Partnership Programs: An Independent Assessment. Washington, DC: World Bank. International Development Association, Special Themes for IDA16, IDA Resource Mobilization Department (CFPIR), May 2010 United Nations Statistical Commission, Report on the fortieth session, 24 to 27 February 2009. United Nations Statistical Commission, Report on the forty-first session, 23 to 26 February 2010. MAPS African Development Bank, Statistical Capacity Building, www.afdb.org/en/knowledge/statistics Badiee, Shaida, Better Data for Better Results - An Action Plan for Improving Development Statistics, Presentation to UN Statistical Commission Eele, Graham, The Role of the World Bank in Supporting Statistical Development in Africa, DECDG, The World Bank, 2006. Kiregyera, Ben, Reference Regional Framework for Statistical Capacity Building in Africa (RRSF), Presentation on 19 October, 2007, African Centre for Statistics, UNECA, Addis Ababa. Willoughby, Christopher and Philip Crook, Marrakech Action Plan for Statistics, Report of an Independent Evaluation, 8 December 2008 World Bank, Better Data for Better Results - An Action Plan for Improving Development Statistics. Presented to the Second International Roundtable on Managing for Development Results Marrakech, Morocco, February 4-5, 2004 Third International Roundtable on Managing for Development Results, Better Data for Better Results – A Background Paper. Presented to the Third International Roundtable on Managing for Development Results, Hanoi, Vietnam, February 5-7, 2007. PARIS21 Roberts, Richard and Claudine Voyadzis. Mark Baldock, Evaluation of PARIS21 (Final Report), November 2009. PARIS21 Bureau, Special Committee on Logical Framework, Governance and Funding Issues, PARIS21’s strategic planning for 2010-2014: logical framework, governance and funding issues, June 2010. PARIS21 Consortium, Dakar Declaration on the Development of Statistics, November 2009. PARIS21 Secretariat, Partner Report on Support to Statistics – PRESS, 2010 Round, Final Report, November 2010. PARIS21 Secretariat, National Strategies for the Development of Statistics - Progress Report: NSDS Summary Table for IDA and Lower Middle Income Countries, March 2011. 62 PARIS21 Secretariat, Advocating for the Development of National Strategies for the Development of Statistics, Country Level Tool Kit, May 2010 PARIS21 Secretariat, A Guide to Use a System Wide Approach to Implement National Strategies for the Development of Statistics, October 2007. PARIS21 Consortium, PARIS21 at Ten: Improvement in Statistical Capacity since 1999. Progress Report for the Partnership in Statistics for Development in the 21st Century (PARIS21) Reporting Period: May 2008 – September 2008 Progress Report for the Partnership in Statistics for Development in the 21st Century (PARIS21), Reporting Period: November 2009–May 2010 PARIS21 Steering Committee Meeting, Record of the Proceedings 26 April 2006. PARIS21 Steering Committee Meeting, Record of the Proceedings, November 2006. PARIS21 Steering Committee Meeting, Record of the Proceedings, November 2007. PARIS21 Steering Committee Meeting, Record of the Proceedings, November 2008. PARIS21 Steering Committee Meeting, Record of the Proceedings, November 2009. PARIS21 Steering Committee Meeting, Record of the Proceedings, November 2010. PARIS21 Steering Committee Meeting, Tracking Progress of the PARIS21 Partnership’s Outputs and Outcomes 2008 Report, October 2008. PARIS21 Steering Committee Meeting, Tracking Progress of the PARIS21 Partnership’s Outputs and Outcomes – 2009 Report, October 2009. Progress Report for the Partnership in Statistics for Development in the 21st Century (PARIS21), Reporting Period: October 2008 to April 2009. Spitz, Pierre, PARIS21 Light Evaluation, Final Draft Report, June 2006. Willoughby, Chris, Anne Thomson, Xavier Charoy and Juan Munoz, An Evaluation of PARIS21, An Initiative to Improve the Contribution of Statistical Services to Development, Oxford Policy Management, August 2003 TFSCB Alguhas Consulting, Evaluation of DFID Support to the World Bank Trust Fund for Statistical Capacity Building Strategic and Management Overview, February 2008 Bodin, Jean-Louis and Chandrakant A. Patel, Report of the Advisory Panel (AP) to the Consultative Group (CG) of the TFSCB, October 14, 2003, Washington DC. Snorrason, Hallgrímur, Andrew J. Flatt and Jette Jensen, World Bank Trust Fund for Statistical Capacity Building Evaluation Report, January 2010. Thomson, Anne, Christopher Willoughby and Ramesh Chander, The World Bank Trust Fund for Statistical Capacity Building - An Evaluation, Final Report, November 2003. World Bank, Trust Fund for Statistical Capacity Building – Minutes of Donors’ Meeting, November 2009. Paris. World Bank, Trust Fund for Statistical Capacity Building – Minutes of Donors’ Meeting, June 2010. Paris. 63 World Bank, Trust Fund for Statistical Capacity Building – Report of the Second Meeting of the Consultative Group, October 2, 2001, Paris. World Bank, Trust Fund for Statistical Capacity Building – Report of the Third Meeting of the Consultative Group, October 16, 2002, Paris. World Bank, Trust Fund for Statistical Capacity Building – Report of the Third Meeting of the Consultative Group, October 14, 2003, Paris. World Bank, Trust Fund for Statistical Capacity Building – Report of the Third Meeting of the Consultative Group, October 14, 2003, Paris. World Bank, Trust Fund for Statistical Capacity Building – Report of the Third Meeting of the Consultative Group, October 14, 2003, Paris. World Bank, Trust Fund for Statistical Capacity Building – Report of the Third Meeting of the Consultative Group, October 14, 2003, Paris. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2001 – April 2002, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2001 – September 2002, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2002 – March 2003, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2002 – September 2003, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2003 – April 2004, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2003 – September 2004, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2004 – April 2005, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2004 – September 2005, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2005 – April 2006, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2005 – September 2006, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2006 – April 2007, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2006 –September 2007, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2007 – April 2008, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2007 – September 2008, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. 64 World Bank, Trust Fund for Statistical Capacity Building Progress Report, May 2008 – October 2008, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, May 2008 – April 2009, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, May 2009 – October 2009, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, October 2009 – April 2010, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress Report, April 2010 – March 2011, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building at Ten – Annual Report 2009/10, TFSCB Internal Management Committee & TFSCB Administration Unit, Washington DC. World Bank, Trust Fund for Statistical Capacity Building Progress – Guidelines and Procedures, August 2005. World Bank, Trust Fund for Statistical Capacity Building Progress – Guidelines and Procedures, September 2009. World Bank, Trust Fund for Statistical Capacity Building Progress – Guidelines and Procedures, August 2005. Other References World Bank, Statistics for Results Facility – Catalytic Fund Annual Report, 2010. World Bank, Statistics for Results Facility – Catalytic Fund Annual Report, 2011. 65 Annex A Annex A. Evaluation Framework for Global Program Reviews Note: This evaluation framework is a general framework that has been designed to cover the wide range of such programs in which the World Bank is involved, encompassing policy and knowledge networks, technical assistance programs, and investment programs. It is not expected that every global program review will cover every question in this table in detail. Annex Table 1. Assessing the Independence and Quality of the Evaluation Evaluation Questions 1. Evaluation process To what extent was the GRPP evaluation independent of the management of the program, according to the following criteria: • Organizational independence? • Behavioral independence and protection from interference? • Avoidance of conflicts of interest? Factors to take into account in answering these questions include: • Who commissioned and managed the evaluation? • Who approved the terms of reference and selected the evaluation team? • To whom the evaluation team reported, and how the evaluation was reviewed? • Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions on access to information, travel, sampling, etc.? 2. Monitoring and evaluation framework of the program To what extent was the evaluation based on an effective M&E framework of the program with: • Clear and coherent objectives and strategies that give focus and direction to the program? • An expected results chain or logical framework? • Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the program? • Systematic and regular processes for collecting and managing data? 3. Evaluation approach and scope To what extent was the evaluation objectives-based and evidence-based? To what extent did the evaluation use a results-based framework — constructed either by the program or by the evaluators? To what extent did the evaluation address: • Relevance • Governance and management • Efficacy • Resource mobilization and financial management • Efficiency or cost-effectiveness • Sustainability, risk, and strategy for devolution or exit 4. Evaluation instruments To what extent did the evaluation utilize the following instruments: • Desk and document review • Consultations/interviews and with whom • Literature review • Structured surveys and of whom • Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being implemented or completed • Case studies • Other Annex A 66 Evaluation Questions 5. Evaluation feedback To what extent have the findings of the evaluation been reflected in: • The objectives, strategies, design, or scale of the program? • The governance, management, and financing of the program? • The monitoring and evaluation framework of the program? Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program Every review is expected to cover the first four criteria in the following table: (a) relevance, (b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover (e) resource mobilization and financial management and (f) sustainability, risk, and strategies for devolution or exit if the latter are important issues for the program at the time of GPR, and if there is sufficient information available on which to base an independent opinion. Evaluation Criteria and Questions Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. 1. Supply-side relevance — the existence of an international consensus that global/regional collective action is required. To what extent does the program reflect an international consensus on the need for action, on the definition of the problem being addressed, on priorities, and on strategies for action? Is the original consensus that led to the creation of the program still present? Is the program still needed to address specific global/regional public concerns? Take into account the origin of the program in answering these questions: • Is the program formally responsible for implementing an international convention? • Did the program arise out of an international conference? • Is the program facilitating the implementation of formal standards and approaches? • Is the program primarily donor-driven? Did donors establish the program with little consultation with developing countries? • Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners? 2. Demand-side relevance — alignment with beneficiary needs, priorities, and strategies. To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as articulated in the countries’ own PRSPs, and in donors’ strategies such as the World Bank CASs, and the UN Development Assistance Frameworks? To what extent has the voice of developing and transition countries been expressed in the international consensus underlying the program? 3. Vertical relevance — consistency with the subsidiarity principle. To what extent are the activities of the program being carried out at the most appropriate level — global, regional, national, or local — in terms of efficiency and responsiveness to the needs of beneficiaries? To what extent are the activities of the program competing with or substituting for activities that individual donors or countries could do more efficiently by themselves? Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or local public goods. 67 Annex A Evaluation Criteria and Questions 4. Horizontal relevance — the absence of alternative sources of supply. What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or technical capacity that is otherwise unavailable to meet the program’s objectives? To what extent are the good and services being provided by the program in the nature of public goods? Are there alternative ways of providing these goods and services, such as by the private sector under regular market conditions? 5. Relevance of the design of the program To what extent are the strategies and priority activities of the program appropriate for achieving its objectives? What are the major activities of the program: • Policy and knowledge networking? • Financing country and local-level technical assistance? • Financing investments to deliver national, regional, or global public goods? (See Annex Table 4.) Has the program articulated an expected results chain or logical framework, along with assumptions that relate the progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to what extent are these assumptions valid? For programs providing global or regional public goods, is the design of the program consistent with the way in which the individual efforts of the partners contribute to the collective outcome for the program as a whole — whether “best shot�, “summation�, or “weakest link?� Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. 6. Achievement of objectives To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made towards achieving these objectives? To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the program should also be held accountable? To what extent are there any positive, unintended outcomes of the program that have been convincingly document? To what extent have these assessments by the program or the evaluation been evidence-based? 7. Progress of activities, outputs, and outcomes. To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes? How did the program or the evaluation aggregate its outputs and outcomes at all levels — global, regional, national, and local — to provide an overall summary of its results? To what extent have factors such as changes in the location of the program, its legal structure, or governance processes affected the outputs and outcomes of the program? To what extent have there been outcomes that can be uniquely attributed to the partnership itself — such as the scale of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host organization? 8. Linkages to country or local-level activities. To what extent has the program established effective operational linkages with country-level activities, taking into account that: • The desired nature of these linkages will vary according to the objectives, design, and implementation of each program? • Positive outcomes at the country or local level are generally a joint product of both global/regional and county- level activities? Annex A 68 Evaluation Criteria and Questions Efficiency or cost-effectiveness: Efficiency — the extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results. Cost-effectiveness — the extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. 9. Efficiency To what extent is it possible to place a monetary value on the benefits arising from the activities of the program? To what extent has the program or the evaluation conducted impact evaluations of representative program activities? To what extent has the program or the evaluation analyzed the program’s costs in broad categories (such as overhead vs. activity costs), and categorized the program’s activities and associated benefits, even if these cannot be valued in monetary terms? 10. Cost-effectiveness To what extent is the program measuring up against its own business plans: • Has the program cost more or less than planned? How did it measure up against its own costing schedule? • Have there been any obvious cases of inefficiency or wasted resources? To what extent is the program delivering its activities cost-effectively in comparison with alternatives: • How do actual costs compare with benchmarks from similar programs or activities? • Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the objectives and activities of the program? How does the program compare with traditional development assistance programs: • For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions costs compared with traditional development assistance programs? • For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and evaluation)? Governance and management: Governance — the structures, functions, processes, and organizational traditions that have been put in place within the context of a program’s authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. Management — the day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Whereas governance is concerned with “doing the right thing,� management is concerned with “doing things right.� 11. Compliance with generally accepted principles of good governance. To what extent are the governance and management structures and processes well articulated and working well to bring about legitimate and effective governance and management? To what extent do governance and management practices comply with the following seven principles: • Legitimacy — the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program — including shareholders, other stakeholders, implementers, beneficiaries, and the community at large? • Accountability — the extent to which accountability is defined, accepted, and exercised along the chain of command and control within a program, starting with the annual general meeting of the members or parties at the top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in some cases, to the beneficiaries of the program? • Responsibility — the extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? 69 Annex A Evaluation Criteria and Questions • Fairness — the extent to which partners and participants, similarly situated, have equal opportunity to influence the program and to receive benefits from the program? • Transparency — the extent to which a program’s decision making, reporting, and evaluation processes are open and freely available to the general public? • Efficiency — the extent to which the governance and management structures enhance efficiency or cost- effectiveness in the allocation and use of the program’s resources? 12. Partnerships and participation To what extent has the program identified a complete list of stakeholders, or “stakeholder map�, including the agreed- upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine programmatic function, updated regularly, and transparently available? Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is limited to financial and other contributors), or a stakeholder model (in which membership also includes non- contributors)? To what extent, if any, is the program’s legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa? 13. Programs located in host organizations To what extent is the location of the program in the Bank or other partner organization adversely affecting the governance, management, or other aspects of the program, such as compliance with the principles of transparency and fairness? For which functions is the program manager accountable to the host organization and the governing body of the program, respectively? Are conflicts of interest being managed appropriately? To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the program objectively? Resource mobilization and financial management: Resource mobilization — the processes by which resources are solicited by a program and provided by donors and partners. Financial management — the processes that govern the recording and use of funds, including allocation processes, crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In cases where funds accumulate over time, this would also include the management of the cash and investment portfolio. 14. Resource mobilization To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from what sources — the Bank, bilateral donors, foundations, etc.? To what extent has the program succeeded in diversifying its funding beyond a small number of donors? To what extent are the sources of funding for the program (including donor restrictions on the use of resources) affecting, positively or negatively: • The strategic focus of the program? • The outputs and outcomes of the program? • The governance and management of the program? • The sustainability of the program? 15. Financial management Are there any issues that have emerged during the course of the review in relation to: • The quality of financial management and accounting? • The methods, criteria, and processes for allocating funds among different activities of the program? • Financial management during the early stages of the program? Annex A 70 Evaluation Criteria and Questions Sustainability, risk, and strategy for devolution or exit: Sustainability — When applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to which the organization or program is likely to continue its operational activities over time. Devolution or exit strategy — a proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. 16. Sustainability of the benefits of the program’s activities What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these changes actually materialize? 17. Sustainability of the program This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its continuity of effective management, and its ability to withstand changing market or other conditions. To what extent is there still a sufficient convergence or accommodation of interests among the major partners to sustain the program financially? To what extent has the program developed institutional capacity such as performance- based management, personnel policies, learning programs, and knowledge management that help to sustain a program? In what areas could the program improve in order to enhance its sustainability, such as better marketing of the program’s achievements in order to sustain its reputation? 18. Prospects for continuation and strategies for devolution or exit To what extent should the program be sustained? Is the continuation of the program the best way of sustaining the results achieved? Should the design of the program be modified as a result of changed circumstances, either positive or negative? What other alternatives should be considered to sustain the program’s results more cost-effectively, in the light of the previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability: • Reinventing the program with the same governance? • Phasing out the program? • Continuing country or local-level activities with or without devolution of implementation? • Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency on external sources? • “Spinning off� from the host organization? 71 Annex A Annex Table 3. Assessing the Bank’s Performance as a Partner in the Program Evaluation Questions 1. Comparative advantage at the global/regional level. To what extent is the Bank playing up to its comparative advantages at the global/regional level — its global mandate and reach and convening power? To what extent is the Bank’s presence as a partner in the program catalyzing other resources and partners for the program? 2. Comparative advantage at the country level. To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the program? To what extent has the Bank’s country operations established linkages to the GRPP, where appropriate, to enhance the effectiveness of both? 3. Oversight. To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as appropriate, whether the program is housed in the Bank or externally managed? To what extent is the Bank’s oversight independent of the management of the program? To what extent does the Bank’s representative on the governing body have clear terms of reference? 4. Risks and risk management. To what extent have the risks associated with the program been identified and are being effectively managed? For example, IEG identified the following risks in its global review: • Bank bears a disproportionate share of responsibility for governing and managing in-house programs? • Confusion at the country level between global program activities, Bank activities, and Borrower activities? • Representation of NGOs and the commercial private sector on program governing bodies? • Unclear role and application of Bank’s safeguards? • Trust-funded consultants and seconded staff representing the Bank on some program governing bodies? 5. Disengagement strategy. To what extent is the Bank engaged at the appropriate level in relation to the Bank’s new strategic framework: • Watching brief? • Research and knowledge exchange? • Policy or advocacy network? • Operational platform? To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in relation to the Bank’s objectives for its involvement in the program: • The program declares “mission accomplished� and closes? • The program continues and the Bank withdraws from all aspects of its participation? • The program continues and the Bank remains engaged, but the degree of the Bank’s engagement in some or all aspects (such as financing) declines over time? Annex A 72 Annex Table 4. Common GRPP Activities Policy and knowledge networking 1. Facilitating communication This includes providing a central point of contact and communication among among practitioners in the sector practitioners who are working the sector or area of development to facilitate the sharing of analytical results. It might also include the financing of case studies and comparative studies. 2. Generating and This comprises two related activities. The first is gathering, analyzing and disseminating disseminating information and information, for example, on the evolving HIV/AIDS epidemic and responses to it, knowledge including epidemiological data collection and analysis, needs assessment, resource flows, and country readiness. The second is the systematic assembling and dissemination of knowledge (not merely information) with respect to best practices in a sector on a global/regional basis. 3. Improving donor This should be an active process, not just the side effect of other program activities. coordination This may involve resolving difficult interagency issues in order to improve alignment and efficiency in delivering development assistance. 4. Advocacy This comprises proactive interaction with policymakers and decision makers concerning approaches to development in a sector, commonly in the context of global, regional, or country-level forums. This is intended to create reform conditions in developing countries, as distinct from physical and institutional investments in public goods, and is more proactive than generating and disseminating information and knowledge. 5. Implementing conventions, Rules are generally formal. Standards can be formal or informal, and binding or nonbinding, but rules, or formal and informal implementing standards involves more than simply advocating an approach to development in standards and norms a sector. In general, there should be some costs associated with noncompliance. Costs can come in many forms, including exposure to financial contagion, bad financial ratings by the IMF and other rating agencies, with consequent impacts on access to private finance; lack of access to OECD markets for failing to meet food safety standards, or even the consequences of failing to be seen as progressive in international circles. Financing technical assistance 6. Supporting national-level This is more directed to specific tasks than advocacy. This represents concrete policy, institutional, and technical involvement in specific and ongoing policy, institutional, and technical reform processes reforms in a sector, from deciding on a reform strategy to implementation of new policies and regulations in a sector. It is more than just conducting studies unless the studies are strategic in nature and specific to the reform issue in question. 7. Capacity strengthening and This refers to strengthening the capacity of human resources through proactive training training (in courses or on-the-job), as well as collaborative work with the active involvement of developing country partners. 8. Catalyzing public or private This includes improving regulatory frameworks for private investment and implementing investments in the sector pilot investments projects. Financing investments 9. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver national loans and credits (more than the financing of studies), the benefits of which accrue public goods primarily at the national level. 10. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies) to deliver public goods such as global/regional public goods conserving biodiversity of global significance and reducing emissions of ozone- depleting substances and carbon dioxide, the benefits of which accrue globally. 11. Financing global/ regional This refers to financing research and development for new products and technologies. investments to deliver These are generally physical products or processes — the hardware as opposed to the global/regional public goods software of development. 73 Annex B Annex B. Program Goals, Objectives, and Activities MAPS PARIS21 TFSCB Programs’ goals MAPS is one important element of an informal PARIS21 was founded in November 1999 with the The TFSCB was set up by the World Bank in 1999 partnership whose main goal is to improve goal of developing a culture of evidence-based as a multi-donor trust fund that aims to improve development statistics agreed at the 2nd policy making and more specifically of reducing the capacity of developing countries to compile International Roundtable for Managing for poverty and achieving the MDGs. and use statistics with the overall objective of Development Results (2004). supporting the management of development MAPS set two key target dates: 2010 by which results. From the beginning the TFSCB has been countries should have prepared strategies for the closely aligned with PARIS21 with whom it shares development of their statistical systems and the overall goal of promoting a culture of should have improved the availability of key evidence-based decision making and more indicators, and 2015 when capacity should be in specifically, of monitoring progress towards the place in order to monitor progress towards the MDGs and poverty reduction strategies. Since MDGs. 2004, TFSCB has been an important instrument for implementing the Marrakech Action Plan for Statistics (MAPS). Programs’ To help develop and strengthen national statistical To encourage and support developing countries to To support the preparation of NSDSs, in line with objectives systems through 6 specific actions: design, implement, and monitor National MAPS and working with PARIS21 to ensure that − Prepare national strategies for development of Strategies for the Development of Statistics all LICs have an integrated and comprehensive statistics for all LICs by 2006; (NSDSs) including: (a) mobilizing resources for plan for the strategic development of their national the implementation of NSDSs; (b) coordinating statistical systems and detailed capacity building − Ensure full participation of developing donor support to statistics; (c) coordinating all programs for statistics. countries in the 2010 census round; and actors within the National Statistical System; − Increase financing for SCB and international (d) producing guidance and documentation; and coordination: (e) providing technical assistance (legislation, − Set up an International Household Survey training, human resources, etc.). Network; − Undertake urgent improvements to monitor the MDGs and other development goals; and Increase accountability of the international statistical system. Annex B 74 MAPS PARIS21 TFSCB How are the Current strategy is to maintain DGF funding for − NSDS methodology: Helping countries Following the launch of MAPS, the TFSCB has activities the major programs until 2013 with a financial exit design, seek funding for, and implement been used increasingly for laying the groundwork organized into strategy that has been discussed and approved National Strategies for the Development of for SCB in developing countries by designing a strategies? with the MAPS Advisory Board and the (WB Statistics. national strategy for the development of statistics internal) Statistical Capacity Building Committee. − Advocacy: Helping statisticians in developing (NSDS). Two financing windows have been Beyond 2013, the Bank is expected to continue to countries with their own advocacy work and to introduced: (a) an NSDS window and (b) a more be engaged in MAPS through a new funding demonstrate the advantages of planning general SCB window for various development facility, the Statistics for Results Facility (SRF) advocacy systematically. projects. established in 2009. It is also expected that the PARIS21 will gain access to fund raising − Donor collaboration: Supporting the expertise, including from the wider partnership... coordination of statistical capacity development worldwide through the production of the annual The novel feature of MAPS has been to assemble Partner Report on Support to Statistics an informal partnership of international agencies (PRESS). with relevant experience that, with the aid of grant finance of limited scale, can develop programs − Support for surveys: Providing guidance and specifically focused on helping poorer developing materials related to all stages of survey countries achieve stronger capacities in the above implementation. areas. − Support for quality of survey data: Helping countries improve their survey programs and make best use of available data. 75 Annex B MAPS PARIS21 TFSCB Programs' Under MAPS, the World Bank’s Development − PARIS21 focuses its efforts on encouraging TFSCB provides small grants of up to $400,000 activities Grant Facility has provided financial support for all developing countries (Africa, other over a period of two or three years to low-income the following programs of activities associated with IDA borrowers, and LMICs) to design and countries and to appropriate regional or PARIS21: international organizations to implement specific implement NSDSs and to have nationally − General support to PARIS21: Mainstreaming capacity building projects. owned and produced data for all MDG strategy planning for improving statistical indicators. As of May 2010, TFSCB has funded NSDS systems in developing countries; projects in 61 countries (of which 36 are IDA − These efforts are under-pinned by a detailed − International Household Survey Network countries) and 3 regional projects. The trend of the NSDS knowledge base including (IHSN): Introducing more efficient approaches overall demand for TFSCB has been on the rise documentation to carry out the processes. to the conduct of household surveys in − Advocacy papers encourage broader developing countries; and recognition by national and international policy − Accelerated Data Program (ADP): Helping makers of the role of statistics in development developing countries make best use of and poverty reduction. available survey data by introducing improved − The PARIS21 partnership provides a forum for practices of data collection, management and dialogue on supporting statistical development dissemination. and shares tools to make this co-ordination a The DGF has also provided financial support to reality. the UN Statistics Division (UNSD) to enable it to do much more than normal for IDA countries, especially in terms of manuals and technical support on methodological standards and good practices in preparing for the 2010 Census Round. Annex B 76 MAPS PARIS21 TFSCB Expected − More countries have either completed an − Production of good practices and guidelines for As of May 2010, 74 of the 79 least developed outputs from the NSDS or are designing strategies for improving the preparation, costing and implementation of countries classified as IDA-eligible countries are in activities statistical systems (PARIS21 and TFSCB). NSDSs. the process of implementing, designing or − More LICs have conducted a census or − Support for regional programs: in support of awaiting adoption of a NSDS. While TFSCB’s planning to do so (UN Statistics Division). NSDSs. initial focus was on SCB activities, following the adoption of MAPS in 2004, more emphasis was − More resources are mobilized for investment in − Production of advocacy toolkit and booklets on given on helping developing countries formulate SCB. the need for better statistics and organization their NSDS. − Improvement on the availability of indicators for of advocacy forums. 81 countries of which 46 are IDA-eligible, have measuring the MDGs. − Publication of the yearly “Partner Report on received grants from the TFSCB with Sub- − Adoption of good practice principles in Support to Statistics� (PRESS). Saharan Africa as the largest recipient of the statistical work. − Development of survey tools and toolkits TFSCB grants: (IHSN). − Support for the documentation, preservation and dissemination of survey data (ADP). How are these − All low-income countries to have NSDSs. − Progress has been notable in the design and − TFSCB has made it possible for most LICs to outputs − The participation of developing countries in the adoption of an NSDS among low-income have an NSDS. supposed to 2010 Census Round to be improved. countries. However, more needs to be done to − TFSCB has acted as a catalyst for other funds contribute to help countries implement their national to support large-scale operations for statistical program − Financial support for SCB to rise. strategies. development. outcomes? − The IHSN to function well. − Donor partners have improved coordination of − TFSCB has provided timely support to the International statistical systems to be their support to statistical capacity organization of regional thematic training strengthened. development. seminars and international meetings on − Capacities to produce, analyze and use emerging issues. statistics in countries have improved. 77 Annex B PARIS21 2007–2010 Logical Framework (revised June 2008) Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks Partnership Goal: – Develop a culture of evidence-based – Demonstrable increase in the use of – UNSG’s annual report on progress to (Goal to Super-Goal) policy making and implementation which policy relevant statistics to manage for UN General Assembly – Better policies based on the better use serves to improve governance and development results and aid effectiveness – Reports on follow-up to Roundtables on of better statistics contribute to achieving government effectiveness in reducing (measured every 3–5 years against a Managing for Development Results and the objectives of the national poverty and achieving the MDGs. 2006/07 baseline) Paris Declaration on Aid Effectiveness, development policy and the MDGs including progress towards indicator/ target no. 11 to “Reduce the proportion of countries without transparent and monitorable performance assessment frameworks by one-third� Partnership Purpose: – Statistical information is used by – Increased use and analysis of statistics – Analysis of national policy documents – The availability and use of improved PARIS21 partners to inform development in policy documents (measured every 3–5 such as PRSPs and MDG Country statistics leads to better economic and policy decisions and to manage their years, when documents can be compared Reports compared with their 2006/07 social development policies and better implementation with their 2006/07 equivalents) equivalents management for results – Analysis of donor country assistance strategies compared with their 2006/07 equivalents Partnership Outcomes and Outputs: National and international statistical programmes of each PARIS21 Partner: 1. Are centered on designing and – Increase in number of countries who – Annual PARIS21 progress report will – Increased investment leads to better implementing NSDSs have (a) designed; and (b) are show trend of NSDS design and statistics that meet the needs of policy implementing NSDSs implementation makers and analysts – Peer reviews and NSDS Check List will – Statistics integrated into national provide a check on quality development policy processes; and funded through national budget 2. Are well coordinated across NSSs and – Percentage of technical cooperation – Development partner reporting system frameworks, incorporating donor support between governments and donors flows that are implemented through (PRESS) coordinated programmes consistent with – Development assistance programmes – Annual PARIS21 progress report for based on sound national data for greater national development strategies other aspects of coordination Annex B 78 Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks 3. Build statistical capacity – Improved capacities to produce, – Statistical capacity indicators (World aid effectiveness and efficiency analyze and use statistics in countries Bank) compared with 2006/07 – Annual PARIS21 progress reports 4. Provide better statistics for immediate – Strengthened national data available on – Inter-Agency and Expert Group on MDG priority needs the MDGs by 2010 indicators (e.g. DFID has a target for internal purposes to “Increase the number of countries by 4% each year, with at least 2 data points, (excluding modeled data by agency) for seven of the MDG indicators�) Secretariat Activities and Outputs: Regional Programmes: Facilitate successful implementation of NSDSs in low-income countries in Africa, the Arab States, Asia and the Pacific, and Latin America; incorporating NSDS country reporting.  Facilitate successful country events  Number and quality of events held  Feedback forms from participants – Recommendations and follow-up lead to and regional workshops; and and joint partner missions to countries strengthened and better co-ordinate strengthen regional and country statistical programmes. partnership  Organize successful peer review  4 per annum successful peer reviews  Feedback from host countries and organized reviewers  Comprehensive reports on country  Comprehensive annual report on  Annual report on progress of progress with NSDS and statistical NSDS global and country progress partnership development Advocacy: Facilitate effective design and implementation of advocacy tools and messages to reach out to the various stakeholder groups.  Develop and deliver PARIS21  Advocacy messages and delivery  Number of leaflets printed and – Advocacy messages are heard and advocacy strategy, messages, mechanisms/ tools developed in line distributed; feedback from countries is acted upon materials, interviews, and events with PARIS21 advocacy strategy positive adapted to audiences  Roll out effective country advocacy  Country Advocacy Toolkit developed  Number of countries assisted; positive tool kits to help countries develop and and 10 countries helped to design, feedback on advocacy impact from implement their own advocacy implement and monitor impact of countries strategies advocacy strategies 79 Annex B Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks Partnership: Engage an expanded partnership in NSDSs at country level including through activities at regional and international level; incorporating development partner reporting.  Expand engagement by the wider  PARIS21 Web site developed and  5 to 10% increase in hits on PARIS21 – Partnership activities lead to PARIS21 partnership information notes distributed; Web site achieved and maintained strengthened and better co-ordinate comprehensive, up-to-date calendar partnership programmes. of events  Hold effective Consortium meeting  80 participants from developing  Meeting reports and evaluation sheets held involving broad representation countries participate, balanced by  High quality background report OECD countries, regional and produced for Consortium meeting international organizations, and research/ analysis community endorse way forward for partnership  PARIS21 members and agenda  Participation of developing country  Numbers, representation and represented in partner events managers at international events engagement of participants at PARIS21 and international events  Agreed donor reporting system that is  Report on donor support to statistical  Coverage of partner reporting system regularly updated development Studies and Knowledge Development: Effective knowledge base on statistical development and strategic planning issues supports NSDS implementation.  Develop the PARIS21 knowledge  20 to 40 new materials loaded into  Number of new materials and studies; base in response to identified needs database how many distributed  Produce studies relating to NSDS  5 new studies produced and  Feedback from reviewers processes, national statistical disseminated  Steering Committee feedback on development, and financial and  Effective task team outputs and quality and relevance technical support meeting reports  Task Teams provide a forum for  Work mainstreamed into Secretariat discussing specific issues relating to work programme statistical development and harmonizing SCB activities, e.g. on training and sub-national statistics Annex B 80 MAPS Results Framework Action/objective Supported by which key partners Expected outcome Indicators of progress 1. Mainstream strategic planning for PARIS21, Trust Fund For Statistical Support for all low income 2009:77% of low-income countries with an NSDS or improving statistical systems in Capacity Building Work Bank And countries (where practical) to have working on one (source: PARIS21) developing countries Trust Fund donors, African a National Strategy for the Development Bank Development of Statistics 2. Prepare for the 2010 Census Round UN Statistics Division, UN Fund for Improvement in participation of New census guidelines prepared and disseminated: 13 (improve participation) Population Activities countries in 2010 census round of the 21 non-participants in the 2000 round participating in the 2010 round (source: UNSD) 3. Increase investment in statistical Bilateral donors, multilateral Finance for statistics increased by Estimated disbursements globally: systems development banks additional $120 million per year 2006: $185 million globally by 2010 2007: $338 million 2008: $246 million 2009: $209 million (source: PARIS21/PRESS) 4. Setup and support the operations of PARIS21, World Bank, network IHSN functioning: 12 pilot countries IHS is operational; around 3800 survey catalogued; an International Household Survey members in documentation phase of progress made in the development of new standards Network (IHSN) Accelerated Data Program (ADP) by and guidelines; 50 countries are now participating in 2009 the ADP (sources: PARIS21/World Bank) 5. Improve data for measuring key UN Statistics Division, UN agencies, No overall outcome target defined Average score of data availability dimension of topics and the MDGs World Bank by MAPS statistical capacity building indicator for 111 developing countries from 72 in 2004 to 77 in 2009 (source: World Bank) 6. Increase accountability of the UN agencies (i.e. members of UN Strengthened the national statistical Key UN agency supported by DGF grants (UIS, UNSD, international statistical system Coordination Committee for Statistical system to meet with the demand for UNECE, and UN habitat). Activities) better International statistics, and to UN principles international statistical system agreed support the needs of developing countries 81 Annex B PARIS21 2007–2010 Logical Framework (revised June 2008) Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks Partnership Goal: - Develop a culture of evidence-based - Demonstrable increase in the use of - UNSG’s annual report on progress to UN (Goal to Super-Goal) policy making and implementation which policy relevant statistics to manage for General Assembly serves to improve governance and development results and aid effectiveness - Reports on follow-up to Roundtables on - Better policies based on the better government effectiveness in reducing (measured Managing for Development Results and Paris use of better statistics contribute to poverty and achieving the Millennium every 3-5 years against a 2006/07 Declaration on Aid Effectiveness, including achieving the objectives of the Development Goals (MDGs). baseline) progress towards indicator/ target no. 11 to national development policy and the “Reduce the proportion of countries without MDGs transparent and monitorable performance assessment frameworks by one-third� Partnership Purpose: - Statistical information is used by - Increased use and analysis of statistics in - Analysis of national policy documents such as - The availability and use of PARIS21 partners to inform development policy documents (measured every 3-5 Poverty Reduction Strategies/ Papers improved statistics leads to better policy decisions and to manage their years, when documents can be compared (PRS/Ps) and MDG Country Reports compared economic and social development implementation with their 2006/07 equivalents) with their 2006/07 equivalents policies and better management for - Analysis of donor country assistance results strategies compared with their 2006/07 equivalents Partnership Outcomes and Outputs: National and international statistical programmes of each PARIS21 Partner: 1. Are centered on designing and - Increase in number of countries who have - Annual PARIS21 progress report will show - Increased investment leads to implementing NSDSs (a) designed; and (b) are implementing trend of NSDS design and implementation better statistics that meet the needs NSDSs - Peer reviews and NSDS Check List will of policy makers and analysts provide a check on quality - Statistics integrated into national development policy processes; and funded through national budget 2. Are well coordinated across NSSs and - Percentage of technical cooperation flows - Development partner reporting system frameworks, incorporating donor between governments and donors that are implemented through coordinated (PRESS) support programmes consistent with national - Annual PARIS21 progress report for other - Development assistance development strategies aspects of coordination programmes based on sound Annex B 82 Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks 3. Build statistical capacity - Improved capacities to produce, analyze - Statistical capacity indicators (World Bank) national data for greater aid and use statistics in countries compared with 2006/07 effectiveness and efficiency - Annual PARIS21 progress reports 4. Provide better statistics for immediate - Strengthened national data available on - Inter-Agency and Expert Group on MDG priority needs the MDGs by 2010 indicators (e.g. DFID has a target for internal purposes to “Increase the number of countries by 4% each year, with at least 2 data points, (excluding modeled data by agency) for seven of the MDG indicators�) Secretariat Activities and Outputs: Regional Programmes: Facilitate successful implementation of NSDSs in low-income countries in Africa, the Arab States, Asia and the Pacific, and Latin America; incorporating NSDS country reporting.  Facilitate successful country events  Number and quality of events held and  Feed-back forms from participants - Recommendations and follow-up and regional workshops; and joint partner missions to countries lead to strengthened and better co- strengthen regional and country ordinated statistical programmes. partnership  Organize successful peer reviews  4 per annum successful peer reviews  Feedback from host countries and organized reviewers  Comprehensive reports on country  Comprehensive annual report on  Annual report on progress of partnership progress with NSDS and statistical NSDS global and country progress development Advocacy: Facilitate effective design and implementation of advocacy tools and messages to reach out to the various stakeholder groups.  Develop and deliver PARIS21  Advocacy messages and delivery  Number of leaflets printed and distributed; - Advocacy messages are heard and advocacy strategy, messages, mechanisms/ tools developed in line feedback from countries is positive acted upon materials, interviews, and events with PARIS21 advocacy strategy adapted to audiences  Roll out effective country advocacy  Country Advocacy Toolkit developed  Number of countries assisted; positive tool kits to help countries develop and 10 countries helped to design, feedback on advocacy impact from and implement their own advocacy implement and monitor impact of countries strategies advocacy strategies 83 Annex B Narrative Summary Objectively Verifiable Indicators Means of Verification Assumptions/ Risks Partnership: Engage an expanded partnership in NSDSs at country level including through activities at regional and international level; incorporating development partner reporting.  Expand engagement by the wider  PARIS21 Web site developed and  5 to 10 % increase in hits on PARIS21 Web - Partnership activities lead to PARIS21 partnership information notes distributed; site achieved and maintained strengthened and better co- comprehensive, up-to-date calendar of  Meeting reports and evaluation sheets ordinated partnership programmes. events  Hold effective Consortium meeting  80 participants from developing  High quality background report produced held involving broad representation countries participate, balanced by for Consortium meeting OECD countries, regional and international organisations, and research/ analysis community endorse way forward for partnership  PARIS21 members and agenda  Participation of developing country  Numbers, representation and engagement represented in partner events managers at international events of participants at PARIS21 and  Agreed donor reporting system that  Report on donor support to statistical international events is regularly updated development  Coverage of partner reporting system Studies and Knowledge Development: Effective knowledge base on statistical development and strategic planning issues supports NSDS implementation.  Develop the PARIS21 knowledge  20 to 40 new materials loaded into  Number of new materials and studies; how base in response to identified needs database many distributed  Produce studies relating to NSDS  5 new studies produced and  Feedback from reviewers processes, national statistical disseminated  Steering Committee feedback on quality development, and financial and  Effective task team outputs and and relevance technical support meeting reports  Task Teams provide a forum for  Work mainstreamed into Secretariat discussing specific issues relating to work programme statistical development and harmonizing statistical capacity building activities, e.g. on training and sub-national statistics Annex B 84 Logical Framework of the Trust Fund for Statistical Capacity Building (Developed by OPM through their evaluation of the Trust Fund in 2003) Narrative Summary Objectively verifiable Indicators Means of Verification Assumptions Goal Improved evidence based policy making Improved poverty analysis and monitoring PRS reviews and and monitoring for poverty reduction in frameworks in PRSs and other monitoring reports. participating countries. government strategy/policy documents. Purpose: Better statistical information available to Timely and relevant statistical reports and Statistical publications and reports. National policy makers use better at least 30 developing countries to plan, analyses published. Reports and publications of users, information to develop and implement pro- implement and monitor their own social Statistical outputs accepted by including PRSP reviews, monitoring poor policies. and economic development. Government and civil society and widely reports, and annual budget documents. National Governments provide used to assess development needs and acceptable level of resources to sustain monitor progress. improvements. Outputs: Easily accessible and responsive Statistical capacity of at least 30 Project reports. WB trust fund activities correctly funding mechanisms to allow countries participating countries increased within 5 WB Country Assistance Strategies. address constraints in statistical capacity. access to primarily catalytic funding for years. Other donor aid frameworks. Countries are aware of and make use of projects related to SCB. Follow up support and assistance the Trust Fund. Trust Fund progress reports. Projects which contribute, directly or projects assimilated into WB / other donor IMF ROSC Data Modules. indirectly, to the development of SCB country strategies with appropriate implemented in at least 10 countries per funding. GDDS metadata. year. Number of projects approved without direct WB involvement in the implementation, and speed of disbursement of funds. Activities Inputs: Management of the Trust Fund by the Total fund: $4 million per year over lifetime Donors are willing to make regular Bank, and implementation of some of the of project. Through small grants (of up to contributions. projects if necessary. $400,00 over two or three years) two WB has sufficient capacity to provide Trust Fund will fund some or all of these kinds of major projects are produced: appropriate technical assistance as activities as needed in country: a) preparation and approval of 10 required. Preparations to assess further sources of statistical development strategies and Sufficient number of expert consultants funding for large SCB activities including master plans in 2004, 15 in 2005 and available within the development STATCAP. b) 5–10 projects per year providing community. Institutional strengthening, including the support to capacity building activities that 85 Annex B Narrative Summary Objectively verifiable Indicators Means of Verification Assumptions improvement of organization, make more effective use of existing data management, human resource and data processes. management and strategic planning. Particular links will be made with other Fostering knowledge of international initiatives, including the IMF’s General standards and methodologies. Data Dissemination System (GDDS). Improvement of statistical outputs and improvement of the dissemination of statistics and statistical analysis. Awareness-raising and other efforts to increase demand within countries. Annex C 86 Annex C. External Evaluation: Major Findings and Recommendations and the Program Response MAPS Recommendations Activities following up on recommendations The main programs supported by the DGF The MAPS exit strategy was revised. Funding should be supported at current funding levels to levels for ADP and IHSN were kept through at least 2012. FY12, while reduced for other programs. The revised exit strategy aligns with the MDG deadline of 2015. The wider MAPS Partnership should be engaged The Statistics for Results Facility was established to accelerate emergence of country success with the aim of scaling up efforts to improve cases. statistical capacity on country level. The role of the Advisory Board in providing strategic advice and direction should be activated. The World Bank internal Statistical Capacity The composition of the Statistical Capacity Building Committee should have a wider Building Committee changed to get wider coordinating function. representation of regions and networks, and the TOR expanded so as to include more of a coordinating function. IDA Country Assistance Strategies need to give A guidance note for country teams preparing a fuller attention to statistical capacity and the use CAS/CPS on how to approach the assessment of of statistics in decision making. national statistical capacity and how to address shortages was developed. Full and effective implementation of NSDSs is a Agree. major challenge in many countries and must remain the top priority for MAPS. The MAPS Partnership should promote the training in data use and interpretation among key government staff. Improvements should be made to the The MAPS continued work with partner agencies specification of development objectives and to strengthen selection of and follow up of means of verification in DGF-supported verifiable development objectives for programs programs. supported. 87 Annex C PARIS21 Recommendations Program Response Broad recommendations PARIS21 should be continued. Secretariat should continue working at the The only technical role the Secretariat plays at country level…not to be a technical assistance country level is in furnishing the advice of role but a continuing facilitation role. consultants, usually retired statisticians. Otherwise the Secretariat’s role is strictly one of facilitation. Secretariat should re-direct its efforts to focus With the new PARIS21 strategy agreed in 2009, more on policy makers. the Secretariat has already scaled up its efforts to reach out to policy makers; however, it takes note of the recommendation to continue these efforts. Efforts should be expanded to support NSOs in Supporting countries in the coordination of their this effort (to coordinate donor partners) and to donor community is systematically undertaken in encourage donor partners at the country level. PARIS21’s country missions. Country coverage should not be expanded The issue of country coverage ought to be based beyond its current coverage and should be on expressed country need, decisions of the reduced over time as current commitments in PARIS21 Board, and the financial and technical countries are completed. resources available to the Secretariat. Furthermore, commitments to countries ought not to be considered “completed� in light of the acknowledged need to assist countries with NSDS implementation, an ongoing and iterative process. Specific recommendations Secretariat should prepare a manual on how to The NSDS guidelines are being updated in promote coordination of donor partners. 2011–12 and will include guidance on donor coordination. Secretariat should advise countries on a process The NSDS guidelines are being updated in to be able to identify and build champions for the 2011–12 and will include guidance on identifying use of statistics in policy. and building champions. Secretariat should scale up the sharing of Peer reviews will continue to be conducted in experiences across countries. Africa and the Secretariat intends to extend this mechanism to other regions. Secretariat should promote the user/producer consultation which is frequently not fully operational, including advice on how to get the most out of user-producer workshops. The Partnership should improve the effectiveness The composition of the PARIS21 Board is of the Board by having more high level currently under discussion with the Executive development and policy managers from donor Committee. institutions involved. Help to build up skills in policy to enable more focus on policy-related aspects of statistics. Annex C 88 TFSCB Recommendations Program Response Noting especially that there is still a great need to No action required. extend grants for statistical capacity building in developing countries, recommend that donor funding continue to be mobilized for the operations of the Trust Fund. Recommend that the Consultative Group address The IMC and the Administration Unit will pay the issue of financing the implementation of closer attention to implementation components of NSDSs, with the aim of securing the gains NSDS projects when assessing future proposals. achieved through the formulation of national However, there is no plan to open a new window. strategies. Consider a special window for implementation grants. Recommend that in future a minimum of 60% of Well over 60% of projects will be to assist the the TFSCB’s resources should be allocated to NSDS process. In principle, all approved projects the NSDS process. Within this process a strong for countries with NSDS’s are therefore to emphasis should be placed on NSDS support the implementation of the NSDS. implementation. Increase the amount of the grants for inflation It has been decided to increase the size from adjustment. $400 to 500 thousand. Amend the TFSCB Guidelines to encourage TFSCB promotes recipient execution of projects. TTLs and client agencies to prepare projects in However, to circumvent cumbersome Bank such a way that individual expert services rather procurement requirements, the TFAU advises than firms are used and to allow more use of that relatively small projects be Bank executed. TFSCB funds to hire local administrative staff. The TFAU should expand its guidance to Task There is no plan for the moment to make Team Leaders from the approval procedures to changes to how reporting is done. measures that will promote uniform and timely results-oriented progress reports. In the absence of substantive completion or There is no plan for the moment to make evaluation reports, it is recommended that a short changes to how reporting is done. terminal report be introduced highlighting tangible results and major lessons learned. Some form of evaluation could help to highlight DECDG is undertaking an evaluation of sample lessons learned for the benefit of future projects. of NSDS’s as to provide feedback and guidance for the future projects. TFSCB management should continue to An agreement with the UN has been reached to negotiate with the concerned UN agencies with a reduce the maximum UN administrative fee rate view to facilitating their execution of TFSCB down to 7%. projects while minimizing the total amount of overhead involved. In order to enhance the TFSCB’s visibility and A new brochure, incorporating these increase awareness among its potential clientele, recommendations, will be produced in Fall 2010. its management should be more proactive in disseminating information about its key features. Incomplete posting of CG reports in the Web site. The progress report will be produced once a year Recommend that these matters be scrutinized but with stronger emphasis on achievements with a view to improving the transparency of the made through funded projects. TFSCB’s governance arrangements. 89 Annex D Annex D. Members of the Governing Bodies MAPS – Advisory Board Shaida Badiee (Chair) Grace Bediako Director Government Statistician Development Data Group Ghana Statistical Service World Bank Mohamed El-Heyba Lemrabott Berrou Adelheid Bürgi-Schmelz (Abadila Berrou) Director Manager Statistics Department PARIS21 Secretariat International Monetary Fund OECD Paul Cheung Jean-François Divay Director Ministry of Foriegn Affairs United Nations Statistics Division France Heidi Grainger Charles Lufumpa Head Director Department for International Development Statistics Department African Development Bank Eduardo Pereira Nunes Pronab Sen President Chief Statistician of India Institute of Geography and Statistics Ministry of Statistics and Program Brazil Implementation India Annex D 90 World Bank Statistical Capacity Building Committee Kyle Peters, Director, OPCCS (Chair) Ayse Akin-Karasapan, Director, OPCDR Yvonne M. Tsikata, Sector Director, ECSPE Shanta Devarajan, Chief Economist, AFRCE Antoine Simonpietri, Senior Statistician, AFTRL Marianne Fay, Chief Economist, SDNVP Louise J. Cord, Sector Manager, LCSPP Jamie Saavedra, Sector Manager, PRMPR Ariel Fiszbein, Chief Economist, HDNCE Harry Patrinos, Lead Education Specialist, HDNED Shaida Badiee, Director, DECDG Grant Cameron, Manager, DECDG 91 Annex D PARIS21 Steering Committee (to be updated by new PARIS21 Board) Co-Chairs - Mr. Brian ATWOOD, OECD/DAC Chairperson (Co-chair) - Mrs. Samia ZEKARIA GUTU, Director, Central Statistics Agency, Ethiopia Partner Countries - Africa – Group A : Mr. Sekouba DIARRA, Coordonateur Cellule Technique CSLP, Mali. - Africa – Group B: Ms. Alfeine S. Soifiat TADJIDDINE, Commissaire Générale au Plan, Comoros. - Africa – Group C: Mr. Joseph TEDOU, Directeur Général, Institut national de la statistique, Cameroon. - Africa – Group D: Mr. Mohamed TAAMOUTI, Director, Direction de la Statistique, Morocco. - Africa – Group E: Mr. Pali Jobo LEHOHLA, Statistician General, South Africa. - Africa Group F: TBD. - Middle East: Dr. Maral TUTELIAN-GUIDANIAN, Directrice générale, Lebanon. - East Asia & Pacific: Mr. Hing Wang FUNG, Commissioner for Census and Statistics, Hong Kong SAR, China. - South Asia: Mr. Abdul Rahman GHAFOORI, President Central Statistics Organization, Afghanistan. - South East Asia: Dr. Romulo VIROLA, Secretary General, Philippines. - Central America and the Caribbean: Mr. Jaime Vaglio MUNOZ, Director, Instituto Nacional de Estadisticas y Censos, Costa Rica. - South America: TBD. - Eastern Europe and CIS: Mr. Zaza CHELIDZE, Executive Director, National Statistics office, Georgia. Regional Banks and Institutions - African Development Bank - Asian Development Bank - Inter American Development Bank - Islamic Development Bank - AFRISTAT - ASEAN - CARICOM - Secretariat of the Pacific Community Bilateral Donors - Australia - Canada - Belgium - Finland - France - Germany - Italy Annex D 92 - Korea - Netherlands - Norway - Spain - United Kingdom Multilaterals - EUROSTAT - European Commission - FAO - International Monetary Fund - UNDP - UN Statistics Division - World Bank - UNESCO - WHO-Health Metrics Network Foundations / Private Sector - Gates Foundation 93 Annex D TFSCB - Consultative Group Shaida Badiee World Bank (Co-Chair of the CG) Samia Zekaria Gutu Central Statistics Agency, Ethiopia (Co-Chair of the CG) Misha Belkindas World Bank (Chair of the IMC) Jean-Louis Bodin Advisory Panel Brian Atwood Chair, OECD/DAC, OECD Raul Suarez de Miguel Switzerland Antony Smith UK Frances Harper UK Jean-Francois Divay France Paul Cheung United Nations Statistics Division Rob Swinkels Netherlands Marit Strand Norway Abadila Berrou PARIS21 Secretariat Annex E 94 Annex E. Sources and Uses of Funds MAPS Programs and Grants Approved (in $ millions) Supported Supported Program FY06 FY07 FY08 FY09 FY10 Agency ($m) ($m) ($m) ($m) ($m) PARIS21 1. Support for NSDS processes 1.50 1.30 1.30 1.00 0.70 PARIS21 2. Accelerated Data Program 2.00 2.00 2.00 2.20 2.20 PARIS21 3. International Household Survey Network 0.90 0.75 0.75 1.00 1.00 UN Statistics 4. Census 2010 1.00 1.00 1.00 1.00 1.00 Division UN Statistics 5. Gender statistics 0.18 0.20 0.10 0.20 0.15 Division HABITAT 6. Urban indicators 0.20 0.25 0.20 - - UNECE 7. Gender statistics (Central and South East - 0.10 0.20 0.15 0.25 Europe) 8. Interstate Statistical Committee of the - - - - 0.25 Commonwealth of Independent States MAPS Unit 9. Advisory Board, monitoring and evaluation 0.12 0.10 0.15 0.15 0.15 Total MAPS 5.90 5.70 5.70 5.70 5.70 Source: World Bank, Development Grant Facility reports (various). 95 Annex E PARIS21 Core Program Contributions (in Euros) Country/ 1999–2007 2008 2009 2010 2011 2012 2013 Institution Austria 200,000 Belgium 1,000,000 200,000 Canada 966,370 254,300 EFTA 80,000 European 16,226 121,185 150,000 Commission Finland 200,000 100,000 France 1,424,000 45,000 Greece 0 200,000 Ireland 762,499 100,000 Italy 0 200,000 Japan 68,602 Netherlands 1,050,000 Norway 579,262 198,879 Spain 0 250,000 250,000 250,000 250,000 Sweden 660,519 Switzerland 597,695 99,445 United Kingdom 3,904,596 833,155 555,436 555,436 277,718 World Bank 3,956,992 780,470 481,729 Other income 25,050 Total Income 15,491,812 1,651,655 1,579,353 1,283,155 805,436 555,436 277,718 Source: PARIS21, Progress Report for the Partnership in Statistics for Development in the 21st Century (Reporting Period: November 2009–May 2010) PARIS21 ADP-IHSN Program (thousands of Euros) 2006 2007 2008 2009 2010 Total 2006–10 World 2,236 3,444 2,446 2,202 – 10,328 Bank Source: PARIS21, Progress Report for the Partnership in Statistics for Development in the 21st Century (Reporting Period: November 2009–May 2010) PARIS21: Expenditures (thousands of Euros) 2010 Expenditure categories 2006 2007 2008 2009 (estimate) Core Programs 1,930 2,068 2,579 3,550 3,440 Satellite programs (ADP- IHSN) 499 2,291 2,355 2,139 3,044 Source: PARIS21 Progress Report (June 2010). Annex E 96 TFSCB Financing ($ millions) Financial Items $ million TFSCB I [1999–2005] A. Donor contributions including investment income 13.57 B. T otal allocation 11.01 Allocation to projects 9.81 TFSCB oversight, proposal review, reporting and monitoring (9%) 1.20 C. Unallocated funds (C = A - B) 2.55 TFSCB II [2004–2010]] D. Donor contributions including investment income 13.26 E. Total allocation 10.40 Allocation to projects 9.81 TFSCB oversight, proposal review, reporting and monitoring (5%) 0.59 F. Unallocated funds (F = D - E) 2.86 TFSCB III [2007–2012] G. Donor contributions including investment income 19.26 H. T otal allocation 18.50 Allocation to projects 15.52 TFSCB oversight, proposal review, reporting and monitoring (2007–13) 2.00 TFSCB project supervision (2007–12) 0.60 Administration fee (2%) 0.38 I. Unallocated funds (I = G - H) 0.76 J. Total available funds (J = C + F + I) 6.17 Note: Allocation to projects is the disbursed amount for closed projects and the allocated amount for active projects. In the previous progress reports, committed amounts were used for all projects. TFSCB oversight and project supervision costs are estimated for the lifetime of TFSCB II and III. Source: World Bank, Development Data Group, The Trust Fund for Statistical Capacity Building at 10 – Investing in Statistical Capacity, October 2010. 97 Annex F Annex F. Persons Consulted PARIS21 Secretariat Mr. Mohamed-El-Heyba Lemrabott (Abadila) Berrou, Manager, PARIS21 Secretariat Mr. Christophe DUHAMEL, Deputy Manager Mr. Eric BENSEL, Partnership and Studies Program Coordinator’ Mr. Samuel BLAZYK, NSDS Program Coordinator, formerly Program Officer for Africa Mr. Jon HALL, Program Officer for Asia, Pacific and Middle East Mrs. Anna SAROTTE, Advocacy Program Coordinator, Program Officer for Latin America Mr. Jean-Marc LANDAIS, Consultant Mr. François FONTENEAU, ADP/IHSN Program Officer PARIS21 Steering Committee/Board Mr. Eckhard DEUTSCHER, former Co-Chair Mr. Pali Jobo LEHOHLA, former Co-Chair DECDG/World Bank Mr. Misha BELKINDAS, former Manager Mr. Grant CAMERON, Manager Mr. Mustafa DINC, Senior Economist/Statistician (in charge of TFAU) Mr. Neil FANTOM, Manager, formerly in charge of MAPS Unit Ms. Nada HAMADEH, Statistical Officer (MAPS Unit) Ms. Barbro HEXEBERG, Senior Economist, in charge of MAPS Unit Ms. Ingrid IVINS, Statistician Mr. Haeduck LEE, Senior Economist/Statistician, Asia/Pacific Ms. Naoko WATANABE, Statistical Officer (TFAU) Africa Region/World Bank Mr. Antoine SIMONPIETRI, Senior Statistician Ms. Ghislaine DELAINE, Senior Statistician IEGPS Ms. Victoria ELLIOTT, Consultant Ms. Lauren KELLY, Evaluation Officer CFPPM (Global Partnership and Trust Fund Operations – Management Unit) Ms. Anju SHARMA, Senior Program Officer Annex G 98 Annex G. Response of the Programs to IEG’s Global Program Review From PARIS21 PARIS21 congratulates the IEG on this comprehensive and meticulous GPR of PARIS21, MAPS and TFSCB. PARIS21 acknowledges and applauds the main findings outlined in Section 6 and welcomes in particular its recommendation in Annex D that the Partnership’s work be continued. PARIS21 adds its voice to the GPR’s call on the Bank to further strengthen its commitment toward statistical development activities (point 6.4 under “Main Findings�) and echoes the strong case made for renewed Bank engagement in funding PARIS21’s core program (point 6.5). PARIS21 finds unfortunate the DGF Management’s decision to gradually downsize and then completely phase out by 2015 its contribution to the core PARIS21 Secretariat program (Coordination, Advocacy, and NSDSs), a move that appears inconsistent with the objectives and impressive results achieved by both the TFSCB and MAPS programs and also in light of the GPR’s assessment that the user community needs to be more systematically involved in statistical capacity development programs (points 6.2 and 6.6). We certainly hope that the Bank will soon resume a long-term and significant financial commitment to the core PARIS21 Secretariat program. From DECDG Management congratulates IEG on a very thorough, thoughtful, and well-written review of the MAPS/PARIS21/TFSCB programs. We found the description and analysis of these programs to be first class. We believe the assessment framework to be appropriate and sensibly innovative – for example, in regards to the approach to assessing MAPS by reviewing progress under the six strategic actions. Overall, we welcome the following key findings of the Review: • The strong relevance of the three programs in supporting, and continuing to support, statistical capacity building in countries (para 11, para 3.2) recognizing they form a coherent package (para 3.15) and are parts of a common effort to build a culture of evidence-based policy making (para 1.2). • Regarding efficacy, the three programs have a strong record of yielding important results under each of the six MAPS actions which has successfully provided an overarching framework for statistical capacity building efforts (para 12, 3.18, 3.19) • The three programs are cost-efficient, with little evidence of wastage (para 13) and their governance and management broadly comply with the generally accepted principles of good governance (para 4.10) and attain high standards in governance efficiency (4.16) • In its role as partner, the Bank’s active involvement helped build a broad network of partners involved in statistical capacity building (para 5.3). 99 Annex G Management also acknowledges areas for improvement identified in the review and will address these issues in the subsequent sections of the note. Management disagreed with one IEG finding which is discussed in more depth below — the SRF/CF can be expected to help partly alleviate the impact of the gradual reduction of DGF (para 5.20). Management also feels the Review could have been clearer on whether alternative source of grant funding from Bank finances (i.e. grants approved by the Bank’s Executive Board outside of the DGF envelope such as CGIAR and the SPF) should be explored in light of the Bank’s partnership role. MANAGEMENT’S REFLECTIONS ON THE FINDINGS AND LESSONS IN CHAPTER 6 Para 6.1: “Significant progress has been achieved in the primary objective of encouraging and supporting developing countries to design National Strategies for the Development of Statistics (NSDSs), but only some progress has been made in NSDS implementation.� Management agrees there has been significant progress in supporting the design of NSDSs and some progress has been made in NSDS implementation. • On NSDS design, over 90% of developing countries now have NSDSs in place or are planning one. This success is due to: (a) the intensity of support provided by PARIS21 (through biannual reports on NSDS status in all developed countries, guidelines for NSDS development, the in-progress revisions to NSDS guidance materials that reflect the experience gained over the past several years, the design of an NSDS quality assessment framework to guide future strategies), (b) the TFSCB (which the report notes has provided significant funding of the development of NSDSs since 2004), and (c) MAPS action 1 which called for the rapid mainstreaming of NSDS development in all low-income countries and “one can safely say that the NSDS processes have helped many governments develop a strategic vision about statistical development� (para 2.15). • On NSDS implementation, about ½ of IDA countries and 46% of lower middle- income countries are implementing NSDSs, but it is clear that not all NSDSs are complete or are of adequate quality. More needs to be done to help countries prepare realistic implementation plans, which sets out what will be done, by whom and in which sequence, as well as a detailed budget and financing plan. In response, PARIS21 has already embarked on an exercise to review NSDS documents and to advise countries on where improvements are needed. However, we must recognize that, even with implementation plans in place, statistics must compete with other development priorities for scarce budgetary resources. • To help fill the financing gap for NSDS implementation, as noted in the IEG Review, a new partnership “Statistics for Results Facility: Catalytic Fund� is in its pilot phase to test mechanisms providing finance for investment in statistical capacity in the poorest countries and acting as a catalyst in mobilizing new finance from other sources. Management is assessing other impediments to implementation including: ensuring that the statistical strategy is closely coordinated with and linked to national Annex G 100 development documents (e.g. poverty reduction strategies, planning processes such as the budgeting cycle and medium-term expenditure frameworks). Para 6.1: The quality of statistics has improved, but the contributions of these three programs to developing a culture of evidence-based policy making, reducing poverty, and achieving MDGs, is difficult to evaluate. Management agrees that is difficult to assess the specific linkages between improvements in statistical capacity and an improved culture of evidence-based policy making and, ultimately, improvements in development outcomes. • As the Review notes, this is a long-term, multi-faceted process. As part of its revised logical framework, PARIS21 is now monitoring key performance indicators relating to “develop a culture of Management for Development Results� such as: (a) the percentage of countries that have developed sound frameworks for monitoring development results (baseline: 9 percent, 2014 target: 40 percent and to be measured through OECD/DAC process) and (b) average score of the use of statistics in the policy-making process (measured by computing a simple average of scores across all countries reviewed). These new indicators complement the indicators and information presented in the Bulletin Board on Statistical Capacity (BBSC) that includes the statistical capacity building indicator referenced in the Report, other information characterizing the national statistical system, and supporting World Bank projects. Para 6.2: “Statistical capacity building programs need to involve users more actively… more particularly policy makers� and “revisit the governance arrangements in all three programs.� Management agrees that more needs to be done to ensure statistical capacity building efforts involve policymakers. • The expanded governance of PARIS21, which recognized the importance of hearing the views of users, held a special session on the views of users at its April 2011 meeting. Principles underlying the Statistics for Results Facility ensure that developing countries demonstrate effective leadership in developing their national statistical systems. Management is working with its partners in statistical capacity building to update the strategic actions of MAPS to ensure prioritization of statistical capacity building explicitly reflect the policy priorities of the government. Monitoring the trends in PARIS21’s key performance indicators, described in our response to para 6.1, will also inform progress. Para 6.3: “There appears to be a shift in the perception of developing countries toward statistical capacity building.� Management agrees that statistical capacity building is receiving higher priority relative to the past. However, management is not yet ready to fully accept IEG’s pronouncement that “statistical development has begun to receive the same priority as other sectors.� 101 Annex G • 40 percent of IDA countries are aiming to have frameworks for monitoring development results by 2014, up from a baseline of 9 percent. However, statistical capacity is a form of supporting (yet invisible) infrastructure, and does not always garner political attention. There will be an on-going challenge to ensure predictable funding can be found as there are many development priorities that require funding. Para 6.4: “There is a need to strengthen the Bank’s commitment toward statistical capacity building activities.� Management agrees that more can be done to intensify statistical capacity building in Bank operations as reflected in the expanded mandate of the Statistical Capacity Building Committee (SCBC). • At its March 2011 meeting, the SCBC requested DEC to develop an action plan to help strengthen the Bank’s commitment. In the coming months, the SCBC will review specific actions in the following areas: Improving awareness of statistical capacity issues in CAS preparation: Under exploration is the idea to pilot in select countries an “environmental scan�. Drawing from NSDSs, ROSCs, the views of staff, etc, the scan would identify critical statistical gaps and the type of intervention that would fill the gap (e.g. support to improve data production and accessibility, requirements for institutional reforms, specific knowledge of conducting surveys, etc.). The summary of this “scan� would be aligned with the CAS preparation process to ensure statistical capacity priorities are considered. Management believes this “scan� is a logical next step from the guidance note developed to inform CAS teams regarding statistical capacity building issues. Improving collaboration between statistical practitioners by facilitating the sharing of experiences and lessons learned of staff working in statistical capacity building operations. DEC is creating a space for knowledge exchange to facilitate collaboration for operational staff working in statistics. Establishing this “statistical practice� experiences can be shared more easily on, for example, how priority gaps were identified and how the bank responded, strategies to improve donor coordination in country, ensure that planning and finance officials were queried for their views on priorities for improving statistics, and project design and implementation issues. Linking statistical capacity building more directly with similar priority areas by improving communication and awareness of synergies between “statisticians� and other sector experts (e.g. M&E, data-intensive sector specific policies such as conditional cash-transfer schemes, etc.). The Virtual Statistical System (VSS) additional development and outreach: The VSS is an online resource for national statistical offices, other data producing agencies, data users, including policy makers, academics, students, or anyone who wants to know more about official statistics. The VSS provides in-depth information on how effective statistical systems/organizations operate and the essential knowledge Annex G 102 prerequisites of official statisticians working in these organizations. The VSS is organized around three main components: (a) Knowledge Base - A repository of reference material on the building blocks of National Statistical Systems and information on thematic or sector-specific statistical operations, (b) eLearning - through a rich library of courses on 10 selected topics, and iii) a Knowledge Exchange designed to allow for the sharing of best practice and collaboration on issues related to national statistical systems development. The VSS went on-line last October and we continue to promote awareness amongst key client segments and adjust the material based on client feedback. Statistics and Program-for-Results (P4R). P4R will focus Bank support directly on improvements in sector or other development programs and will place more direct emphasis on results by making them the basis for disbursement. This results orientation will require quality statistical information to be successful. Integration of statistical capacity building with other public sector reforms by encouraging greater participation of staff involved in statistical capacity building in related formal and informal knowledge communities (e.g. PSD thematic groups on civil service reform, sub-national governments, demand for good governance, etc.). Staff running these communities need to acknowledge the value of including statistical capacity building. This could help “brand� or integrate statistical work into broader institutional/public sector reforms supporting transparent government. Organizational arrangements: Over the longer term, we could make relevant sector boards more aware of statistical capacity building across three sub-themes (i.e. data production and accessibility, institution-wide reforms, and specific technical expertise) as core skill requirements (in particular, the Public Sector Governance Board and the Economic Policy Board). Management needs to continue to experiment on how to best deliver statistical capacity building within the matrix environment. Para 6.5: “There is also the need to revisit the Bank’s engagement following the decision to phase out DGF core funding for PARIS21’s core work program.� As cited in the Review, management agrees that the Bank can play a critical role in donor coordination based on its active experience in these partnerships. • The timing of the re-orientation of the DGF towards supporting short-term innovated partnerships is unfortunate in light of the Review’s expectations that donor funding for statistical capacity building will decline in the coming years at a time when reliable statistics will be increasingly more important (para 4.2). These declining trends in funding are already in evidence. The PRESS 2010 round reports that donor commitments to statistical capacity building have declined to $182m in 2010 compared to $336m in 2008. Given the Review’s finding of strong relevance for the three programs over the medium-term, and the recent increase in in-country demand for statistics, the Bank must strive to ensure funding levels remain robust and predictable. 103 Annex G • Declining DGF support for the PARIS21 Secretariat is especially unfortunate as the Review finds PARIS21 to be well managed and the outputs were commensurate with staff levels and expenditures (para 3.57). Funding declines will affect recipient countries as they have benefited from: (a) the effectiveness of PARIS21 staff and consultants during missions, (b) well organized and conducted regional workshops, (c) peer reviews that were well organized and efficiently facilitated, (d) regional network of that advisors played multiple roles thus saving money, and (e) building on easy-to-use technology reduced costs for training and other development activities. Para 6.6: “More attention should be given to training and to the quality of statistics in order to more actively involve users of statistics.� Management agrees that past efforts need to be expanded upon. Actions are already underway to ensure critical users of statistics are able to effectively use them in their day-to- day activities. • To train statisticians to produce, analyze and talk with users about the data in an effort to improve the data production, most TFSCB (that are not for NSDS development), STATCAP and SRF projects have considerable training elements. • As mentioned in the response to para 6.2 above, the partnership has identified this as an area of special focus and the PARIS21 consortium is increasing efforts to hear the voices of policy makers and revisions to the MAPS actions under development will reflect this issue more explicitly. Para 6.8: “Effectiveness requires explicit strategies for achieving outcome objectives.� Management agrees with the Review’s suggestions to (a) make NSDSs more realistic and sustainable, (b) reinforce NSDSs as a continuous process with regular feedback on implementation (para 3.50), and agree that statistical data producers need to understand the policy environment and requisite data requirements (para 3.51). • As evidenced by its findings from STATCAP and SRF countries, implementation progress is much faster where a quality NSDS with clear implementation priorities is in place. Implementation strategies, where in-country capacity requires significant improvements to National Statistical System, must be sequenced with other work program priorities. Reforms need to take place alongside the regular statistical work program. With many countries participating in the 2010 round of population census, a complex and expensive exercise, implementation strategies become even more essential. Management believes that the recent efforts by PARIS21 to improve the quality of NSDSs, and develop lessons of good practice, will support more rapid implementation in the future. • Regarding the reporting of project results, the TFSCB’s 10-year anniversary report showcased outcomes of selected projects. Management intends to continue the documentation of tangible results from relatively big projects by directly obtaining detailed project outcome information from Task Team Leaders. However, we should Annex G 104 keep in mind that relative cost associated with project evaluation for small projects, such as those funded by this program, is high and may outweigh the benefit. We will review recommendations from other global partnership programs that have improved results reporting to inform prospective improvements in the TFSCB. Para 6.9: “Some selectivity may be needed to make progress.� Management agrees and notes the Reviews findings that the SRF pilot approach may be more realistic when large-scale reforms are required and agrees that further progress can be made to improve selectivity in prioritizing and sequencing smaller-scale reforms. • Through revisions to MAPS actions, the Bank will suggest prioritizing implementation of statistics supporting high-priority country-level development objectives. Here, the emphasis would be on selective improvements to statistical information critical to developing policy priorities, country-level M&E systems and results frameworks. Para 6.10: “Coordinated financial support across donors for statistical capacity building at the country level is important for moving the agenda forward and could benefit from documenting and sharing different approaches widely.� Management agrees that donor coordination is a long-term effort and that broader efforts – including through basket funds – support a broader program focus to statistical capacity building. • Supplementing the ideas suggested by the Review are the identification of best practices through our experiences in the SRF/CF where efforts are made to align donor support for statistics with the national strategic framework and NSDS implementation. For example, donor-funded in-country statisticians within SRF pilot countries have proved to be a very important step in improving dialogue with government counterparts and coordination amongst the donor community. Para 6.11: Awareness gap on the need for statistical development between DECDG and the Bank’s regional operational activities may require stronger advocacy efforts inside the Bank. Management agrees that greater action is required to ensure operational teams are fully informed about gaps in critical statistical areas, and the technical and financial support that can help fill these gaps. • The Bank’s Statistical Capacity Building Committee is a well-positioned to actively advocate for the importance of statistical development in operations. As outlined in our response to para 6.4, the Committee has begun exploring options to bridge the informational gap between statistical experts in DECDG and operational staff. The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor Issue #2: Global Development Network Issue #3: Global Forum for Health Research Issue #4: Global Invasive Species Program Volume #4, Issue #1: Stop Tuberculosis Partnership Issue #2: International Assessment of Agricultural Knowledge, Science, and Technology for Development Issue #3: The Global Water Partnership Volume #5, Issue #1: Multi-Donor Trust Fund for the Extractive Industries Transparency Initiative Issue #2: The Mesoamerican Biological Corridor Issue #3: Marrakech Action Plan for Statistics, Partnership in Statistics for Development in the 21st Century, and Trust Fund for Statistical Capacity Building GPR The Partnership in Statistics for Development in the 21st Century (PARIS21), the Trust Fund for Statistical Capacity Building (TFSCB), and the Marrakech Action Plan for Statistics (MAPS) are part of an international effort to strengthen national statistical sys- tems and the use of development statistics in developing countries. The World Bank has played an important role in this international effort as founder and active player in all three programs. Although the programs have similar goals, they continue to be separate because they arose in different contexts, at different times, and with different sources of funding — PARIS21 at the UN Conference on Development in 1999 to promote a culture of evidence- based policy making, the TFSCB as a World Bank-administered trust fund in 1999 to provide small grants to help countries strengthen their statistical systems, and MAPS at the Second Round Table for Managing for Development Results in 2004 as a global plan to improve development statistics. IEG’s Global Program Review confirms the findings of recent evalu- ations of the three programs on their strong relevance, their strong record of outputs in the six MAPS’ priority areas, cost-efficiency, and compliance with generally accepted principles of good governance. The Review found that significant progress has been achieved in the primary objective of encouraging and supporting developing countries to design National Strategies for the Development of Statistics (NSDSs), but only some progress in NSDS implementation. The Review suggests a number of measures to accelerate this progress including: (a) making NSDSs more relevant, realistic and sustainable; (b) reinforcing NSDSs as a continuous process with regular feedback on implementation; (c) more actively involving the users of statistics in capacity building efforts; and (d) increasing the volume of financial and technical resources to strengthen statistical systems.