GPR 57640 CONSULTATIVE GROUP TO ASSIST THE POOR GLOBAL PROGRAM REVIEW Volume 3 Issue 1 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi- ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. The Consultative Group to Assist the Poor (CGAP) October 26, 2008 Corporate and Global Evaluations and Methods http://www.globalevaluations.org ©2008 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: grpp@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. Cover photo: Local women's group in Bangladesh teaching women about nutrition, health, and basic mathematics to help them qualify for a loan. Photo by Shehzad Noorani, 2002, courtesy of the World Bank Photo Library. ISBN-13: 978-1-60244-099-9 ISBN-10: 1-60244-099-9 Printed on Recycled Paper Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) E-mail: grpp@worldbank.org Telephone: 202-458-4497 IEG Mission: Enhancing development effectiveness through excellence and independence in evaluation. The Independent Evaluation Group (IEG) of the World Bank reviews global and regional partnership programs (GRPPs) in which the Bank is engaged as one partner among many for two main purposes: (a) to provide accountability in the achievement of the program's objectives by providing an independent opinion of the program's effectiveness, and (b) to identify and disseminate lessons learned from the experience of individual GRPPs. The preparation of a global or regional program review (GPR) is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. The first purpose includes validating the findings of the GRPP evaluation with respect to the effectiveness of the program, and assessing the Bank's performance as a partner in the program. The second purpose includes assessing the independence and quality of the GRPP evaluation itself and drawing implications for the Bank's continued involvement in the program. Assessing the quality of GRPP evaluations is an important aspect of GPRs, since encouraging more consistent evaluation methodology and practice across Bank-supported GRPPs is one of the reasons why IEG embarked on this new product in 2005. IEG annually reviews a number of GRPPs in which the Bank is a partner. In selecting programs for review, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming sector studies; those for which the Executive Directors or Bank management have requested reviews; and those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR is a "review" and not a full-fledged "evaluation." It assesses the independence and quality of the relevant evaluation; provides a second opinion on the effectiveness of the program; assesses the performance of the Bank as a partner in the program; and draws lessons for the Bank's engagement in global and regional programs. The GPR does not formally rate the various attributes of the program. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside of the World Bank or Washington, DC. Key stakeholders include the Bank's representative on the governing body of the program, the Bank's task team leader (if separate from the Bank's representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached as an annex to the final report. After the document has been distributed to the Bank's Board of Executive Directors, it is disclosed to the public on IEG's external Web site. ii Abbreviations and Acronyms ADB Asian Development Bank AFD Agence Française de Développement AfDB African Development Bank CAS Country Assistance Strategy CDB Development Bank of China CEO Chief Executive Officer CODE Committee on Development Effectiveness (World Bank) CGAP Consultative Group for Assisting the Poor CLEAR Country-Level Effectiveness & Accountability Review DAC Development Assistance Committee (OECD) DFI Development Finance Institutions DGF Development Grant Facility (World Bank) EBRD European Bank for Reconstruction & Development EXCOM Executive Committee (CGAP) EU European Union FPDVP Financial & Private Sector Development, Vice president FSAP Financial Sector Assessment Program GRPP Global & Regional Partnership Programs GPR Global Program Review IDB Inter-American Development Bank IEG Independent Evaluation Group (World Bank) IFAD International Fund for Agricultural Development IFC International Finance Corporation IFI International Financial Institutions ILO International Labor Office MDG Millennium Development Goals MFI Microfinance Finance Institution MIX Microfinance Information eXchange OECD Organisation for Economic Cooperation and Development PPI Progress Out of Poverty Index PPIC Pro-Poor Innovation Challenge QER Quality Enhancement Review RAS Retail Advisory Service SME Small and Medium Enterprises TOR Terms of reference UNCDF United Nations Capital Development Fund UNDP United Nations Development Program Fiscal Year of Program July 1 to June 30 Director-General, Independent Evaluation Mr. Vinod Thomas Director, Independent Evaluation Group (World Bank) Ms. Cheryl Gray Manager, Corporate and Global Evaluations and Methods Mr. Mark Sundberg Global Programs Coordinator Mr. Chris Gerrard Consultant Mr. Khalid Siraj iii Contents ABBREVIATIONS AND ACRONYMS................................................................................................... II PROGRAM AT A GLANCE: CONSULTATIVE GROUP TO ASSIST THE POOR (CGAP).................V KEY BANK STAFF RESPONSIBLE DURING PERIOD UNDER REVIEW .........................................VI GLOSSARY..........................................................................................................................................VII PREFACE..............................................................................................................................................IX EXECUTIVE SUMMARY.......................................................................................................................XI 1. PROGRAM OBJECTIVES, ACTIVITIES, GOVERNANCE, AND COSTS................................. 1 Objectives and Activities ........................................................................................................ 1 Governance .............................................................................................................................. 2 Funding and Costs .................................................................................................................. 4 2. THE EXTERNAL EVALUATION OF CGAP ............................................................................. 5 Scope, Approach and Process .............................................................................................. 5 Independence and Quality...................................................................................................... 6 3. THE EFFECTIVENESS OF CGAP ......................................................................................... 10 Relevance ............................................................................................................................... 10 Efficacy ................................................................................................................................... 15 Promoting Institutional Diversity ......................................................................................................20 Promoting Diverse Financial Services to a Broad Range of Clients ................................................21 Building Financial Market Infrastructure ..........................................................................................22 Fostering Sound Policy and Legal Frameworks ..............................................................................23 Improving the Effectiveness of Microfinance Funding .....................................................................24 Efficiency................................................................................................................................ 25 Governance ............................................................................................................................ 26 4. THE WORLD BANK'S PERFORMANCE AS A PARTNER .................................................. 28 Providing Comparative Advantages at the Global Level .................................................. 28 Promoting Microfinance at the Country Level.................................................................... 29 Exercising Oversight............................................................................................................. 31 Risk Management .................................................................................................................. 32 iii iv Disengagement Strategy...................................................................................................... 32 5. LESSONS ............................................................................................................................... 33 REFERENCES..................................................................................................................................... 35 ANNEX A. EVALUATION FRAMEWORK FOR GLOBAL PROGRAM REVIEWS............................ 37 ANNEX B. PROGRAM GOALS AND OBJECTIVES ......................................................................... 44 ANNEX C. PROGRAM TIMELINE ...................................................................................................... 45 ANNEX D. MEMBERS OF THE GOVERNING BODIES .................................................................... 46 ANNEX E. CGAP: SOURCES AND USES OF FUNDS ..................................................................... 53 ANNEX F. PERSONS CONSULTED .................................................................................................. 55 ANNEX G. RESPONSE OF THE PROGRAM TO IEG'S GLOBAL PROGRAM REVIEW ................ 58 Tables Table 1: Evaluation Recommendations and Management Response ............................................ 8 Table 2. CGAP: Major Planned and Actual Outputs ....................................................................... 17 Table 3. The World Bank's Lending for Micro and SME Finance, FY2000­07 (US$ millions).... 30 Table 4. Recognition of CGAP and Microfinance in CASs ............................................................ 30 Annex Table 1. Assessing the Independence and Quality of the Evaluation .............................. 37 Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program......... 38 Annex Table 3. Assessing the Bank's Performance as a Partner in the Program...................... 43 Figures Figure 1. CGAP's Broad Themes, Phases I­IV ................................................................................. 2 Figure 2. CGAP Organizational Chart ................................................................................................ 3 Figure 3. The World Bank's Share in Total CGAP Funding ............................................................. 5 Figure 4. CGAP Expenditures on Five Strategic Priorities, FY2004­07 (Phase III) ..................... 16 v Program at a Glance: Consultative Group to Assist the Poor (CGAP) Start date 1995 Mission CGAP's basic mission -- to help develop a sustainable microfinance industry -- has been the same since its establishment in 1995, but its emphasis has evolved. For Phase III (FY04­08), the mission was to help build financial systems that work for the poor, providing large numbers of people with diverse financial services through a wide range of organizations. In its current strategic plan for FY09­13, CGAP's mission is to help build efficient local financial markets that are integrated into the mainstream financial system and that serve all the unbanked, including very poor and harder-to-reach clients with ever more innovative, convenient, and affordable financial services. Strategic priorities In Phase III, CGAP's strategic priorities were: (a) Promoting institutional diversity (b) Promoting diverse financial services to a broad range of clients (c) Building financial market infrastructure (d) Fostering sound policy and legal frameworks (e) Improving the effectiveness of microfinance funding Activities During Phase III, CGAP's current generic activities were: (a) Providing advisory services (b) Developing and setting standards (c) Advancing knowledge (including sector intelligence and information sharing) (d) Offering training and capacity building WBG contributions Special Grants (FY95­98) $30.7 million DGF (FY99­08) $58.9 million Total $89.7 million Other donor Other 32 members have contributed $92.8 million ($64.9 million core contributions and contributions $28.9 million designated contributions). Location World Bank Governance and During Phase III, CGAP's governance consisted of the following: management (a) Council of Governors: Consists of a representative from each of the 31 CGAP members. The Governing Council approves amendments to the Charter and strategy, provides policy and other inputs to annual work plans and budgets and elects Executive Committee members. (b) Executive Committee (EXCOM): Consists of nine members, four representing different constituencies on the Council of Governors, four representing the broader microfinance industry (not members of CGAP), and one representing the World Bank. The Director/CEO of CGAP is also an ex-officio member. EXCOM provides guidance to the Operational Team, approves new members, work plans and budgets after consultations with the Council of Governors, and proposes amendments to the Charter of CGAP. (c) Investment Committee: Consists of 9­14 members designated by the World Bank. It approves all CGAP commitments over $100,000 (from $100,000­250,000 on a no-objection basis). (d) Operational Team: Led by the Director/CEO, the Operational Team is responsible for the day-to-day management and operations of CGAP. Latest program- CGAP Phase III Mid-Term Evaluation (July 2003­June 2006), March 2007, conducted by level evaluation a group of private consultants (core team: Klaus Maurer, Sarah Foster, Michael Mithika; and for Francophone Africa: Christine Poursat). vi Key Bank Staff Responsible during Period under Review Position Person Period Program Manager Elizabeth Littlefield, 1999 to present Executive Director Global Program Task Carlos Cuevas 1999 to January 2008 Manager Martin Holtman February 2008 to present Vice President Nemat Talaat Shafik, Jan. 1999 ­ Sept. 2004 Vice-President, Private Sector Development & Infrastructure Cesare Calari, Vice-President, Jan 2005 ­ Aug 2006 Financial Sector Development Michael Klein, Vice-President, Sept 2006 to present Financial and Private Sector Development (World Bank and IFC) Trust Fund Operations Arif Zulfiqar, Director June 1999 to September 2008 Global Programs & Margaret Thalwitz, Director May 2004 to September 2008 Partnerships vii Glossary Donor Any organization or entity that makes a financial or in-kind contribution to a program that is reflected in the audited financial statements of the program. Therefore, this includes not only "official donors" but also developing countries that contribute annual membership dues, seconded staff, or office space, provided that these are formally recognized in the financial statements of the program. Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. The term is also used as a broader, aggregate measure -- encompassing relevance and efficiency as well -- of the overall outcome of a development intervention such as a GRPP. Efficiency The extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs. Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or project, its design, implementation, and results. The aim is to determine the relevance and achievement of its objectives, and its developmental effectiveness, efficiency, impact, and sustainability. Exit strategy A proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it ha achieved its objectives or that its current design in no longer the best way to sustain the results which the program has achieved. Fund providers In the case of CGAP, donors, foundations, microfinance investment funds, and others that provide financial resources for the development of the microfinance industry or to providers of retail microfinance services. Governance The structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. It is the framework of accountability and responsibility to users, stakeholders and the wider community, within which organizations take decisions, and lead and control their functions, to achieve their objectives. Impacts Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Independent An evaluation that is carried out by entities and persons free from the control evaluation of those involved in policy making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from interference, and avoidance of conflicts of interest. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Legitimacy As a criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large. viii Management The day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. Networks In the context of CGAP, networks are organizations that encompass a number of institutional partners with interests in the microfinance business. These serve the interests of their partner organizations in various ways such as launching new institutions, developing standards, providing technical assistance, implementing knowledge management and leading policy reforms. Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development intervention. Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Partners Stakeholders who are involved in the governance or financing of the program (including the members of the governing, executive, and advisory bodies). Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular public good accrue across all or many countries, then the good is deemed a global or international public good. Relevance The extent to which the objectives and design of the program are consistent with (a) the current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. Stakeholders The parties who are interested in or affected, either positively or negatively, by the program. Stakeholders are often referred to as "principal" and "other", or "direct" and "indirect". While other or indirect stakeholders -- such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries -- may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. Sustainability When the term is applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When the term is applied to organizations or programs themselves, the extent to which the organization or program is likely to continue its operational activities over time. Source: Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Independent Evaluation Group ­ World Bank, 2007. ix Preface This is the Global Program Review (GPR) of the Consultative Group to Assist the Poor (CGAP). Established in 1995 as a consortium of public and private donors with its secretariat located in the World Bank, the mission of CGAP has been to help build efficient local financial markets that are integrated into the mainstream financial system and that serve all the unbanked, including very poor and harder-to-reach clients with ever more innovative, convenient and affordable financial services. Its specific objectives have been to generate and disseminate knowledge, to catalyze the movement toward good-practice performance standards, and to build consensus among its many and varied stakeholders. In pursuit of these objectives, CGAP's generic activities have consisted of (a) providing advisory services, (b) developing and setting standards, (c) advancing knowledge (including sector intelligence and information sharing) and (d) offering training and capacity building services. CGAP's governing body has commissioned periodic independent evaluations since its inception. The most recent one was in 2006, evaluating CGAP's third phase. This was conducted by a three-person team of external consultants and completed in 2007. This GPR assesses the quality and independence of the 2007 evaluation. In addition, it provides a second opinion on the effectiveness of CGAP's work, assesses the performance of the World Bank in its management of and support to CGAP, and draws some lessons. It covers the period from FY04­07, which corresponds to the first four years of CGAP's Phase III (FY04­08). The period prior to FY04 was covered by an earlier IEG case study of CGAP as a part of its 2004 evaluation of the World Bank's approach to global programs. This review follows IEG's Evaluation Framework for Global Program Reviews (Annex A). It is based on a desk review of the 2007 Evaluation report as well as relevant CGAP documents (annual reports, internal documents, strategy papers, etc.), publications and Web site. In addition, IEG independently obtained opinions and views on CGAP and its activities by interviewing some members of CGAP's Council of Governors, all Executive Committee and Investment Committee members, CGAP's managers and staff, and a number of World Bank managers and staff knowledgeable about CGAP (Annex F). The World Bank's internal database was used to collect information on the Bank's support to the microfinance sector. Finally, relevant World Bank publications were also reviewed. IEG gratefully acknowledges all those who made their time available for interviews and provided useful information and insights that made this review possible. It wishes to especially acknowledge the availability of CGAP staff and their cooperation in providing all necessary information and documents. Throughout the process of this review they showed a keen interest in learning ways of improving CGAP's approach and activities. Following IEG's normal procedures, copies of the draft GPR were sent to CGAP and other World Bank units that have responsibility for the Bank's involvement with global programs as well as for the World Bank's operational activities for supporting the development of microfinance sector. Their comments have been taken into account in finalizing the GPR. The formal response received from CGAP management is attached in Annex G. xi Executive Summary Objectives, Governance & Funding 1. CGAP was established in 1995 as a consortium of public and private agencies with the mission of expanding access to financial services for the poor. The World Bank provided funding of $30 million from its Special Grants Program for a three-year period and also agreed to act as the overall administrator and to host the CGAP secretariat inside the World Bank. In 1998, the Council of Governors extended CGAP for another five years (Phase II, FY99­03). Subsequently, the Council of Governors approved a further extension for Phase III (FY03­08) in 2003 and most recently approved a five-year strategy (FY09­13). 2. Over the years, CGAP's priorities and focus of activities have evolved significantly, reflecting the increasing maturity and experience of CGAP, and the rapidly changing environment in the microfinance sector. CGAP's current mission, defined in its Charter, is to help build efficient local financial markets that are integrated into the mainstream financial system and that serve all the unbanked, including very poor and hard-to-reach clients with innovative, convenient and affordable financial services. Since its inception, the specific objectives of CGAP have been to generate and disseminate knowledge, to catalyze the movement toward good-practice performance standards, and to build consensus among its many and varied stakeholders. In pursuit of these objectives, CGAP's generic activities have consisted of (a) providing advisory services, (b) developing and setting standards, (c) advancing knowledge (including sector intelligence and information sharing) and (d) offering training and capacity building services. 3. CGAP is a consortium of public and private agencies whose secretariat is located in the World Bank, and whose membership is open to funding organizations with mandates to promote public goods. The governance structure consists of four parts: the Council of Governors (all CGAP members), the Executive Committee (EXCOM) elected by the Council of Governors, the Investment Committee appointed by the World Bank, and the Operational Team headed by a Director/CEO who is selected by the World Bank and endorsed by the Council of Governors. The rest of the Operational Team is recruited by the CEO according to World Bank recruitment procedures and employed by the World Bank. A World Bank Vice President has so far chaired the Council of Governors as well as the Investment Committee. During Phase III, EXCOM comprised four representatives of the Council of Governors, four representatives of the microfinance industry and one representative of the World Bank, and is chaired by a non-World Bank member elected by it. The Council of Governors provides major strategic direction and overall guidance to CGAP. EXCOM provides operational guidance while the Investment Committee approves investments and commitments over $100,000 (between $100,000 and $250,000 on a "no-objection basis") in exercise of the World Bank's fiduciary responsibility. So far, CGAP's current governance structure has served CGAP well as reflected in its smooth and friction-free functioning. 4. CGAP's funding requirements are provided by its members. Total cumulative member contributions since 1995 stand at $171 million, comprising $144 million (or 84 percent) of core contributions and $28 million designated for specific purposes. In aggregate xii the World Bank has been the largest single contributor but its share has declined from 81 percent in Phase I to 57 percent in Phase II and 18 percent in Phase III. Starting in July 2008, CGAP will make it mandatory for all members to make a core cash contribution; and those who do not meet this requirement would become non-voting members. This decision emanates from CGAP's resource mobilization strategy for FY09­13 emphasizing a broader resource base, reduced reliance on the World Bank and greater flexibility in the use of funds contributed by members. Some of the multilateral institutions might not be able to meet the new core contribution requirement due to their internal restrictions, though two such institutions are reported to have found a way out of this problem and it is likely that some of the others will follow suit. However, the non-voting members will be able to continue to participate in CGAP annual meetings and activities. The External Evaluation of CGAP 5. To assist the Council of Governors in taking some decisions on CGAP's future and also to fulfill a requirement of the DGF, EXCOM commissioned a team of external consultants in 2006 to conduct an independent evaluation, which was completed in 2007. The evaluation covered the entire CGAP system (the Council of Governors, EXCOM, the Investment Committee and the Operational Team) and used the standard evaluation criteria of relevance, efficacy, efficiency and accountability. The present review found the 2007 Evaluation to be credible, comprehensive and consistent with the IEG/DAC Sourcebook. The process of evaluation was managed by EXCOM in a fair and transparent manner. The evaluation team was fully independent and it was given sufficient freedom in its work for which EXCOM made available adequate financial and administrative resources and full access to staff, information and documents. The role of the Operational Team vis-à-vis evaluation was supportive without being in a position to dominate the outcome. On the whole, the quality of the evaluation was satisfactory. The evaluation report covered all relevant aspects with adequate analysis and was quite candid. The 2007 Evaluation is a reasonable basis to judge the performance of CGAP. The evaluation report emphasized the need for CGAP to put in place a system to monitor and report the extent to which its activities and initiatives contribute to achieving its intended outcomes and priority objectives, including its overarching objective of building inclusive financial systems. However, the 2007 Evaluation's assessment of CGAP's efficacy could have been more convincing if the evaluation had compiled information on intermediate outcomes and CGAP's contribution to achieving them. The analysis underpinning the findings of the evaluation report were logical and appropriate with one exception. The discussion on the relevance of CGAP could have made a more forceful case for CGAP as a global platform and forum for the microfinance industry. 6. Overall, the findings of 2007 Evaluation were extremely positive towards CGAP, concluding that it is a powerful and pivotal force in the microfinance field, playing a critical role in helping a broad range of stakeholders (governments, donors, investors, market players, etc.) to build inclusive financial systems. The report made recommendations concerning the current activities and directions of CGAP as well as the future directions of CGAP beyond Phase III. xiii 7. The 2007 Evaluation made a number of recommendations mostly on specific aspects of CGAP's initiatives and activities. CGAP management has positively responded to almost all the recommendations by taking appropriate actions. The Effectiveness of CGAP 8. Relevance. CGAP's basic mission has been to help develop a sustainable microfinance industry. On the supply side, CGAP was established in response to a broad international consensus that the availability of microfinance helps to reduce poverty, which has since been reaffirmed in a number of international forums such as the UN General Assembly's designation of 2005 as the Year of Microcredit and the awarding of the 2006 Nobel Peace Prize to Dr. Muhammad Yunus and the Grameen Bank for their work on the development of microfinance. The CGAP Client Survey conducted as part of the 2007 Evaluation also found that CGAP's contributions are viewed as very important to the success of microfinance globally. On the demand side, the demand for microfinance in developing countries has been rising rapidly and the number of microfinance institutions has increased dramatically in the last decade. A plethora of studies have concluded that enhancing the availability of microfinance is an effective tool to reduce poverty and help poor people manage their vulnerabilities, but a recent World Bank research study concluded that statistical research evidence on the benefits of microcredit is not yet overwhelming and made the case for further research to obtain more robust conclusions. However, the reservations in the latter study only apply to the impact of microcredit, not to other elements of microfinance (such as savings, insurance, and payment facilities), and the study recognized the benefits of other financial services for the poor as a contributor to achievement of most of the Millennium Development Goals. 9. CGAP's activities are overwhelmingly of a global nature aimed at benefiting the microfinance industry as a whole, with the share of such activities having increased from 60 percent in 2003 to 94 percent in 2007. Despite the impressive accomplishments of the microfinance industry over the past decade, the availability of microfinance has not yet spread over all -- or even a majority -- of developing countries. While a number of change agents are likely to contribute to the further development of the industry, there is no other global program/entity competing with CGAP at the present time in terms of providing global public goods. Its broad donor support, its adaptable business model, and the quality of its outputs have enabled CGAP to position itself as the recognized leader and voice of the microfinance industry. Areas in which there is scope for CGAP to continue to play this role include finding ways to protect the interest of the poorest, setting standards, contributing to policy reforms, exploring new frontiers, providing market intelligence and a common forum for dialogue. 10. Efficacy. CGAP's activities relate to its five strategic priorities, namely, (a) promoting institutional diversity, (b) promoting diverse financial services to a broad range of clients, (c) building financial market infrastructure, (d) fostering sound policy and legal frameworks, and (e) improving the effectiveness of microfinance funding. CGAP's achievements in terms of its activities and outputs in each of these five areas have been impressive, as detailed immediately below. But there is little systematic evidence relating to its achievements at the outcome level, or their attribution to CGAP's activities, due to xiv weaknesses in CGAP's monitoring and reporting system. The latter is well established at the level of measuring the outputs of its activities, but not at the level of measuring their outcomes. Therefore, the 2007 Evaluation concluded, and the present GPR concurs, that CGAP needs to develop and adopt a comprehensive monitoring and reporting system to assess the outcomes of its activities. Although the nature of CGAP's operation makes the assessment of outcomes and impacts complicated and difficult, this is needed to form a definitive judgment on the value added of CGAP's activities at the outcome level. Since the 2007 evaluation, CGAP management has started work on identifying ways of measuring outcomes at both the overall and activity levels. 11. A notable feature of CGAP's activities and outputs in Phase III has been the phase- out of its grant-based programs for institution-building of MFIs and Networks, since these no longer reflected CGAP's current strategic priorities. Accordingly, CGAP did not commit any new grants for MFI institution-building in Phase III, or for Networks after FY05. Some of CGAP's programs still involved grants in Phase III, but their objectives were broader than basic institution-building, such as enhancing the quality of information disclosure and encouraging pro-poor services. 12. In terms of outputs, CGAP achieved the following during Phase III in its five strategic priority areas: · Promoting Institutional Diversity. The objective of this strategic priority was to explore and deploy new delivery mechanisms for financial services for the poor to complement existing financial intermediaries. CGAP's outputs under this strategic priority exceeded the plan since almost all planned and two new activities were carried out. At the broader industry level, there has been progress in terms of outputs. But in the absence of an in-depth analysis of the causal relationship between CGAP outputs and developments in the industry, it is not possible to attribute the progress to CGAP's outputs. · Promoting Diverse Financial Services to a Broad Range of Clients. The objective of this strategic priority was to promote diversity in financial services and this objective was to be realized through documenting and piloting new models of financial services for the poor. CGAP's actual outputs exceeded what was planned including launching of the pioneering Graduation Program, which by developing methodologies to graduate the poorest people out of food insecurity into appropriate financial services could lead to a breakthrough for poverty alleviation policies and strategies. While the activities undertaken by CGAP to realize their basic objective seem appropriate, they are not at a stage of completion to allow a meaningful assessment of their outcomes. CGAP needs to bring the ongoing work to a closure, establish demonstration effects of the work done, and disseminate lessons. · Building Financial Market Infrastructure. The objective of this strategic priority was to enhance financial transparency of service providers to the poor, a pre-requisite for the development of the market. By the end of the fourth year of Phase III, CGAP had made good progress in implementing all planned activities. A notable achievement was the emergence of the now spun-off Microfinance Information xv eXchange (MIX) as the leading business information provider on the microfinance sector thereby addressing, to a large extent, the issue of the lack of reliable, comparable and publicly available information on microfinance institutions. In addition, through the Financial Transparency Awards program, CGAP provided an incentive to MFIs to improve the quality of their information and disclosures. The program provided recognition as well as financial rewards to MFIs. However, it is likely that financial institutions would have participated in the competition for recognition even without financial rewards. · Fostering Sound Policy and legal Frameworks. The objective of this strategic priority was to establish supportive legal and regulatory frameworks that safeguard poor people's money, facilitate diversity of institutions and financial products, and promote competition. In the first four years of Phase III, CGAP made considerable headway in achieving its targeted outputs. The most notable output was the creation of the Law Library, an online library on microfinance-related laws and regulations of more than 50 developing countries, thereby establishing CGAP as a major source of policy advice on microfinance and branchless banking. CGAP's policy work has raised the issues of the continuity of CGAP's involvement and the need for coordination with donors whose policy work would often overlap with CGAP's work. This calls for a better coordination between CGAP and donors. Due to the absence of relevant data, it is not possible to assess fully the outcome of CGAP's work on the policy regimes of relevant countries. However, there are anecdotal indications of CGAP advice having had some positive effect on policy makers. · Improving the Effectiveness of Microfinance Funding. In Phase III, CGAP aimed at working with fund providers (both donors and private sources of funds) to improve the effectiveness, efficiency and accountability of their operations in microfinance. CGAP's outputs in the first four years of Phase III exceeded the target. Assessing the outcome of these activities is difficult in the absence of an assessment of the extent to which the effectiveness, efficiency and accountability of fund providers, mainly donors, increased as a result of CGAP's outputs. This would be a complicated exercise because many of CGAP outputs have been of a general nature and directed to all fund providers. On the other hand, the work done by CGAP so far offers two important lessons: (a) Since CGAP does not have the capacity to work with all or even a majority of fund providers, it has had to evolve its strategy; and (b) the success of CGAP's efforts in enhancing the effectiveness of microfinance funding depends to a large extent on the willingness of fund providers to seek and benefit from CGAP's support, which in turn is heavily dependent on internal incentives and even the political considerations of each donor/funder. Therefore, the expected outcomes from CGAP's aid-effectiveness work should be kept at a modest level. The value of this activity lies in generating appropriate tools that fund providers who are serious about improving their internal systems can use readily. 13. Efficiency. The uniqueness of CGAP in terms of its mission and activities makes it difficult to evaluate its cost-effectiveness since there are no comparable benchmarks, further complicated by the absence of a systematic evaluation of the CGAP's overall impact on the industry and cost benefit analysis. However, strong anecdotal evidence suggests that CGAP's xvi activities are cost-effective. This conclusion is also shared by the 2007 Evaluation and confirmed by the Council of Governors and EXCOM members in interviews with IEG. This review did not find any evidence of noticeable waste of resources. On the contrary, there are indications that CGAP is highly cost-conscious. Finally, CGAP has demonstrated a high degree of adaptability to the evolving requirements of the microfinance sector to remain relevant. 14. Governance. CGAP's governance structure, which comprises a combination of shareholders and stakeholders, has served CGAP well without giving rise to any major problems. This governance structure has a strong legitimacy arising from the effective participation of almost all major stakeholders, the service providers, MFIs, networks, donors, other financiers, etc. The incorporation of the former Policy Advisory Group into a revised EXCOM in 2002 has generated a positive synergy between representatives of donors and industry and led to better policy and operational guidance to CGAP. Two recent decisions should further enhance the legitimacy of CGAP -- treating the World Bank at par with other members in terms of representation on EXCOM and inviting the Development Bank of China (CDB) to join CGAP as the first bilateral member from a developing country. CGAP has maintained a high degree of transparency. Its Web site provides detailed information on its governance, its strategy, finances and operating results and outputs as well as full disclosure of its external evaluation reports. However, there are a few areas where transparency could be further enhanced, such as a more transparent language in CGAP's Charter on the core contribution requirements for members and the adoption of a code of conduct for EXCOM members. There is also a high degree of accountability and fairness in the governance structure. The World Bank's Performance 15. CGAP's close association with the World Bank gives it access to many advantages, perhaps, the most valuable being the convening power and prestige of the Bank. It is doubtful if a majority of CGAP members, who represent almost all major international donors and fund providers, would have shown the same level of interest in CGAP in the absence of direct World Bank involvement. CGAP has used the convening power it derives from its association with the World Bank to successfully organize many international, regional and country-level conferences and meetings with positive results and to promote global consensus on its various policy and operational guidelines. CGAP's close proximity to the World Bank gives it access to the World Bank's knowledge reservoir and technical expertise. CGAP also enjoys significant tangible institutional advantages, especially access to policy makers in developing countries, ability to hire world-class staff and cost-effectiveness as the World Bank provides many support services. 16. Complementing CGAP's activities, the World Bank has played a significant role in promoting the development of microfinance and inclusive financial systems in developing countries. At the country level, the World Bank support comes in the form of lending (on average $516 million annually) and analytical work. At the global level, there were two notable World Bank contributions -- an international conference and publication of a major policy research report. These two efforts provide an excellent overview of the major trends and in-depth analysis of the emerging issues relating to microfinance and access to finance. xvii The World Bank has also benefited from the work of CGAP. A number of managers and staff interviewed by IEG expressed a high degree of respect for the quality of CGAP's work and expertise and there is a growing awareness in the World Bank of the importance of microfinance and access to finance, which, at least partly, could be attributed to the knowledge generated by CGAP and its advocacy work. 17. The World Bank has performed its fiduciary functions diligently, consistent with its legal responsibilities under various trust fund agreements. However, this close relationship also entails a potential risk, i.e., the possibility of CGAP being asked to play some of the anchor functions for microfinance operations in the World Bank. In principle, this function is consistent with CGAP's strategic priorities, but a substantial and regular involvement of CGAP in the internal management of the World Bank could generate a perception on the part of other CGAP members of pre-emption of CGAP's resources by the World Bank. 18. CGAP was initially established as a three-year program. Since then the Council of Governors has extended it three times. While taking these decisions no explicit consideration has been given to the option of closure or phasing out of CGAP. The justification for the continued existence of CGAP comes from its strong relevance discussed earlier. Therefore, in the foreseeable future, there are strong reasons for CGAP to remain in business in its current format and for the World Bank to support it. Before the end of CGAP's current strategy in 2013, the World Bank and its partners should start thinking about the long-term future of CGAP and the World Bank's role. In all probability, CGAP has a long-term role as an organization to serve the collective interest of the microfinance industry. However, the World Bank's involvement is only justified as long as CGAP continues to produce public goods effectively and efficiently. Lessons 19. The experience with CGAP offers a number of lessons, as follows: · Some of the factors that contributed to the effectiveness of CGAP are of generic relevance to other global programs. These include: (a) the basic mandate being highly consistent with a universally accepted development agenda; (b) a high degree of stability reflected in its organizational set up, governance structure and broad mission; and (c) a pragmatic approach of DGF towards enforcing its cost-sharing requirement. · Irrespective of the strong relevance and impressive performance of a GPP, it is important that a monitoring and reporting system be put in place to assess the outcomes of its activities. In the case of CGAP, there is no doubt that the program has produced high quality outputs which have been regarded as useful by many stakeholders. However, the extent to which microfinance contributes to alleviating poverty and CGAP's contribution towards such results need to be more firmly established. · The expected outcomes from a global program's activities aimed at improving the institutional set up and practices of donors and other fund providers should be kept at a modest level. As in the case of CGAP, such activities are laudable, but their xviii effectiveness depends on two critical factors -- the capacity of a global program to work with fund providers and the capacity and willingness of the recipients. A host of factors affect the outcome, including the incentive systems within each entity, the importance of the relevant sector in the overall context of the entity's mission, the complexity of the organization, the nature of relationship with the GPP, etc. · The effectiveness of the policy and legal reform work of global programs depends to a large extent on effective follow-up on the recommendations as well as coordination with donors to ensure consistency with the ongoing broader sector policy dialogue. As in the case of CGAP, close coordination with donors such as the World Bank, especially those actively working on policy reforms, both for follow-up and consistency, contributes to the effectiveness of the policy work of global programs. · Locating a global program in the World Bank can play a positive role in its development. However, the World Bank and its partners need to reassess the World Bank's long-term role in each program from time to time. In the case of CGAP, there is also the possibility of it getting too closely involved with the internal management processes of the World Bank, which could generate perceptions of unfairness and inequity on the part of other CGAP members. 1 1. Program Objectives, Activities, Governance, and Costs Objectives and Activities 1.1 The original concept of Consultative Group for Assisting the Poor1 (CGAP) was presented by the World Bank President at the International Conference on Actions to Reduce Global Hunger, held in December 1993, where he announced the World Bank's willingness to join with other donors to explore ways of systematically increasing the resources available to the very poor. Following this, CGAP was established in 1995 as a consortium of nine public agencies with the objective of expanding access to financial services for the poor. The World Bank provided funding of $30 million from its Special Grants Program for a three-year period and also agreed to act as the overall administrator and to host the CGAP secretariat inside the World Bank. After a mid-course review in 1998, the Council of Governors extended CGAP for another five years (Phase II, FY99­03). Since then the Council of Governors has approved two more extensions, Phase III (FY04­08) and a strategic plan for FY09­13.2 These extensions took into account the findings of two separate independent evaluations in 2002 and 2007, respectively. The World Bank has supported CGAP in its extended phases through grants from DGF. This review covers CGAP's activities and performance during Phase III.3 1.2 Over the years, while CGAP's basic mission of helping to develop a sustainable microfinance industry has remained unchanged, its strategic priorities and focus of activities have evolved significantly, reflecting the increasing maturity and experience of CGAP, and the rapidly changing environment in the microfinance sector and therefore the evolving needs of the market (Figure 1). In Phase I, its activities were geared to developing sustainable MFIs and the necessary supporting technical tools. In Phase II, the emphasis shifted to capacity building and enhancing transparency of MFIs. Phase III focused on diversity and integration, while the strategic plan for FY09­13 will promote "equity" and "efficiency" in the access of the poor to finance. 1.3 As defined by its Charter, CGAP's mission in Phase III was to "help build financial systems that work for the poor, providing large numbers of people with diverse financial services through a wide range of organizations." Its activities focused on the following five strategic priorities:4 Promoting institutional diversity Promoting diverse financial services to a broad range of clients Building financial market infrastructure Fostering sound policy and legal frameworks Improving the effectiveness of microfinance funding 1. This was the original name of the program. Subsequently, the word "Poorest" was changed to "Poor" in order to avoid definitional problems. 2. While approving the strategy for FY09­13, the Council of Governors decided to remove the notion of phases, ostensibly so that CGAP would be viewed as an ongoing operation. 3. CGAP's activities and performance in Phase II were covered by an IEG (then OED) case study of CGAP, which was completed in 2004. 4. "CGAP Phase III Strategy, 2003-08", January 2003. 2 Figure 1. CGAP's Broad Themes, Phases I­IV Source: CGAP, CGAP Phase III Strategy, 2003­2008, January 2003. 1.4 Since its inception, the specific objectives of CGAP have been to help build the field of microfinance (or access to finance), to generate and disseminate knowledge, to develop a common language, to catalyze the movement toward good-practice performance standards, and to build consensus among its many and varied stakeholders. CGAP's generic activities have consisted of (a) providing advisory services, (b) developing and setting standards, (c) advancing knowledge (including sector intelligence and information sharing), and (d) offering training and capacity building services. The outputs of these activities have been disseminated widely through a strong communications strategy including through online information, publications, seminars and conferences. CGAP provides its knowledge and advisory services to a variety of stakeholders, including donors, MFIs, industry as a whole, etc. CGAP leverages its limited resources by entering into discrete partnership arrangements with a variety of stakeholders.5 Governance 1.5 CGAP is a consortium of public and private agencies whose secretariat is located in the World Bank. Trust fund agreements delegate to the World Bank the role of administrative and fiduciary oversight and execution of CGAP.6 In 2002, the Council of Governors adopted a Charter outlining CGAP's mission and governance structure. Since then it has been amended four times to refine the provisions and to conform to the changing industry 5. CGAP employs a range of partnership arrangements, broadly defined. These partnerships fall into six categories: (a) CGAP provides grant to an external organization; (b) CGAP and donors or service providers are co-funders, co- organizers or collaborators on a project on a fully equal basis; (c) CGAP initiates and leads an effort/activity, inviting other partners to join; (d) External partners initiate and lead an effort/activity, inviting CGAP to join; (e) CGAP works at the request of a donor/other organization to provide inputs; and (f) CGAP contracts a service provider. 6. The oversight function performed by the World Bank relates to ensuring that CGAP operates within the policy framework laid out by the Council of Governors and consistent with the terms of trust fund agreements with CGAP members. 3 environment. According to the Charter, CGAP membership is "open to funding organizations with mandates to promote public goods".7 The governance structure consists of four parts: the Council of Governors, the Executive Committee (EXCOM), the Investment Committee, and the Operational Team (Figure 2). Figure 2. CGAP Organizational Chart Council of Governors Executive Committee Investment Operational Team Committee 1.6 The Council of Governors consists of representatives of all 31 CGAP members, with the World Bank's Vice President, Financial & Private Sector Development (FPDVP) as its chair. It meets once a year and approves critical policy decisions and provides overall guidance on strategy. It also amends CGAP Charter, confirms the appointment of the Director/CEO, elects members of EXCOM, provides input to the annual work plan, budgets and annual report and adopts CGAP consensus documents. During Phase III, EXCOM comprised nine members, four representing each of the Council of Governors and the microfinance industry, respectively, and one representing the World Bank. The CEO of CGAP was also an ex-officio member. For purposes of electing EXCOM members from CGAP members, CGAP membership was divided into four constituency groups. However, in July 2008, the number of constituencies increased to five with the World Bank included in one of them. Thus, the World Bank will no longer have a reserved seat on EXCOM. The four industry representatives are elected by the Council of Governors. Each EXCOM member is elected for a two-year term, renewable for another term; and the chair is elected by members and rotated every four years. EXCOM's responsibilities include providing operational guidance to the Operational Team, approving new members, work plans and budgets after consultation with the Council of Governors and proposing amendments to the Charter, when necessary. EXCOM meets 3­4 times a year. All elected EXCOM members serve for a term of two years extendable for up to another two years. 1.7 In exercising its fiduciary role under bilateral trust fund agreements, the World Bank designates the Investment Committee, which was composed of seven to nine members during Phase III, including the FPD Vice President as its Chair. All Investment Committee members were senior World Bank group staff. At least one member was required to be from the World Bank's Legal Vice Presidency. Two official observers who were not World Bank staff, each nominated by the Council of Governors and the EXCOM, were invited to participate in the Investment Committee meetings. As the result of an amendment to CGAP's Charter, the 7. CGAP Charter, as amended in June 2007. 4 Investment Committee's strength will increase to 9­14 including the Chair, starting July 2008, to allow for the attendance of a reasonable number of members who are all senior World Bank Group staff and travel frequently. The status of the representatives from the Council of Governors and the EXCOM was also elevated from "observer" to "full voting member". The Council of Governors' Chair (FPDVP) also chairs Investment Committee meetings although this is not required by Charter. The Investment Committee approves grants and initiatives costing more than $100,000, but those between $100,000 and $250,000 are approved on a no-objection basis. 1.8 The CEO is selected by the World Bank, endorsed by the Council of Governors, and heads the Operational Team. So far, the Council of Governors has endorsed without objection the persons selected by the World Bank, though there have been only two such appointments. Currently, the Operational Team consists of 41 in-house staff comprising 19 operational staff and 18 consultants and 4 administrative staff. They are all recruited by the CEO and employed by the World Bank. A majority of staff is located at CGAP headquarter in Washington DC while 7 are in the representative office in Paris. Funding and Costs 1.9 CGAP's funding requirements are provided by its members.8 Up to the end of Phase III, members had the option of either making a core cash contribution or providing funds for specific CGAP activities, called "designated contributions". As of January 2008, the total member contributions since the inception of CGAP stood at $171 million, comprising $144 million (84 percent) of core contributions and $28 million of designated contributions.9 The World Bank has been the largest single contributor of funds with 46 percent in cumulative contributions. However, over the years, the World Bank's share has been declining steadily from 81 percent in Phase I to 57 percent in Phase II and 18 percent in Phase III (Figure 3). This is due partly to larger contributions from other members and partly to a decision by the World Bank to reduce its contribution between FY2006­08 following a one-time interest income accrual to CGAP on the unused balances of the World Bank's contributions.10 1.10 After the World Bank, the largest contributor has been the Bill and Melinda Gates Foundation with $19 million, thanks to its designated contribution for technology work. By category of members, bilateral institutions contributed $61 million, private foundations $20 million, multilaterals (excluding the World Bank) $9 million, and development finance institutions $2 million. 1.11 In Phase III (up to FY07), the average annual cost of CGAP's operations was $18 million for a total of $71.9 million. These costs were fully covered by the available resources, which on a commitment basis amounted to $81.8 million, leaving an adequate cushion of available reserves (Annex E). 8. The other relatively minor sources are interest income and foreign exchange gains. 9. The size of designated contributions jumped sharply in the last two years of Phase III as a result of a large contribution ($19 million) from Melinda & Bill Gates Foundation for technology related activities. Before this, such contributions were roughly about 5 percent of total contributions. 10. In FY05, the World Bank's Legal Department determined that that the interest income on unused trust fund balances should belong to CGAP. 5 Figure 3. The World Bank's Share in Total CGAP Funding US$ Millions 200 180 160 Other Members 140 World Bank 120 100 80 60 40 20 0 Phase I Phase II Phase III Cumulative (1995-98) (1999-03) (1994-08) (1995-08) Source: CGAP internal documents. 2. The External Evaluation of CGAP Scope, Approach, and Process 2.1 In mid 2006, the Council of Governors began the planned process of commissioning an independent evaluation of CGAP with a view to facilitating its decision on the future of CGAP after the end of its Phase III in June 2008. This decision also complied with the DGF requirement of an independent evaluation every 3­5 years. This evaluation was completed by a team of external consultants. Similar evaluations had also been conducted earlier in 1997 and 2001. 2.2 The purpose of the 2007 Evaluation was three-fold: (a) To assess CGAP's performance, so far, under Phase III in relation to its current five priority objectives; (b) To glean lessons learned and areas for improvement over the remainder of Phase III; (c) To serve as an input to a broader reflection among CGAP's members about the need and future directions for CGAP or a similar entity. 2.3 The evaluation covered the entire CGAP system (the Council of Governors, EXCOM, the Investment Committee and the Operational Team) and used the standard evaluation criteria of relevance, efficacy, efficiency and accountability.11 The methodology of the evaluation consisted of review of relevant documents, interviews of a number of relevant stakeholders, selective participation in some CGAP processes and a client survey. A web- 11. Sarah Foster, Klaus Maurer, Michael Mithika, CGAP Phase II Mid-term Evaluation (July 2003-June 2006), March 16, 2007. 6 based survey, conducted by an independent market research firm, provided the views of a wider circle of CGAP stakeholders, clients and the microfinance industry at large by asking questions related to the overall context of microfinance as well as CGAP strategy, performance, role, identity and communication with clients.12 The survey response was credible as it represented a wide array of stakeholders in almost all regions of the world. This response was complemented by 106 interviews of a representative variety of stakeholders conducted by the evaluation team. Therefore, the 2007 Evaluation was credible and comprehensive. Its scope was wide enough to cover all the relevant areas and the methodology was consistent with the IEG/DAC Sourcebook.13 2.4 The process of the evaluation was managed by EXCOM. It approved the TOR, made the final selection of the team of consultants including their remuneration,14 monitored the progress of the work and reviewed and extensively commented on the interim and final draft reports. A sub-committee of EXCOM members was formed to oversee the evaluation work. The role of the Operational Team role was largely to support EXCOM and to provide administrative and professional assistance to the consultants. The Council of Governors was indirectly involved in this process. They recommended names of potential consultants and provided comments to EXCOM on the draft TOR as well as on the final draft report. 2.5 The selection of consultants was made through a consultative process. Based on the recommendations of the Council of Governors, EXCOM prepared a short list and made a final selection after an in-depth assessment of the three individual consultants who were invited to form a team. The team leader was appointed by EXCOM from among the three selected individuals. The team selection was based on each member's professional background and experience, their knowledge of microfinance sector work and their suitability for the job. The team leader was associated with the previous two external evaluations of CGAP as a team member, while another person had done evaluation of some of the activities of CGAP. The third person had no prior direct CGAP experience but was chosen given his knowledge and experience of the microfinance sector, particularly in Africa. Overall, the evaluation team had the necessary complementary and relevant expertise and background suitable for the task of evaluating CGAP. It could be argued that EXCOM's decision to unilaterally form a team was risky as this could have resulted in the disruption of the work due to incompatibility among team members since they had never worked as a team before. Fortunately, the team worked harmoniously and this risk did not materialize. Independence and Quality 2.6 The present review concludes that the 2007 Evaluation was appropriately impartial, free from conflicts of interests and both organizationally and behaviorally independent. The members of the evaluation team did not have any previous compromising commercial relations with CGAP. Their prior relationship with CGAP was entirely for evaluation work. The team was 12. Survey questionnaire was sent out to CGAP's entire contact database of about 5000 stakeholders and was also posted on the Web site of CGAP as well as three other industry websites. 13. IEG, Sourcebook for Evaluating Global and regional Partnership Programs: Indicative Principles and Standards, 2007. 14. The total cost of the 2007 E valuation was $132,425 or 0.7 percent of the total expenditures in FY07. 7 given sufficient freedom in its work for which EXCOM made available adequate financial and administrative resources, as confirmed by the evaluation team in its discussion with IEG. The evaluation team had unrestricted access to all CGAP documents, processes and staff. Also, the TOR did not place any constraint on the team's work. Moreover, an open and transparent review process was used for the review of the interim findings and draft final output of the evaluation. As stated above, the review process was managed by EXCOM, which obtained inputs from members of the Council of Governors as well as Operational Team. The review of the evaluation process shows that the comments on draft reports raised substantive issues and provided the perspectives of the commentators on issues, but the incorporation of these comments was left entirely to the judgment of the evaluation team. Finally, throughout this process the role of the Operational Team role was supportive without being in a position to dominate the outcome. 2.7 On the whole the quality of the evaluation was satisfactory. The evaluation report was candid both in terms of praise and criticism of CGAP. Generally, it covered all the tasks specified in the TOR. In their discussions with IEG members of the Council of Governors generally expressed the view that the evaluation was comprehensive, candid and useful in terms of identifying CGAP's strengths and weaknesses and facilitating planning for the future of CGAP. 2.8 The breadth the 2007 Evaluation complied with all the requirements of the TOR, and in some areas went well beyond. For example, it included a chapter containing useful observations on the future of CGAP beyond 2008, which was not a requirement of the TOR. The evaluation report emphasized the need for CGAP to put in place a system to monitor and report on the extent to which CGAP's activities and initiatives contribute to achieving its intended outcomes and priority objectives, including its overarching objective of building inclusive financial systems. The 2007 Evaluation used anecdotal evidence to overcome the absence of such a system. However, the assessment of CGAP's efficacy could have been more convincing if the evaluation had compiled information on intermediate outcomes and CGAP's contribution to achieving them.15 The analyses underpinning the findings of the evaluation report were logical and appropriate, with one exception. The discussion on the relevance of CGAP could have made a more forceful and direct case for CGAP as global platform and forum for the microfinance industry. On the whole, this GPR found the 2007 Evaluation a reasonable basis to judge the performance of CGAP. 2.9 Overall, the findings of 2007 Evaluation were extremely positive towards CGAP. The principal findings of the evaluation are summarized below: · Overall. CGAP is a powerful and pivotal force in the field, which is playing a critical role in helping others to build inclusive financial systems. It produces high quality and value products and provides good value for money. · Relevance. CGAP's work has continued to be of central relevance to the development of microfinance during Phase III and regarded as having more 15. Some possible examples of intermediate outcomes are: (a) the degree of diversity of institutions, services and clients; (b) the increased transparency such as numbers and shares of MFIs with audited financial statements, number of countries requiring certain disclosure rules for MFIs and number and shares of MFIs rated by well-recognized agencies; (c) the number of countries where CGAP has been involved that have improved specific policies and regulations; and (d) the improved donor practices. 8 legitimacy and being more readily heard than any other organization in the field of microfinance. · CGAP's strategy. Overall, CGAP's strategy is clear, coherent and relevant to the global challenges facing microfinance today; and it has increased CGAP's relevance to and influence over mainstream financial institutions and provided an opportunity to place tackling financial inclusion at the top of the mainstream financial sector agenda. · CGAP's comparative advantage. CGAP was born with certain comparative advantages. Over the past ten years, CGAP has developed many "second generation" comparative advantages. · Efficacy. With a few exceptions, CGAP is on-track to produce almost all expected outputs by the end of Phase III and in some cases will exceed original expectations. · Accountability. CGAP has adequate governance and management systems that are transparent and thorough in providing information about the program. · Efficiency. CGAP is a cost-effective and competitive organization and it compares favorably against the efficiency metrics of similar organizations. 2.10 The 2007 Evaluation made a number of recommendations mostly on specific aspects of CGAP's initiatives and activities. CGAP management has positively responded to almost all recommendations by taking appropriate actions (Table 1). Table 1: Evaluation Recommendations and Management Response Evaluation Recommendations Actions Taken by Management Distill and synthesize lessons from the Retail Advisory An evaluation of the RAS is underway. Service (RAS). A toolkit of lessons learned is planned in FY08. Provide recognition of other microfinance service A review of state banks is being conducted. providers (state banks and cooperatives), which play a CGAP completed and published the study on India's dominant role in many parts of the world. Self-Help Groups. Commit resources to translate the money transfers, The graduation project already occupies center stage savings, PPIC and graduation work into action, and in CGAP's efforts to transition the poorest into bring the social performance work to center stage. microfinance and being significantly scaled up. The savings work has been integrated into CGAP initiatives and in its advocacy vis-à-vis governments, financial institutions, and funders. CGAP is documenting learning from the PPIC with recommendations for future grant programs. Send a strong message to IFIs and DFIs about their Already a part of CGAP's strategy. role in helping formal financial institutions to expand savings services for the poor. Highlight the contradictions between IFIs policies and the consequences of their credit-focused projects. Run the annual Financial Transparency Awards Done. through to the end of CGAP Phase III Expand CGAP's Information System Fund. In FY08, CGAP is co-funding the costs of IS consultants for 20 MFIs. 9 Evaluation Recommendations Actions Taken by Management Continue to support the Microfinance Information CGAP will continue to chair MIX board, provide eXchange (MIX). strategic guidance, and improve synergies between MIX and CGAP. Work with the Investment Funds to improve their CGAP has developed performance benchmarks for transparency, reporting and linkages to the market. microfinance investment funds. Continues to gather market intelligence on investment funds. Utilize the close link to the World Bank and engage in CGAP's policy work is closely coordinated with more systematic working with the World Bank and the WB/IMF. IMF as the major organizations involved in financial CGAP is engaged in an ongoing collaboration with the sector policy work. World Bank's Financial Integrity Unit to assure harmonized messages on the impact on financial access of measures to combat money laundering and terrorist financing Increase CGAP's presence at mainstream policy CGAP regularly participates in international policy forums (IMF/WB annual meetings, Basle Committee, forums. World Economic Forum etc) and make the case for At the invitation of the World Bank CGAP just building inclusive financial systems. commenced a year-long project with members of the Basel Committee's International Liaison Committee to provide technical input on a paper clarifying application of the Revised Basel Core Principles to microfinance. Develop clear and consistent policy messages. CGAP's "Guiding Principles on Regulation and Supervision of Microfinance" (2003) are written in plain, non-technical language and are increasingly accepted and applied by the policy and regulatory community in countries around the world. Plans are in development for a 2nd edition to be drafted in 2008. Involve the members of the Council of Governors more The members of the Council of Governors are actively actively in policy work/country diagnostics. involved in donor policy coordination at the regional and country level. Virtually all CGAP-sponsored country-level policy diagnostic work is conducted in partnership with one or more Council of Governors members. Carry out a mapping exercise of donor comparative In 2007, mappings of donor and investors' activities in advantages and activities on a country level Central Asia, Africa and MENA were completed. CGAP developed plans to consolidate and harmonize all of its surveys on funders' microfinance programs and investments to be completed in 2008. Following internal discussions, CGAP determined that mapping comparative advantages for all donors could not be usefully well done. Rather, initiatives such as the SmartAid Index for Microfinance provide a sense of the strengths and weaknesses of individual donors. Also, the above-mentioned comprehensive CGAP Microfinance Funder Survey collects information on donor activities at the country level. Tie in aid effectiveness work more systematically to This is already happening, especially with the equity, content areas. policy and technology teams. Consider whether and how best to work with other The FY08 work plan includes stepped up engagement funders -- apexes, investment funds -- and develop a with development finance institutions (DFIs). work plan for the remainder of Phase III. Microfinance investment vehicles and greater outreach to apexes. Source: Document provided by CGAP Secretariat to IEG and IEG's discussions with CGAP Secretariat. 10 3. The Effectiveness of CGAP Relevance 3.1 Supply-side Relevance: CGAP's basic mission has been to help develop a sustainable microfinance industry. It was established in response to a broad international consensus that the availability of microfinance helps to reduce poverty. This consensus has since been reaffirmed in a number of international forums such as (a) the UN General Assembly's designation of 2005 as the Year of Microcredit; and (b) the awarding of the 2006 Nobel Peace prize to Dr. Muhammad Yunus for his work on the development of microfinance.16 In addition, the UN Millennium Project17 (2005) emphasized in its recommendations the important role of access to microfinance in catalyzing the achievement of the MDGs.18 Other international endorsements of the important role played by microfinance include (a) the Microcredit Summit, held in Washington DC in 1997, of over 2800 world leaders, practitioners and international organizations (such as USAID, UNDP and the World Bank); (b) the G8 Action Plan of 200419 and the Declaration of G8 Parliamentarians of 2005;20 (c) the Brussels Programme of Action, adopted by the Third United Nations Conference on the Least Developed Countries (2001); and (d) the report of the Commission on Private Sector Development convened by the UN Secretary-General (1994).21 Therefore, CGAP's mission continues to enjoy broad international consensus. 3.2 The CGAP Client Survey conducted as part of the 2007 Evaluation also found that an overwhelming majority of respondents (85 percent) viewed CGAP's contributions to microfinance as very important to the success of microfinance globally. Over 80 percent agreed that CGAP had a clearly defined mission and it adapted its priorities to respond to emerging trends. This feedback was fairly consistent among the various groups of respondents, including governments, policy makers, donors, MFIs, etc. Many members of CGAP's Council of Governors in their interviews with IEG also confirmed this view. The more than four-fold increase in CGAP membership since 1995 is another strong indication of international recognition of CGAP as the leader of microfinance industry. 3.3 CGAP's objectives and operations are consistent with the World Bank Group's Financial Sector Strategy (2007).22 This strategy pays special attention to improving access to finance for the under-served and highlights the role of CGAP in this respect. It identifies focus areas for CGAP including "sound policies and best practices, with an increasing 16. The Grameen Bank was the co-winner of the 2006 Nobel Peace Prize. 17. The United Nations Millennium Project is an independent advisory body commissioned by the UN Secretary-General. 18. United Nations Capital Development Fund, Microfinance and the Millennium Development Goals, New York, 2005. 19. G8, G8 Action Plan: Applying the Power of Entrepreneurship to the Eradication of Poverty, 2004. 20. G8, Edinburgh Declaration, June 2005. 21. The Commission was co-chaired by Canada's Prime Minister Paul Martin and Mexico's former President, Ernessto Zedillo. 22. World Bank, Financial Sector Strategy For the World Bank Group, March 2007. 11 emphasis on the regulatory and market development implications of the use of modern technologies." These areas form a core part of CGAP's operational focus. The World Bank as well as the International Monetary Fund recognizes CGAP's unique expertise as they often invite CGAP to participate in the Financial Sector Assessment Program (FSAP) to assess the areas relating to access to finance.23 3.4 Demand-side Relevance. The demand for microfinance in the developing countries has been rising rapidly. The number of MFIs, the overwhelming majority of which operates in developing countries, has increased dramatically in the last decade. Currently, over 1000 MFIs report to the MIX, the leading source of microfinance business information created by CGAP. In addition, over 80 private investment funds have also emerged with over $2 billion in outstanding investment, almost all in developing countries. Some of these countries are also in the forefront of discovering innovative and cost-effective ways of delivering microfinance services to the poor. Five countries are already using cellular technology to deliver financial services to the poor.24 Indeed, support to microfinance is high on the development agenda of the governments of many countries. 3.5 Microfinance encompasses not only credit but also other financial services such as savings facilities, insurance, transfer payments, and even micro-pensions. The poor use financial services not only for business investment in their microenterprises but also to invest in health and education, to manage household emergencies, and to meet the wide variety of other cash needs that they encounter. A number of research studies of microfinance clients around the world have found that access to financial services enables poor people to increase their household incomes, build assets, and reduce their vulnerability to the crises. These studies have also found that access to financial services translates into better nutrition and improved health outcomes and more poor children going to school. It has made women clients more confident, assertive and better able to confront gender inequities. Finally, a major argument used to prove the beneficial effects of microcredit programs is the impressive record of repayments of most such programs. If the poor were not benefiting they would not be able to sustain such impressive repayment performance over a long period of time. 3.6 A plethora of studies have concluded that enhancing the availability of microfinance is an effective tool to reduce poverty and help poor people manage their vulnerabilities. According to the United Nations, despite some gains, one fifth of the population of the developing countries (about one billion people) still live in extreme poverty.25 The magnitude of such poverty remains a major challenge for the development community, and underpins both the importance of microfinance and the relevance of CGAP. 3.7 However, a number of research studies have questioned, on methodological grounds, the professed positive correlation between microfinance and poverty alleviation. In particular, a recent research report by the World Bank found that statistical research evidence 23. So far, CGAP has participated in only four FSAPs (5 percent of total FSAPs done in FY04-07) although it was invited a much greater number of times. Because of its resource constraint CGAP has been very selective with respect to its FSAP work. 24. The countries are Brazil, India, South Africa, Kenya and the Philippines. 25. United Nations, The Millennium Development Goals Report, New York, 2007. 12 on the benefits of microcredit is not yet overwhelming and made a case for further research to obtain more robust conclusions.26 But this study also conceded that individual or household welfare is difficult to measure, which biases microanalysis against finding a positive effect of access to credit. The reservations in the latter study also apply only to the impact of microcredit, not to other elements of microfinance (such as savings, insurance, and payment facilities). In fact, the study recognized the benefits of other financial services for the poor. It pointed out that various research suggests that access to financial services is an important direct or indirect contributor to the achievement of most of MDGs. Moreover, it argued that "improving access for small firms and for the nonpoor entrepreneurial households can be a powerful mechanism for helping the poor."27 Therefore, on the whole, there is much evidence of microfinance being an effective tool to alleviate poverty. 3.8 Since 1995 when CGAP was created, the microfinance industry has undergone a dramatic transformation. Not only there are a large number of successful MFIs all over the world, but also microfinance has increasingly become integrated in the formal financial system, as more commercial banks and other commercial retail institutions take up microfinance as a business line and new types of service providers with a wider range of financial services are entering the market. A variety of new funders have entered the field with large sums raised from the capital market. An increasing number of market players are recognizing the importance of transparency, professional skills and standardized accounting in the sector. CGAP has played a pivotal role in guiding the development of the microfinance industry along sound lines through a variety of programs and activities discussed in the Section on "Efficacy" (paras. 3.16­3.31). Most of CGAP's outputs were of a public good nature. 3.9 However, the microfinance industry has not yet reached a stage of universal maturity as it continues to have a number of gaps and constraints. Impressive developments in microfinance are not yet spread over all -- or even a majority -- of developing countries. Many countries still do not have high-quality financial services and for those who have there is limited competition restricting the scope for enhancing the quality of service in terms of delivery, pricing and outreach. Service expansion is also constrained by client remoteness in many countries. The market infrastructure for microfinance remains under-developed due to the absence of such critical agents as rating agencies, payment systems, credit bureaus, etc. In addition, the expansion of microfinance services is constrained by policy and regulatory regimes not always friendly to microfinance and also by the absence of local debt and equity markets. Therefore, there is a large agenda for the future development of the of the microfinance industry. 3.10 Subsidiarity. Overwhelmingly, CGAP's activities are of global nature aimed at benefiting the microfinance industry as a whole, with the share of such activities having increased from 60 percent in 2003 to 94 percent in 2007. This shift in focus reflects, appropriately, the trend in the development of the microfinance industry and the evolution in CGAP's strategic priorities. CGAP's business strategy has consistently focused on issues that 26. World Bank, 2008. Finance for All: Policies and Pitfalls in Expanding Access, a Policy Research Report. Washington DC: World Bank. 27. Implicitly, the reference to the "nonpoor" is relative, meaning not "the poorest". In the microfinance context "poor" does not mean the "poorest". 13 were not being addressed by its partners or any other active player in the industry. Consequently, CGAP has gradually pulled out of work at the national level, where donors (including the World Bank) and other fund providers have stronger presence than CGAP. However, while the focus of its activities is predominantly global, CGAP has been cognizant of the regional/country-level differences in the needs for microfinance services. This regional/country link is provided by CGAP's regional representatives in North Africa, West Africa, Anglophone Africa, Central Asia and South Asia. In addition, CGAP maintains country-level activities through pilot programs such as the technology and graduation programs, and its policy work at the country level. 3.11 Horizontal Relevance. There is no other global program/entity competing with CGAP at the present time. While there are many other stakeholders in the microfinance industry who contribute to the development of the microfinance industry, their work is understandably guided by their own commercial and/or institutional interests. They have neither the incentive nor capacity to devote resources to producing public goods. Although public sector donors are more suited to providing public goods, they lack a critical mass of expertise and have other competing demands on their resources. Therefore, it has been more efficient for donors to support CGAP either by delegating tasks to CGAP or by working in partnership with CGAP. Both this broad donor support and the quality of its outputs have enabled CGAP to position itself as the recognized leader and voice of the microfinance industry. For example, the G8 Action Plan of 2004 asserted that "G8 countries will work with the World Bank-based Consultative Group to Assist the Poor (CGAP) to launch a global market-based microfinance initiative." In the last two years alone, there have been more that 500 press citations quoting CGAP. CGAP is invited to practically all international conferences on microfinance and access to finance. 3.12 CGAP has played a prominent role in shaping and defining the future development of the microfinance industry. In particular, its role has been pivotal in addressing issues of common good, including looking after the interests of users of microfinance services. Specifically, CGAP is needed to focus on the following areas: · Protecting the Interest of the Poorest. Increasing participation of commercial institutions in the microfinance business entails the risk that they will target services to the upper end of the poor leaving out the lower end of the spectrum of the poor. Therefore, there is a need for a centralized effort to find ways of protecting the consumers' interest and undertake research to develop innovative approaches and experimentations to serve the poorer of the poor and push outwards the frontiers that limit the microfinance industry at present. This includes ensuring that funding goes not only to the most attractive markets and market segments, but that patient capital is available for the long work of building capacity. The examples of such work include CGAP's graduation program and technology work. · Exploring New Frontiers. Despite a rapid growth in the microfinance industry, financial services are still not accessible to large segment of the poor population. According to one crude measurement, over 80 percent of the household in many Sub- Sahara African countries do not have bank accounts. The same ratio in Asian countries ranges between 40­60 percent. A dramatic expansion in accessible financial 14 services to the poor requires considerable research to develop innovative and cost- efficient delivery mechanisms, a major activity of CGAP.. · Setting Standards. In many ways, the microfinance industry is still evolving. It needs credible standards for institutional development as well as performance measurement. This challenging task requires a common and credible platform. The examples of areas where standard setting are important include disclosure of loan costs and terms, financial performance of deposit-taking institutions, consumer protection principles and codes of conduct governing business practices. · Contributing to Policy Reforms. In many countries, government policies and regulations hinder the development of microfinance. Where enabling policies and regulation have been adopted, capacity to implement them effectively is often weak. Thus, there is a role for an entity like CGAP to develop specialized knowledge and expertise in this area and provide support and guidance to governments and regulators. · Providing Market Intelligence. A rapid and sound growth of the microfinance sector is facilitated greatly by the availability of relevant information and analysis on the status, performance and trends in the industry in areas such as funding supply, demand, flows, poverty impact and client outreach. An important part of this intelligence is the analysis and dissemination of the cross-country experiences. This service is needed not only by market agents but also by policy makers in governments and donors who play a critical role in the development of the microfinance sector. · A Common Forum for Dialogue. A common forum is needed to discuss and exchange views on the crosscutting issues/topics of common interest as well as to develop appropriate advocacy strategies to protect and project the interest of the industry. This is especially true as the number of actors working in microfinance multiplies, often bringing quite divergent views and approaches. CGAP has gained sufficient credibility to serve as an effective body where issues can be debated. 3.13 CGAP is well-suited to play these roles. Exploiting its comparative advantages, it has established itself as a recognized leader of the microfinance industry. As the 2007 Evaluation has noted, CGAP was born with four distinct comparative advantages derived from its close association with the World Bank: convening power, credibility, global perspective and its links to the World Bank. Over the years through a variety of its work CGAP has developed new comparative advantages such as credibility of its brand, ability to attract high quality staff, cutting-edge communication skills and recognized objectivity/neutrality. These comparative advantages are fully reflected in its work and its standing in the industry. CGAP has served as an objective convening platform and is recognized as a knowledge resource center for the industry. CGAP is also widely perceived as a standard setter for microfinance, a global network for microfinance, and a think tank/research institute. 3.14 Relevance of Design. CGAP's design is appropriate to address its objective of promoting microfinance industry in developing countries so as to make microfinance services available to an increasing number of poor. Its extensive access to the donor community and other major stakeholders enables it to form partnerships for specific tasks 15 that result in building consensus as well as sharing of costs. From the very beginning CGAP has followed a business model that is based on constant adaptability to changing requirements of the microfinance industry. Thus, while in Phases I and II working with MFIs dominated its activities, in Phase III, when the industry's requirements changed, the emphasis shifted to broader issues related to developing inclusive financial systems, including developing more innovative and cost-efficient mechanisms for the delivery of financial services, more effective delivery of assistance by donors, policy reforms, etc. CGAP has remained a relatively lean organization, which was made possible by partnership arrangements. In Phases II and III, there was a significant increase in the range and types of CGAP's activities, partly due to the overlapping of the activities directed at MFIs and those for the broader development of financial sector. However, the MFI-related activities, mainly institution building grants, will be largely phased out in FY08, prior to the beginning of the strategy for the next five years. 3.15 The 2007 Evaluation has pointed out, as was done earlier by the 2004 IEG Case Study,28 that CGAP's monitoring and reporting systems are well-developed at the level of measuring outputs of its activities, but not at the level of measuring their outcome or impact. While there are many anecdotal evidences to indicate the positive outcomes of CGAP, there is no systematic methodology used by CGAP yet to provide a convincing conclusion either in the form of intermediate or final outcomes. Therefore, the 2007 Evaluation recommended that "with regard to measuring, monitoring and evaluating outcomes, more needs to be done" and further that "CGAP should develop a concept for measuring its own overarching key objective, i.e., building inclusive financial systems". As a broad objective this recommendation is appropriate, but the monitoring and reporting should enable CGAP to measure its impact at the level of specific activities as well as CGAP as a whole. Outcome and impact assessment at the activity level is necessary: (a) to serve as a necessary planning tool; (b) to add transparency to the selection of CGAP's specific activities and initiatives; and (c) for members of the Council of Governors to decide the future directions of CGAP. Indeed, the nature of CGAP's operations is such that it makes the outcome and impact assessment quite complicated and difficult, more so because CGAP acts as a catalyst for change through advocacy and the actual outcomes are achieved through other actors. Yet this kind of exercise is needed to provide a basis for forming a definitive judgment on the real value added of CGAP. An encouraging development in this respect is that CGAP management has started work to identify ways of measuring impact at both an overall and activity level. Efficacy 3.16 CGAP's activities in Phase III related to its five strategic priorities, namely, (a) promoting institutional diversity, (b) promoting diverse financial services to a broad range of clients, (c) building financial market infrastructure, (d) fostering sound policy and legal frameworks, and (e) improving the effectiveness of microfinance funding.29 Of these, activities relating to two priorities (building financial market infrastructure and fostering 28. IEG, Consultative Group to Assist the Poor: Case Study, 2004. 29. While the five strategic priorities have remained unchanged during Phase III, their scope has been evolving and with that the description of each priority has also changed. This review uses the latest description from CGAP's 2007 Annual Report. 16 sound policy and legal frameworks) accounted for over 75 percent of the total amount of money spent by CGAP in Phase III (Figure 4). Figure 4. CGAP Expenditures on Five Strategic Priorities, FY2004­07 (Phase III) Building Financial Market Infrastructure Fostering Sound Policy & legal Framework Promoting Diverse Financial Services Promoting Institutional Diversity Improving the Effectiveness of Microfinance Funding 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 US$ Millions Source: Annex E. CGAP: Sources and Uses of Funds 3.17 The 2007 Evaluation report assessed the efficacy of CGAP's activities in the first three years of Phase III. This review takes into account the outputs of FY07 and forms a judgment on the degree of CGAP's achievements in meeting its operational targets under each of its strategic priorities. In terms of outputs, CGAP's performance in Phase III can be characterized as impressive (Table 2). However, judgments on outcomes are constrained by the fact that CGAP's monitoring and reporting system provides comprehensive information on outputs, but little on the outcomes or impacts. Therefore, while the nature of CGAP's activities and outputs is consistent with its stated objectives, there is not strong systematic evidence that its intended outcomes have been achieved, or that achievements can be attributed to CGAP's activities. 3.18 A notable feature of CGAP's activities and outputs in Phase III has been the phase- out of its grant-based programs for institution building of MFIs and Networks,30 since these no longer reflected CGAP's current strategic priorities. Accordingly CGAP did not commit any new grants for MFI institution-building in Phase III, or for Networks after FY05.31 Some of CGAP's programs still involved grants in Phase III, but their objectives were broader than 30. In the context of CGAP. "networks" are organizations that encompass a number of institutional partners with interests in the microfinance business. These serve the interests of their partner organizations in various ways such as launching new institutions, developing standards, providing technical assistance, implementing knowledge management and leading policy reforms. CGAP has supported institutional building of a number of networks. 31. The scale of new grants for Networks in Phase III was considerably lower, a total of $890,000 compared $5.4 million in Phase II. 17 Table 2. CGAP: Major Planned and Actual Outputs Planned Industry Objectives Actual Output I. Promoting Institutional Diversity 1. Publish updated appraisal 1. Variety of financial institutions 1. Tool on appraisal for microfinance and evaluation framework for tool for microfinance and the with different objectives, networks finalized and published. new evaluation framework for methodologies, and services will 2. Joint Appraisal & Monitoring Program discontinued in 2004 in view of networks. serve the poor. lack of demand. 2. Under Joint Appraisal & 2. Scaling up through use or 3. Completed 6 analysis/assessments of different types of institutional Monitoring Program, coordinate restructuring of existing models of financial institutions; and seventh study (state development 3­5 joint appraisals and funding infrastructure of large-scale banks) is in advance stage. consortiums per year of MFIs institutions such as state banks. 4. Launched Retail Advisory Service: (a) Assisted 10 commercial banks to and Networks. 3. More commercial approach. develop products and services to serve the poor clients; and (b) A toolkit is 3. Develop 2­3 pilots or case 4. Expand institutional and under preparation to present lessons learned from this program to be used studies per year on alternative human resource base. also for a workshop. delivery mechanisms for 6. To find new alternative delivery mechanisms, launched, in 2006, a financial services to the poor. Technology Program: (a) Published a Focus Note, "Using technology to Build Inclusive Financial Systems" providing a rationale for CGAP's scaled- up technology program; (b) Initiated a program to assist 8 demonstration projects (out of 200 proposals) to deliver financial services to poor people using technology solutions; (c) Conducted survey of low-income mobile phone banking users; (d) Developed relationship with major technology- related organizations; and (e) Published Focus Note on the use of agents in branchless banking based on experience of 5 pioneering countries. II. Promoting Diverse Financial Services to a Broad Range of Clients 1. Pilot and document at least 5 1. Greater diversity and 1. A paper published on small savings. models of financial services for innovation in financial services 2. Draft technical guide on money transfers prepared and being field the poor. available to the poor. tested. 2. Different segments of the poor 3. Developed Progress Out of Poverty Index (PPI), a score-card to track are served by different kinds of poverty levels of MFIs' clients. financial services adequate to 4. Launched Graduation Program including a pilot in 4 countries to develop their demands. methodologies to create a pathway for the poorest to escape chronic food 3. Ability of financial institutions to insecurity and dependence on subsidies. measure social performance. 5. Under Pro-Poor Innovation Challenge, awards given to 54 MFIs for 4. Ability to provide evidence of innovative financial products and methodologies to serve very poor. impact. 6. Work advancing on a global reporting format on social performance of MFIs. 18 Planned Industry Objectives Actual Output 7. CGAP cooperating with a research on developing the Progress Out of Poverty Index, a country-specific scorecard to track the poverty levels of MFIs' clients. 8. Work underway on a set of performance indicators linked to the Millennium Development Goals. III. Building Financial Market Infrastructure 1. Have 600 of MFIs reporting 1. Improved quality, quantity, and 1. Over 1000 MFIs reporting on MIX (www.themix.org). to the MIX using MIX financial availability of financial and 2. Over 300 MFIs reported at least one social performance indicator to the and poverty reporting standards institutional information on MFIs. MIX. 2. Convince 150 FIs to report on 2. Ability to provide evidence of 3. 99 funds are reporting to and profiled on MIX. depth of poverty outreach to the impact. 4. More than 400 ratings and assessments completed. Impact assessment MIX and 100 FIs to report on of Rating Program will be launched after its closure in FY08. social performance 5. Conducted three rounds of Financial Transparency Awards attracting 3. Have 50 of investors more than 600 participants. reporting to the MIX 6. Nearly 100 MFIs assisted for assessing building their information 4. Complete rating of 400 MFIs services. by well-recognized agencies IV. Fostering Sound Policy and Legal Frameworks 1. Publish information on Improved policy and regulatory 1. Established the Microfinance Gateway (www.microfinancegateway.org) regulatory regimes in 50 environment for financial that has information on microfinance related laws and regulations on more countries in CGAP's online institutions serving the poor. than 50 countries. database. 2. Provided policy advice and guidance on laws and regulations, and 2. Engage in policy dialogue on access to finance strategies to more than 30 countries including full 5 microfinance regulation at substantial country-level policy diagnostic assessments. specific country level (5 3. Held/organized about 50 conferences, seminars and policy related substantial country training programs. engagements per year). 4. Published Focus Notes/Occasional Paper on policy-related issues, 3. Organize conference/training interest rate sensitivity, competition and efficiency, loan guarantees, anti- presentations (10 per year). money laundering/combating the financing of terrorism and regulation of branchless banking. 5. Conducted diagnostics of regulatory framework for branchless banking in 7 countries. 6. Assisted regulators in 3 countries to develop branchless banking regulations. 19 Planned Industry Objectives Actual Output V. Improving the Effectiveness of Microfinance Funding 1. Complete 17 Peer Reviews Improved effectiveness, 1. 17 Peer Reviews completed (program since closed). of donor agencies. efficiency, and accountability of 2. 4 updates completed (program since closed). 2. Complete 10 updates of peer donor support, including that 3. Published "Good Practice Guidelines for Funders of Microfinance". reviews. executed through/by CGAP. 4. Completed country-level effectiveness and accountability reviews 3. Train donor staff (500 by (CLEARS) in 6 countries. Year 5). 5. Preparing a paper synthesizing lessons learned from CLEARS. 4. Develop and provide tailored 6. Based on the lessons learned from the Peer Reviews and CLEARS, resources for donors (50 Donor developed SmartAid Index to create standards for measuring and scoring Briefs, 50 Presentations, 30­35 donor agencies' effectiveness in supporting microfinance. 7 agencies training courses by Year 5). participated in the pilot phase. 5. Establish Donor Helpdesk 7. Supporting 5 major regional programs/initiatives (European Union and and promote its use by donors. African, Caribbean and Pacific (ACP) States, survey of fund providers in 6. Develop New Guidelines for Africa, MENA Initiative, the regional hub for ECA, etc.). Supporting Microfinance. 8. Hosted 2nd High-Level Meeting of CGAP Members ("Better Aid for Access to Finance") resulting in a compact reaffirming the sound principles for "Better Aid for Access to Finance". 9. Organized a donor meeting in Sri Lanka to improve support to microfinance in that country. 10. 260 donor staff trained through CGAP/UNCDF course. Conducted thematic workshops on donor reporting, social performance and role of government to more than 100 participants. 11. Helpdesk launched and operational. 12. Developed a range of resources for donors including 25 donor briefs, 24 presentations and 13 training modules. Source: CGAP Annual Reports, documents and 2007 Evaluation Report. 20 basic institution-building such as enhancing the quality of information disclosure (i.e., promoting credible external ratings and improving information services) and encouraging pro-poor services. The overall quantum of grants under these programs is about half of what it was in Phase II32 and many of these programs came to a close by the end of Phase III in FY08. PROMOTING INSTITUTIONAL DIVERSITY 3.19 The objective of this strategic priority was to explore and deploy new delivery mechanisms for financial services for the poor to complement existing MFIs. The activities planned for this purpose included the publication of appraisal tools for microfinance, implementing a program of appraisal and monitoring of MFIs for donors to bring the right mix of funding for MFIs and identifying/developing alternative delivery mechanisms for financial services to the poor.33 The actual outputs far exceeded the plan since all planned activities were carried out, except for the appraisal and monitoring of MFIs, which has had to be discontinued due to low demand from donors. A large amount of effort was exerted on identifying/developing alternative delivery mechanisms for financial services for the poor. CGAP published a detailed Focus Notes/Donor Brief on linkages between commercial banks and MFIs, cooperatives, community-managed loan funds and the Self-help Group Model in India, and work reached an advance stage for a paper on state development banks. This work served a useful purpose by demonstrating that delivery of financial services to the poor is commercially feasible using a variety of delivery channels. 3.20 In addition to its planned activities, CGAP launched two new initiatives. The first was an advisory service to assist mainstream financial institutions (mainly commercial banks) in developing products and services for the poor. CGAP assisted 13 commercial banks under this service. The second was a technology program, launched in 2006, with the objective of increasing access to finance for poor people by demonstrating viable technological approaches to overcome constraints to accessing payments services and ultimately loan and savings services to the poor, and advocating regulatory frameworks that balance opportunities for expanded access with adequate protection for consumers and the financial system.34 This program started, in 2006, with the publication of a CGAP Focus Note, "Using Technology to Build Inclusive Financial Systems". The progress towards implementing this program has been impressive. The outputs include a survey of low-income mobile phone service users, developing working relationships with industry organizations, finding solutions to regulatory issues related to branchless banking35 and launching nine demonstration 32. CGAP's total commitments for grant-based programs in Phase III were $4.4 million, about 50 percent of the level in Phase II. 33. Some of CGAP's activities contribute to multiple strategic priorities. This review covers each activity under the most relevant priorities, not necessarily corresponding to the listing in CGAP documents. 34. Bill and Melinda Gates Foundation supports the technology program through a $24 million four-year grant. This works in partnership with a variety of stakeholders, such as mobile networks, payment system operators, banks and other financial institutions. 35. "Branchless banking" a term coined by CGAP refers to banking services provided through the use of mobile phones. 21 projects36 to deliver financial services to the poor using technological solutions. These projects are focused primarily on the agent model and using small retail outlets. 3.21 The 2007 Evaluation observed that by the end of Phase III, CGAP will have significantly contributed to the outcomes expected at the outset, namely: A variety of financial institutions with different objectives, methodologies, and services serving the poor Scaling up through use or restructuring of existing infrastructure of large-scale institutions such as state banks Adopting a more commercial approach Expanding the institutional and human resource base. At a broader industry level, there has certainly been progress in terms of outputs. But in the absence of an in-depth analysis of the causal relationship between CGAP outputs and developments in the industry, it is not possible to attribute the progress to CGAP's outputs. PROMOTING DIVERSE FINANCIAL SERVICES TO A BROAD RANGE OF CLIENTS 3.22 The objective of this strategic priority was to promote diversity in financial services and this objective was to be realized through documenting and piloting new models of financial services for the poor. Again, CGAP's actual outputs exceeded what was planned. It completed and published detailed analyses of savings and money transfers. The paper on savings highlighted the importance of mobilization as a source of funding for microfinance and tackled issues relating to factors affecting supply and demand for deposit services. In recent years, CGAP has integrated the importance of savings mobilization in its ongoing advocacy with financial institutions, governments, and microfinance funders. CGAP developed and published a technical guide on money transfers for MFIs interested in providing such services to their customers and is co-funding a global fund for remittances, which provides technical assistance to MFIs that show a credible strategy for providing money transfer services in poor, rural areas. CGAP also launched the Graduation Program, which aims to develop methodologies to graduate the poorest people out of food insecurity into appropriate financial services. Under this program a pilot is under implementation in four countries, which coordinate a broad package of subsidized services (e.g., food aid, asset/grant transfers, etc.) to develop methodologies to create a pathway for the poorest to escape chronic food insecurity and dependence on subsidies. This is a pioneering initiative and could lead to a breakthrough for poverty alleviation policies and strategies. CGAP was also instrumental in forming a task force of experts to establish standardized measures of social performance in microfinance. This task force (including representatives of commercially-oriented microfinance service providers) is working on a common set of indicators of social performance. Complementing this work, CGAP and the Grameen Foundation collaborated in sponsoring the development of the Progress Out of Poverty Index, a country-specific scorecard to track the poverty levels of MFIs' clients. Finally, CGAP supported innovative financial products and methodologies aimed at extremely poor and marginalized people by recognizing MFIs that provide such products through a grant- 36. The selection was made on competitive basis from out of more than 200 proposals received by CGAP. 22 based recognition program. This program, called the Pro-Poor Innovation Challenge (PPIC), provides a global platform to widely disseminate knowledge about pro-poor products. 3.23 At the beginning of Phase III, CGAP expected the following outcomes from its work on promoting diverse financial services to a broad range of clients: To introduce greater diversity and innovations in financial services available to the poor For different segments of the poor to be served by different kinds of services adequate to their demands For financial institutions to be able to measure social performance To be able to provide evidence of impact. The wide variety of CGAP's initiatives towards realizing these objectives seem appropriate, but these are short of the stage where their real outcomes can be assessed. As the 2007 Evaluation commented, CGAP needs to establish demonstration effects of the work on money transfers, savings and graduation program, to disseminate lessons from PPIC and to bring the work on social performance measurement to a closure by publishing the results. BUILDING FINANCIAL MARKET INFRASTRUCTURE 3.24 The objective of this strategic priority of CGAP was to enhance financial transparency of service providers to the poor, a pre-requisite for the development of the market. The strategy for achieving this objective called for substantially increasing the disclosures on the Microfinance Information eXchange (MIX) by MFIs and fund providers and facilitating external ratings of MFIs to enhancing the credibility of their disclosures. CGAP made good progress in implementing this strategy by the end of the fourth year of Phase III. Over 1000 MFIs and almost 100 investment funds are currently included on MIX compared to a target of 600 and 50, respectively. More than 400 MFIs were assisted in getting rated by independent rating agencies of good repute. CGAP plans to do an assessment of this program after it closes in FY08. In FY05, CGAP started a program (Financial Transparency Awards) to recognize outstanding quality of disclosures by MFIs by giving financial awards and recognition. So far, three rounds of awards have been completed in which more than 600 MFIs participated. Good disclosure requires an effective information system to collect and provide relevant information. To promote development of appropriate information systems, CGAP started, in FY04, a program to help MFIs assess their information service needs and, so far, nearly 100 MFIs have been assisted. 3.25 In Phase III, CGAP made substantial progress in achieving the objective of enhancing financial transparency of service providers to the poor. This is confirmed by the growth of MIX, both in size and substance. Created by CGAP and spun-off as a not-for-profit corporation in 2002, MIX has since evolved into the leading business information provider on the microfinance sector.37 In addition to providing detailed data on fund providers, investors, networks and service providers, MIX also provides customized performance reports with comparisons with peers. Through MIX, CGAP has addressed, to a large extent, the issue of the lack of reliable, comparable and publicly available information on 37. The other CGAP partners in MIX are Citigroup Foundation, Deutsche Bank Americas Foundation, Open Society Institute and Rockdale Foundation. 23 microfinance institutions. CGAP's support to MFIs through its rating service and information services assessment program also contributed, indirectly, to the availability of reliable information. The Financial Transparency Awards provided an incentive to MFIs to improve the quality of their information and disclosures. However, whether financial awards were necessary is debatable. The 2007 Evaluation noted that "it is not the financial incentives offered by CGAP...that is valued by most by institutions, but the recognition by CGAP. The CGAP brand and recognition is very powerful and highly coveted." Therefore, it is most likely that financial institutions would have participated in the competition for recognition even without financial rewards. FOSTERING SOUND POLICY AND LEGAL FRAMEWORKS 3.26 CGAP's objective in this strategic priority was to establish supportive legal and regulatory frameworks that safeguards poor people's money, facilitates diversity of institutions and financial products and promotes competition. The specific activities planned under this priority included providing information on a variety of regulatory regimes, engaging in policy dialogue on microfinance regulation in specific countries and disseminating knowledge on policy issues through conferences and trainings. In the first four years of Phase III, CGAP made considerable headway in achieving its targeted outputs. A major output was the creation of Microfinance Gateway, an online library on microfinance- related laws and regulations of more than 50 developing countries.38 In addition, CGAP established itself as a major source of policy advice on microfinance and branchless banking. It provided such policy advice to more than 30 countries, in addition to diagnostics of the regulatory environment for branchless banking in three countries. For its policy work, CGAP has developed a Policy Advisory Consultant Cadre comprising of policy experts, which operates under the supervision of CGAP's own core team of policy experts. In addition, CGAP invariably involves one or more of its members in its policy advisory activities. Finally, to disseminate knowledge on policy matters, CGAP published a technical tool for assessing the regulatory environment for branchless banking and organized more than 50 conferences, seminars and policy-related training programs. 3.27 The 2007 Evaluation noted that in Phase III, CGAP underwent a major shift in thinking and strategy towards policy work. Initially, CGAP advocated a hands-off stance, but later pursued a more proactive approach in Phase III to creating a conducive policy environment. This shift was justified considering the rapid growth of the microfinance sector, its growing complexity and a wider range of financial services being provided (e.g., savings, money transfers, micro-insurance, use of mobile phones, etc.). In this new scenario, policy issues needed to be addressed not only for consumer protection but also to facilitate the development of new channels of delivery of financial services. However, this involvement of CGAP in the policy area also raises a few operational issues. By its nature, policy work is a long-term process that involves diagnostics, prescriptions and follow-up to ensure implementation of recommendations. CGAP has some constraints to its ability to do the follow-up work. Its resources are limited and it does not have the physical presence on the ground to continually shepherd reforms. In fact, CGAP works only as a short-term technical 38. Managed by IRIS Center of the University of Maryland, the Law Library features over 7,000 online documents (laws, regulations, research publications, news, etc.), over 1000 listings of microfinance focused organizations and over 400 consultant profiles. On an average every month, 50,000 users from 175 countries visit this site. 24 advisor and not a long-term technical assistance provider or a donor. This problem could be overcome by instituting specific mechanisms to ensure that the donors would include microfinance issues in their overall policy dialogue agenda. In the World Bank, such mechanisms would need to focus on the process of preparation of the country assistance strategy as well as the work of relevant country/sector units. The other issue is coordination with other donors, especially the World Bank to ensure consistency with the ongoing broader financial sector policy dialogue. To a large extent, CGAP has managed this issue by coordinating its policy work with the World Bank and other relevant donors and confining itself to issues specific to microfinance services where it has a distinct comparative advantage. CGAP will need to continue to pay attention to coordination with donors to maximize the outcome of its policy work. 3.28 There is no data available on the actual outcomes of CGAP's advice on the policy regimes of countries beneficiaries of CGAP's work. However, there are anecdotal indications of CGAP advice having had some positive effect on policy makers.39 This is also an area where CGAP needs to develop a monitoring and reporting system to measure the outcome of its work. IMPROVING THE EFFECTIVENESS OF MICROFINANCE FUNDING 3.29 The fund providers (donors as well as private sources of funds) play a major role in the development of microfinance and promotion of access to finance. In Phase III, CGAP aimed at working with these players to improve the effectiveness, efficiency and accountability of their operations in microfinance. For this purpose, a range of activities were planned including reviews of the effectiveness of the institutional set up of donors to handle microfinance work, steps for building their capacity and dissemination of good practice guidelines for fund providers. CGAP's output in the first four years of Phase III exceeded the target. With respect to the reviews of the effectiveness of the institutional set-up of fund providers, CGAP completed reviews of 17 donors involving technical staff and management.40 The lessons learned from these reviews were published in a Global Results paper and in a shorter piece drawing out policy implications for development decision leaders. Besides, under a program called Country-Level Effectiveness & Accountability Reviews (CLEARSs), CGAP conducted reviews in six countries where it focused on aid- effectiveness and coordination issues at the country-specific level. Going one step further and using the knowledge gained from both the peer reviews and the country-level reviews, CGAP has developed and piloted a tool, called SmartAid for Microfinance Index, which creates standards for measuring and scoring donor agencies' effectiveness in supporting microfinance. Seven agencies participated in the pilot phase. CGAP also completed a review of the microcredit portfolio of the World Bank and UNDP and published a Focus Note summarizing the findings. CGAP delivered assistance to fund providers by establishing a 39. CGAP's own assessment shows that in four countries (Pakistan, Syria, Russia, and Ecuador) and Latin America Region, its policy advisory work contributed to adoption of a comprehensive policy framework for branchless banking, adoption of comprehensive regulations or amendment/withdrawal of policies/regulations detrimental to the interest of microfinance. 40. Each review was done by a team of three-four members drawn form other donors and coordinated by a CGAP staff. All 17 reports are available on CGAP's website. Four updates were also done. 25 helpdesk, training over 260 staff and publishing Donor Briefs41 and creating a Donor Information Resource Center containing a range of practical briefs, case studies and operational tools concerning good practice relating to microfinance. For dissemination of good practice, CGAP published "Good Practice Guidelines for Funders of Microfinance." In an effort to promote greater commitment to following sound practices in microfinance, CGAP organized a high-level meeting that gathered 29 of its members including top management and microfinance staff, which issued a compact (Better Aid for Access to Finance) reaffirming the sound principles of microfinance. 3.30 CGAP's work on improving effectiveness of microfinance funding was considerably scaled up in Phase III. However, assessing the outcome of this high-volume output is difficult since it is dependent on the assessment of the extent to which the effectiveness, efficiency and accountability of fund providers, mainly donors, increased as a result of CGAP's outputs, and this has not been done. However, the 2007 Evaluation reports that all members of the Council of Governors praised CGAP's performance in this area with 62 percent giving it an "excellent" rating. Also, many donors are reported to have implemented some of the recommendations in CGAP reviews.42 3.31 Nonetheless, work done by CGAP, so far, on enhancing institutional accountability among fund providers and their aid effectiveness offers two important lessons. The first lesson is that CGAP does not have the capacity to work with all or even a majority of fund providers. After doing 17 reviews and four updates, CGAP realized that more informal and opportunistic engagements with individual fund providers were a better strategy than formal reviews and updates, which were a burden on core CGAP staff time. In fact, CGAP has had to turn down many requests even for informal engagements. Another lesson is that the degree of the success of CGAP's efforts in enhancing the effectiveness of microfinance funding depends to, a large extent, on the willingness of fund providers to seek and benefit from CGAP's support, which, in turn, is heavily dependent on internal incentives and even the political considerations of each donor/funder. An independent assessment commissioned by CGAP did not find strong evidence that SmartAid for Microfinance Index provides incentives for an agency to improve its internal systems.43 Therefore, the expectations of the outcome of CGAP's aid-effectiveness work should be kept at a modest level. The value of this activity lies in generating appropriate tools that fund providers serious in improving their internal systems can use readily. Efficiency 3.32 A major indicator of the efficiency of CGAP has been its ability to adapt its activities to the evolving requirements of the microfinance industry. As mentioned earlier, CGAP 41. "Donor Briefs" are concise two-page presentations of issues affecting microfinance programming and operations of fund providers. 42. Some of the more notable changes to internal systems include AFD's decision to make an exception to its average project size to allow for smaller programs, more appropriate for microfinance. Another example is the flexibility KfW gained to work directly with private actors by using the Peer Review letter to negotiate with the German government the ability to take capital shares in MFIs with a state guarantee. 43. Mark Flaming, SmartAid for Microfinance Index (SmartAid); Assessment of Pilot Round 2007, August 2008. 26 followed a conscious strategic decision in Phase III to move from primarily grant-making to wide range of self-implemented industry-wide initiatives that enabled it to establish itself as a respected brand name in and the authoritative voice of the industry. CGAP's vigilant management and effective governance structure contributed to its capacity for appropriate adaptation. In addition, each initiative and activity is subjected to strict budgetary control as well as critical tests (demand, the value for money and potential impact). Moreover, CGAP monitors the progress of each activity and its continued relevance; and, when necessary, frequently makes appropriate mid-course adjustments. 3.33 The uniqueness of CGAP's organization in terms of its mission and activities makes it difficult to evaluate its efficiency or cost-effectiveness since there no comparable benchmarks, which is further complicated by the absence of a systematic assessment of CGAP's overall impact on the industry. However, some anecdotal evidence suggests that CGAP's activities are cost-effective. The 2007 Evaluation found that a great majority of the members of the Council of Governors felt that CGAP was a cost-effective organization. In the interviews conducted by IEG the Council of Governors and EXCOM members expressed similar views. The substantial volume of CGAP's output during Phase III also supports this view. This review did not find any noticeable waste of resources. On the contrary, there are strong indications that CGAP is highly cost-conscious. For example, it has opted to use consultants rather than have permanent staff for temporary assignments without a clear long- term need, which gives it flexibility in changing the skills-mix as needed. Another example is the decision to shift CGAP headquarters offices to a new location adjacent to the World Bank at a lower cost. Also, the decision to set up an office in Paris has cut down travel costs.44 Governance 3.34 This section assesses the effectiveness of CGAP's governance structure, which was described in paras. 1.5­1.8. CGAP's governance structure is a combination of shareholders and stakeholders. The Council of Governors represents all members, while EXCOM consists of the representatives of the Council of Governors as well as the industry, a major stakeholder, and Investment Committee comprises nominees of the World Bank, the fiduciary agent. Over the years, the governance structure has served CGAP well with appropriate division of responsibilities and inter-linkages. CGAP has a strong legitimacy with an effective participation of almost all major stakeholders -- the service providers, MFIs, networks, donors, other financiers, etc. The Council of Governors consists of almost all donors and fund providing agencies, while the microfinance industry is represented by four members on EXCOM. The Council of Governors elects industry representatives out of candidates recommended by EXCOM after a thorough search and scrutiny process. The incorporation of the former Policy Advisory Group into a revised EXCOM in 2002 has generated a positive synergy between fund providers and practitioners leading to better policy and operational guidance to CGAP. Until July 2008, the World Bank had a permanent seat on EXCOM. Going forward, the World Bank will be included in one of the five constituencies45 each of which elects one EXCOM member. This development further 44. CGAP estimates that the saving from a cheaper location of its headquarters and reduced travel cost due to the establishment of Paris office is over $200,000 annually. 45. For purposes of the election of EXCOM, members of the Council of Governors are divided into five constituencies, two for bilateral and one each for multilaterals, development finance institutions and foundations. 27 enhances CGAP's legitimacy as a representative of the industry without the domination of the World Bank. In addition, through its regular consultation and partnership programs, CGAP seeks the views and inputs from a wide range of stakeholders. As a result, CGAP is now recognized as the leader of microfinance industry as confirmed by a client survey done for the 2007 Evaluation which found that over 97 percent of respondents were familiar with CGAP. The rapid growth in CGAP's membership and their contributions are also an indicator of its growing legitimacy. 3.35 Two recent decisions by the Council of Governors/EXCOM have a direct bearing on CGAP's legitimacy. The EXCOM recently decided to accept the membership request from the Development Bank of China (CDB) and to invite CDB to join CGAP as the first bilateral CGAP member from a developing country. While CDB's acceptance of the invitation is still pending due to management changes in China, this move could enhance the legitimacy of CGAP by recognizing the growing role of CDB as a cross-border fund provider for the development of microfinance. The other decision, taken in late 2007, requires all members to make core funding in cash starting from July 2008, which had not been a mandatory requirement before then. This change reflects CGAP's resource mobilization strategy for FY09­13, emphasizing a broader resource base, reduced reliance on the World Bank and greater flexibility in the use of funds contributed by members. The enforcement of this decision could result in the reduction of the total membership by up to five members as many multilateral institutions maintain that their internal charters do not allow them to make such contributions.46 However, CGAP's Charter would allow these institutions to continue as non- voting members. They would also be eligible, as in the past, to participate in CGAP activities as partners. Already, three members, IFAD, UNCDF and ILO, have indicated that they have found a way to comply with the new requirement. 3.36 CGAP has maintained a high degree of transparency. Its Web site provides detailed information on its governance, the strategy, finances and operating results and outputs. All publications are available on the Web site as well as in hardcopies, which are widely distributed often in multiple languages. CGAP has also posted its completed external evaluations on its Web site. However, there are two areas where transparency could be further enhanced. CGAP's Charter, being an overarching document, could usefully be posted on its Web site along with greater clarity with respect to funding. The Charter currently specifies "core funding in cash through flexible mechanisms," the objective of which has been to give CGAP flexibility to maximize contributions from its members. While this open language has so far worked to CGAP's advantage, the adoption of additional transparent guidelines could further strengthen the negotiating position of CGAP. Such guidelines could be built around four broad principles: (a) the capacity of the member to pay; (b) consistency within the constituency (peer group); (c) CGAP's funding requirements; and (d) cost-sharing under partnership arrangements. In addition, a code of conduct for EXCOM members could be developed and posted on the Web site. Such a code of conduct would be broader than the current CGAP policy that EXCOM members cannot be grant recipients of CGAP and that any advisory work directly benefiting an EXCOM member must be cleared with EXCOM. This could help minimize the risk to CGAP's reputation, especially in view of the fact that the institutions which the members represent could conceivably benefit from the work program and activities of CGAP. 46. The relevant multilateral institutions are: ILO, AsDB, AfDB, UNCDF and IDB. 28 3.37 According to CGAP's Charter, EXCOM is accountable to the Council of Governors for carrying out CGAP's strategy. The overall performance of CGAP is reviewed annually by the Council of Governors. The Investment Committee is required to exercise the World Bank's fiduciary responsibility while considering proposals for grants and initiatives to ensure compliance with DGF criteria and report its decisions to EXCOM. The Executive Director submits interim reports, the Annual Reports, budgets and work plans to EXCOM. These procedures and reporting arrangements are regularly followed. In IEG's interviews the Council of Governors and EXCOM members did not raise any compliance issue. The Operational Team takes its accountability quite seriously and has applied it throughout the organization. The staff are expected to cooperate fully with external evaluations, including the 2007 Evaluation. The findings of the evaluation were analyzed and discussed internally and a follow-up action plan developed. In addition, as a common practice, CGAP undertakes assessments of initiatives on completion, identifying successes as well as failures and drawing lessons. Finally, building on the existing system of monitoring of its initiatives, CGAP has started work on developing an overall and systematic monitoring and reporting system to assess the outcomes and impacts of its activities on the microfinance sector, currently a gap in its accountability. 3.38 CGAP ensures a high degree of fairness in the conduct of its business in a number of ways. The activities involving grants and awards are routinely advertised on CGAP's own and, in some cases, on other appropriate Web sites. In addition, the award decisions are made by panels of independent experts. Moreover, as a policy, CGAP involves one or more of its members as partners in all its major activities by making information available well ahead of the time through EXCOM members who represent almost all the major active players in the field of microfinance. 4. The World Bank's Performance as a Partner 4.1 The World Bank plays multiple roles vis-à-vis CGAP. It is one of CGAP's founding members and historically the largest single provider of funds. As the executor of the bilateral trust funds and host institution for CGAP's Operational Team, it performs fiduciary and administrative oversight functions. It is also actively involved in the development of inclusive financial systems and microfinance. These roles often interact with each other in complex ways. For this review, the World Bank's role is reviewed under five broad topics: (a) playing up to its comparative advantages at global, (b) promoting microfinance at the country level; (c) exercising oversight functions; (d) managing risks; and (e) disengagement strategy. Providing Comparative Advantages at the Global Level 4.2 CGAP's close association with the World Bank gives it access to many advantages. Perhaps, the most valuable has been the convening power and prestige that comes from its World Bank association. Almost all the world's major donors, bilateral as well as multilateral, have become member of CGAP and are making increasing financial contributions, as described in para. 1.9. It is doubtful if a majority of them would have shown the same degree of interest in the absence of a direct World Bank involvement. CGAP has appropriated this convening power to organize many international, regional and country-level 29 conferences and meetings with positive results and to promote global consensus on its various policy and operational guidelines. CGAP's close proximity to the World Bank gives it access to the World Bank's knowledge reservoir and technical expertise, which is becoming increasingly critical as CGAP expands its policy work (paras. 3.26­3.28). 4.3 At the global level, there were two notable World Bank contributions: (a) an international conference on "Access to Finance: Building Inclusive Financial Systems" held at the World Bank headquarters in May 2006, for which CGAP was one of the co-sponsors, and the proceedings of which were published by the World Bank in a book in 2007; and (b) the publication of a major policy research report in 2008, "Finance for All? Policies and Pitfalls in Expanding Access". These two efforts provide an excellent overview of the major trends and in-depth analysis of the emerging issues relating to microfinance and access to finance. 4.4 In addition to the macro-level comparative advantages, CGAP also enjoys significant tangible institutional advantages. Bilateral trust fund agreements between CGAP members and the World Bank have assigned to the World Bank the role of execution and administrative oversight of CGAP. As a result CGAP is located in the World Bank and CGAP is subject to the administrative policies and procedures of the World Bank similar to other global programs located in the Bank. Overall, this arrangement has worked to the benefit of CGAP. The areas of clear benefit include the ability to recruit staff world-wide, the use of the many support services provided by the World Bank (legal, disbursement, accounting, auditing, etc.), and access to the reservoir of knowledge residing in the World Bank. While this closeness to the World Bank could be perceived as compromising CGAP's professional independence, the benefits have so far outweighed the costs. Promoting Microfinance at the Country Level 4.5 The World Bank has played a significant role in promoting the development of microfinance and inclusive financial systems in developing countries, which has complemented CGAP's work. The World Bank Group's Financial Sector Strategy (2007) recognizes access to finance for the under-served as an area requiring special attention and highlights the role of CGAP. At the country level, the World Bank provides support through its lending and analytical work. Between FY00 and 07, the World Bank's average annual lending commitments were about $516 million and even higher ($635 million) in more recent years, FY04­07 (Table 3). In comparison, the average annual lending by IFC has been a little over $300 million,47 AsDB, $65 million, and the USAID, $188 million.48 Between FY00­07, World Bank spent an average of $17 million annually on 111 pieces of analytical work, including policy advice and technical assistance. 4.6 The World Bank's country knowledge and connections with local policy makers, enables CGAP to do its country level work more effectively. As a matter of policy, CGAP maintains a 47. IEG, Financing Micro, Small, and Medium Enterprises: An Independent Evaluation of IFC's Experience in Frontier Countries, 2008. In FY07, IFC's lending was about $200 million. 48. USAID, Microenterprise Results Reporting: Annul Report to Congress, Fiscal Year 2006, June 2007. Data include support to microfinance and microenterprises. AsDB data relate to FY90-05 and USAID, FY00-06. 30 close liaison with the World Bank's country offices and staff -- a strategy that contributes to the sustainability of CGAP's policy work in the sense that CGAP provides technical inputs which Table 3. The World Bank's Lending for Micro and SME Finance, FY2000­07 (US$ millions) IBRD IDA Total FY00 370 68 438 FY01 253 210 463 FY02 74 66 140 FY03 481 65 546 FY04 390 239 629 FY05 266 88 355 FY06 732 178 910 FY07 470 178 648 Source: World Bank Business Warehouse. the World Bank can integrate in its continuing country policy and operational work. Country level knowledge is also imparted by members of the Investment Committee. 4.7 The World Bank has also benefited from the work of CGAP. A number of managers and staff interviewed by IEG expressed a high degree of respect for the quality and usefulness of CGAP's work and expertise. There is a growing awareness in the World Bank of the importance of microfinance and access to finance, which could be attributed, at least partly, to the knowledge generated by CGAP and its advocacy work. A review of Country Assistance Strategies (CAS) from FY99­07 revealed that a growing number of country managers are including a discussion of microfinance, access to finance and the role of CGAP. The ratio of CAS covering such a discussion rose from about half to two-thirds between FY99­03 and FY04­07. There is also evidence that these discussions are more in- depth attention than before (Table 4). Table 4. Recognition of CGAP and Microfinance in CASs FY99­03 FY04 FY05 FY06 FY07 FY04­07 No. of CASs reviewed 141 35 33 17 26 111 % CAS discussing microfinance, access issues and CGAP 53% 60% 58% 82% 77% 67% % of in depth discussion (5 or more 20% 33% 16% 57% 55% 39% references) 4.8 In FY05, CGAP did an evaluation of the World Bank's support to microfinance through lines of credit.49 This evaluation revealed a rather disappointing picture with more than three-quarters of the projects either failing or appearing unlikely to produce long-lasting results. CGAP attributed this outcome to agency environments and systems that did not give staff the right incentives, information and resources for such projects. Partly this outcome reflected the weaknesses in the World Bank's quality assurance systems for such projects. 49. This evaluation followed an earlier evaluation by IEG of the World Bank lending for lines of credit (OED Review of Bank Lending for Lines of Credit, 2005, report No. 31131). 31 The World Bank responded positively by taking a number of steps to improve the situation. In FY05, a formal and mandatory system of quality enhancement reviews (QERs) of all projects involving lines of credit was instituted under the supervision of the Financial Sector Board. In late 2006, the Quality Assurance Group did an assessment of the impact of the QER system on the quality of lines of credit projects. This assessment found that the QER contributed noticeably to the better quality of projects, but many projects involving lines of credit did not have had QER. To follow-up on these findings the Vice President, Operational Policy and Country Services and FPDVP jointly issued a memorandum to staff requiring compliance with the QER requirement. In addition, FPD has issued detailed guidelines for lines of credit projects and has organized workshops to enhance quality of lines of credit projects and their compliance with the World Bank's policy. These are appropriate steps, but need to be sustained to be effective. Exercising Oversight 4.9 CGAP's Executive Director reports administratively to FPDVP.50 Since FPDVP also chairs the Council of Governors and the Investment Committee, he has the advantage of having a full engagement in the policy and strategic directions set by the Council of Governors and the details of CGAP's activities. The World Bank also performs its fiduciary responsibility through the Investment Committee, which provides valuable technical guidance to CGAP in the process of reviewing and approving all proposals for grants and initiatives costing over $100,000. The minutes of the meetings indicate in-depth discussion of all proposals, often leading to modifications or re-design.51 In addition, the World Bank diligently performs its fiduciary responsibility by managing all trust funds involving negotiating and executing trust funds and ensuring their use consistent with the terms of agreements. The World Bank's procurement and disbursement guidelines are applied to CGAP as well. 4.10 While the World Bank's administrative and fiduciary oversight of CGAP has worked well to the distinct advantage of CGAP, there is also a potential risk. This risk arises from the possibility of CGAP being asked to play some of the anchor functions52 for microfinance operations in the World Bank. This matter has acquired some seriousness given that the size of this anchor is too small to be effective. Since CGAP has the necessary expertise and it is within the administrative jurisdiction of FPDVP, a proposal is currently under consideration to seek CGAP's help for some of the anchor tasks. This would be done on a cost- reimbursement basis. In principle, this function is consistent with one of CGAP's five strategic priorities (i.e., improving the effectiveness of microfinance funding). However, a substantial and regular involvement of CGAP in the internal management of the World Bank 50. From 1999 to 2004, the CGAP Director reported to the Vice President of Private Sector Development and Infrastructure, and from January 2005 to August 2006, to the Vice President of Financial Sector Development, as a result of organizational changes in the World Bank. (See page vi.) None of these organizational changes, including the creation of the joint World Bank-IFC Vice Presidency for Financial and Private Sector Development in September 2006, have had any negative impacts on the Bank's relationship with or oversight of CGAP. 51. IEG attended one IC meeting. 52. Anchor functions include quality enhancement, operational support, strategy and policy work, training, guidance, etc. 32 could generate a perception on the part of other CGAP members of pre-emption of CGAP's resources by the World Bank. So far, as confirmed in interviews with IEG, most members feel that generally the World Bank has performed its fiduciary role fairly and equitably. Risk Management 4.11 The risks to the World Bank associated with CGAP are generic to many global programs: (a) a weak commitment of CGAP members its mission resulting in the World Bank providing the bulk of the funding; (b) misdirected strategy; (c) weak governance; and (d) possible conflicts with World Bank policies and operations. None of these risks has materialized so far. CGAP members have consistently shown their strong commitment and support to CGAP's mission by actively and positively participating in its governance as well as initiatives and activities and, above all, coming up with increasing financial contributions. As described in Chapter 1, the total contribution from members other than the World Bank has steadily increased from 19 percent in Phase I to 43 percent in Phase II and 82 percent in Phase III. There are strong indications that this level of interest and commitment will be sustained in FY09­13 as well. Because of its strong governance structure and commitment of members in serving the interest of the microfinance industry, CGAP has been following a well thought out strategy that is constantly monitored and modified in accordance with the changing industry environment. This constant adaptation signifies the dynamic role played by CGAP's members with active support from the World Bank. Finally, the active involvement of the Investment Committee with CGAP's activities mitigates the risk of possible conflicts with World Bank policies and/or operations globally or at the country level. Disengagement Strategy 4.12 Initially, CGAP was set up, in 1995, as a three-year program.53 Since then, the Council of Governors has renewed it three times -- for Phase II (FY99­03), for Phase III (FY04­08) and more recently in terms of a FY09­13 strategy. A review of relevant minutes of the Council of Governors' meetings shows that discussions focused on issues in the microfinance sector and the appropriateness of the strategy, but the option of closure or phasing out of CGAP was not explicitly discussed during the approval of the FY08­13 strategy (although it was discussed and dismissed in the 2007 Evaluation). Since the World Bank chaired these meetings, it evidently agreed with the substance of the discussion and the decision. Also, there has been no discussion of disengagement from CGAP at any other level of the World Bank. There are good reasons for this approach. The justification for the continued existence of CGAP comes from its strong relevance, discussed in Chapter 3 (paras. 3.1­3.15). Despite all its accomplishments the microfinance industry has not yet reached a stage of universal maturity and it continues to have a number of gaps and constraints. While there are many stakeholders of the microfinance industry who are working to fill some of these gaps, there are some areas of common interest and public good where CGAP is best suited to play a critical role that no other stakeholder is likely to play as effectively. Therefore, at least, in the foreseeable future, there are strong reasons for CGAP 53. As stated in CGAP's Charter. However, the original document (M95-16, dated April 3, 1995) on the basis of which the Board of Executive Directors of the World Bank approved participation in CGAP was silent on this matter. 33 to remain in business in its current format and for the World Bank to support it. But before the end of CGAP's current strategy in 2013, after which CGAP would have been in existence for almost two decades, the Bank and its partners should start thinking about the long-term future of CGAP and the World Bank's role. In all probability, CGAP has a long-term role as an organization to serve the collective interest of the microfinance industry. However, the World Bank's involvement, as a fund provider as well as the host institution, is only justified as long as CGAP continues to produce public goods effectively and efficiently. 5. Lessons 5.1 This review concludes that CGAP has a high degree of relevance and it has performed efficiently. Over the years, CGAP has established itself as the respected voice and leader of the microfinance industry. It has developed and disseminated a high level of knowledge about the microfinance sector that did not exist when it was established. The nature and volume of CGAP activities and outputs have been consistent with its stated objectives, but there is not strong enough evidence to conclude that the desired outcomes have been fully achieved. These views largely mirror the findings of the 2007 Evaluation. This experience offers a number of lessons discussed below: · Some of the factors that contributed to the effectiveness of CGAP are of generic relevance to other global programs. First, CGAP's basic mission of helping to help build financial systems that work for the poor has been highly consistent with the broader development agenda. Since there were no arguments about the importance of microfinance, CGAP helped generate and galvanize support from all donors. Second, CGAP has enjoyed a high degree of stability throughout its existence, which is reflected in its organizational set up, governance structure and broad mission. There have been no sudden and unsettling changes. The original decision to house CGAP in the World Bank has also contributed to the stability. The World Bank has also contributed to the stability by selecting and retaining competent CEOs (in consultation with the Council of Governors) and staff. In its 12 years of existence, CGAP has had only two CEOs and the staff turnover rate has been very low. The clear definition of the respective roles and responsibilities of the components of governance structure and their revision, as and when warranted, has contributed to the smooth functioning of CGAP. Third, the pragmatic approach of the DGF towards enforcing its cost-sharing requirement has paid off. Gradually, other CGAP members have increased their contribution to close to the level required by DGF. A more sudden withdrawal or reduction in DGF funding could have derailed this progress. · Irrespective of the strong relevance and impressive performance of a GPP, it is important that a monitoring and reporting system be put in place to assess the outcomes and impacts of its activities. In the case of CGAP, there is no doubt that the program it has produced high quality outputs which are regarded as useful by many stakeholders. However, the extent to which microfinance contributes to alleviating poverty and CGAP's contributions towards such results need to be more firmly established.. 34 · The expected outcomes from a global program's activities aimed at improving the institutional set up and practices of donors and other fund providers should be kept at a modest level. As in the case of CGAP, such activities are laudable, but their effectiveness depends on two critical factors -- the capacity of a global program to work with fund providers and the capacity and willingness of the recipients. A host of factors affect the outcome, including the incentive systems within each entity, the importance of the relevant sector in the overall context of the entity's mission, the complexity of the organization, the nature of relationship with the GPP, etc. · The effectiveness of the policy and legal reform work of global programs depends to a large extent on effective follow-up on the recommendations as well as coordination with donors to ensure consistency with the ongoing broader sector policy dialogue. As in the case of CGAP, close coordination with donors such as the World Bank, especially those actively working on policy reforms, both for follow-up and consistency, contributes to the effectiveness of the policy work of global programs. · Locating a global program in the World Bank generally plays a positive role in its development. However, the World Bank and its partners need to reassess the World Bank's long-term role in each program from time to time. In the case of CGAP, there also is a possibility of it getting too closely involved with the internal management processes of the World Bank, which could generate perceptions of unfairness and inequity on the part of other CGAP members. 35 References Barr, Michael S, Anjali Kumar and Robert E. Litan, 2007. Building Inclusive Financial Systems: A framework for Financial Systems. Washington Dc: Brookings Institution Press. CGAP, 2003. CGAP Phase III Strategy, 2003­2008. Washington DC. ------, 2003. Is Microfinance An Effective Strategy to Reach the Millennium Development Goals? Focus Note No. 24. Washington DC: World Bank. ------, 2004. Scaling Up Poverty Reduction: Case Studies in Microfinance. Washington DC: World Bank. ------, 2005. Crafting Money Transfers Strategy: Guidance for Pro-Poor Financial services Providers. Occasional Paper No. 10. Washington DC: World Bank. ------, 2006. Using Technology to Build Inclusive Financial Systems. Focus Note No. 32. Washington DC: World Bank. ------, 2006. Graduating the Poorest into Microfinance: Linking Safety Nets and Financial Services. Focus Note No. 34. Washington DC: World Bank. ------, 2006. Aid Effectiveness in Microfinance: Evaluating Microfinance Projects of the World bank and the United Nations Development Programme. Focus Note No. 35. Washington DC: World Bank. ------, 2006. Community-Managed Loan Funds: Which Ones Work?. Focus Note No. 36. Washington DC: World Bank. ------, 2006. Safe and Accessible: Bringing Poor Savers into the Formal Financial System. Focus Note No. 37. Washington DC: World Bank. ------, 2006. Use of Agents in Branchless Banking for the Poor: Rewards, Risks, and Regulations. Focus Note No. 38. Washington DC: World Bank. ------, 2006. Financial Inclusion 2015: Four Scenarios for the Future of Microfinance. Focus Note No. 39. Washington DC: World Bank. ------, 2006. Beyond Good Intentions: Measuring the Social Performance of Microfinance Institutions. Focus Note No. 41. Washington DC: World Bank. ------, 2007. Sustainability of Self-Help Groups in India: Two Analyses. Occasional Paper No. 12.. Washington DC: World Bank. ------, 2008. Regulating Transformational Branchless Banking: Mobile Phones and Other Technology to Increase Access to Finance. Focus Note No. 43. Washington DC: World Bank. ------, 2006. CGAP 2006 Annual Report. Washington DC. ------, 2007. CGAP 2007 Annual Report. Washington DC. ------, 2007. Strategic Directions, 2008­2013. Washington DC. G8, G8 Action Plan: Applying the Power of Entrepreneurship to the Eradication of Poverty, New York, 2004. ------, Edinburgh Declaration, Edinburgh, June 2005 Helms, Brigit, 2006. Access for All: Building Inclusive Financial System. Washington DC: World Bank. Independent Evaluation Group, 2007. Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Washington DC: World Bank. Independent Evaluation Group ­ IFC, 2008. Financing Micro, Small, and Medium Enterprises: An independent Evaluation of IFC's Experience in Frontier Countries. Washington DC: World Bank. 36 Operations Evaluation Department, 2004. Addressing Challenges of Globalization: An Independent Evaluation of the World bank's Approach to Global Programs, CGAP Case Study. Washington DC: World Bank ------, 2005. OED Review of Bank lending for Lines of Credit. Washington DC: World Bank. UN Millennium Project 2005. Halving Hunger: It Can Be Done. Summary version of the report of the Task Force on Hunger. The Earth Institute at Columbia University, New York. United Nations 2007. The Millennium Development Goals Report, 2007. Ney York: the United nations Department of Economic and Social Affairs. United Nations Capital Development Fund, Microfinance and the Millennium Development Goals, New York, 2005 USAID, 2007. Microfinance Results Reporting: Annual Report to Congress, Fiscal year 2006. Washington DC: USAID. World Bank, 2007. Financial Sector Strategy for the World Bank Group. Washington DC: World Bank ------, 2008. Finance for All: Policies and Pitfalls in Expanding Access, a Policy Research Report. Washington DC: World Bank. 37 Annex A Annex A. Evaluation Framework for Global Program Reviews Note: This evaluation framework is a general framework that has been designed to cover the wide range of such programs in which the World Bank is involved, encompassing policy and knowledge networks, technical assistance programs, and investment programs. It is not expected that every global program review will cover every question in this table in detail. Annex Table 1. Assessing the Independence and Quality of the Evaluation Evaluation Questions 1. Evaluation process To what extent was the GRPP evaluation independent of the management of the program, according to the following criteria: · Organizational independence? · Behavioral independence and protection from interference? · Avoidance of conflicts of interest? Factors to take into account in answering these questions include: · Who commissioned and managed the evaluation? · Who approved the terms of reference and selected the evaluation team? · To whom the evaluation team reported, and how the evaluation was reviewed? · Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions on access to information, travel, sampling, etc.? 2. Monitoring and evaluation framework of the program To what extent was the evaluation based on an effective M&E framework of the program with: · Clear and coherent objectives and strategies that give focus and direction to the program? · An expected results chain or logical framework? · Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the program? · Systematic and regular processes for collecting and managing data? 3. Evaluation approach and scope To what extent was the evaluation objectives-based and evidence-based? To what extent did the evaluation use a results-based framework -- constructed either by the program or by the evaluators? To what extent did the evaluation address: · Relevance · Governance and management · Efficacy · Resource mobilization and financial management · Efficiency or cost-effectiveness · Sustainability, risk, and strategy for devolution or exit 4. Evaluation instruments To what extent did the evaluation utilize the following instruments: · Desk and document review · Consultations/interviews and with whom · Literature review · Structured surveys and of whom · Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being implemented or completed · Case studies · Other Annex A 38 Evaluation Questions 5. Evaluation feedback To what extent have the findings of the evaluation been reflected in: · The objectives, strategies, design, or scale of the program? · The governance, management, and financing of the program? · The monitoring and evaluation framework of the program? Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program Every review is expected to cover the first four criteria in the following table: (a) relevance, (b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover (e) resource mobilization and financial management and (f) sustainability, risk, and strategies for devolution or exit if the latter are important issues for the program at the time of GPR, and if there is sufficient information available on which to base an independent opinion. Evaluation Criteria and Questions Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. 1. Supply-side relevance -- the existence of an international consensus that global/regional collective action is required. To what extent does the program reflect an international consensus on the need for action, on the definition of the problem being addressed, on priorities, and on strategies for action? Is the original consensus that led to the creation of the program still present? Is the program still needed to address specific global/regional public concerns? Take into account the origin of the program in answering these questions: · Is the program formally responsible for implementing an international convention? · Did the program arise out of an international conference? · Is the program facilitating the implementation of formal standards and approaches? · Is the program primarily donor-driven? Did donors establish the program with little consultation with developing countries? · Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners? 2. Demand-side relevance -- alignment with beneficiary needs, priorities, and strategies. To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as articulated in the countries' own PRSPs, and in donors' strategies such as the World Bank CASs, and the UN Development Assistance Frameworks? To what extent has the voice of developing and transition countries been expressed in the international consensus underlying the program? 3. Vertical relevance -- consistency with the subsidiarity principle. To what extent are the activities of the program being carried out at the most appropriate level -- global, regional, national, or local -- in terms of efficiency and responsiveness to the needs of beneficiaries? To what extent are the activities of the program competing with or substituting for activities that individual donors or countries could do more efficiently by themselves? Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or local public goods. 39 Annex A Evaluation Criteria and Questions 4. Horizontal relevance -- the absence of alternative sources of supply. What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or technical capacity that is otherwise unavailable to meet the program's objectives? To what extent are the good and services being provided by the program in the nature of public goods? Are there alternative ways of providing these goods and services, such as by the private sector under regular market conditions? 5. Relevance of the design of the program To what extent are the strategies and priority activities of the program appropriate for achieving its objectives? What are the major activities of the program: · Policy and knowledge networking? · Financing country and local-level technical assistance? · Financing investments to deliver national, regional, or global public goods? (See Annex Table 7.) Has the program articulated an expected results chain or logical framework, along with assumptions that relate the progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to what extent are these assumptions valid? For programs providing global or regional public goods, is the design of the program consistent with the way in which the individual efforts of the partners contribute to the collective outcome for the program as a whole -- whether "best shot", "summation", or "weakest link?" Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. 6. Achievement of objectives To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made towards achieving these objectives? To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the program should also be held accountable? To what extent are there any positive, unintended outcomes of the program that have been convincingly document? To what extent have these assessments by the program or the evaluation been evidence-based? 7. Progress of activities, outputs, and outcomes. To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes? How did the program or the evaluation aggregate its outputs and outcomes at all levels -- global, regional, national, and local -- to provide an overall summary of its results? To what extent have factors such as changes in the location of the program, its legal structure, or governance processes affected the outputs and outcomes of the program? To what extent have there been outcomes that can be uniquely attributed to the partnership itself -- such as the scale of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host organization? 8. Linkages to country or local-level activities. To what extent has the program established effective operational linkages with country-level activities, taking into account that: · The desired nature of these linkages will vary according to the objectives, design, and implementation of each program? · Positive outcomes at the country or local level are generally a joint product of both global/regional and county- level activities? Annex A 40 Evaluation Criteria and Questions Efficiency or cost-effectiveness: Efficiency -- the extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results. Cost-effectiveness -- the extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. 9. Efficiency To what extent is it possible to place a monetary value on the benefits arising from the activities of the program? To what extent has the program or the evaluation conducted impact evaluations of representative program activities? To what extent has the program or the evaluation analyzed the program's costs in broad categories (such as overhead vs. activity costs), and categorized the program's activities and associated benefits, even if these cannot be valued in monetary terms? 10. Cost-effectiveness To what extent is the program measuring up against its own business plans: · Has the program cost more or less than planned? How did it measure up against its own costing schedule? · Have there been any obvious cases of inefficiency or wasted resources? To what extent is the program delivering its activities cost-effectively in comparison with alternatives: · How do actual costs compare with benchmarks from similar programs or activities? · Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the objectives and activities of the program? How does the program compare with traditional development assistance programs: · For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions costs compared with traditional development assistance programs? · For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and evaluation)? Governance and management: Governance -- the structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. Management -- the day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Whereas governance is concerned with "doing the right thing," management is concerned with "doing things right." 11. Compliance with generally accepted principles of good governance. To what extent are the governance and management structures and processes well articulated and working well to bring about legitimate and effective governance and management? To what extent do governance and management practices comply with the following seven principles: · Legitimacy -- the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large? · Accountability -- the extent to which accountability is defined, accepted, and exercised along the chain of command and control within a program, starting with the annual general meeting of the members or parties at the top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in some cases, to the beneficiaries of the program? · Responsibility -- the extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? · Fairness -- the extent to which partners and participants, similarly situated, have equal opportunity to influence 41 Annex A Evaluation Criteria and Questions the program and to receive benefits from the program? · Transparency -- the extent to which a program's decision making, reporting, and evaluation processes are open and freely available to the general public? · Efficiency -- the extent to which the governance and management structures enhance efficiency or cost- effectiveness in the allocation and use of the program's resources? · Probity -- the adherence by all persons in leadership positions to high standards of ethics and professional conduct over and above compliance with the rules and regulations governing the operation of the program? 12. Partnerships and participation To what extent has the program identified a complete list of stakeholders, or "stakeholder map", including the agreed- upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine programmatic function, updated regularly, and transparently available? Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is limited to financial and other contributors), or a stakeholder model (in which membership also includes non- contributors)? To what extent, if any, is the program's legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa? 13. Programs located in host organizations To what extent is the location of the program in the Bank or other partner organization adversely affecting the governance, management, or other aspects of the program, such as compliance with the principles of transparency and fairness? For which functions is the program manager accountable to the host organization and the governing body of the program, respectively? Are conflicts of interest being managed appropriately? To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the program objectively? Resource mobilization and financial management: Resource mobilization -- the processes by which resources are solicited by a program and provided by donors and partners. Financial management -- the processes that govern the recording and use of funds, including allocation processes, crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In cases where funds accumulate over time, this would also include the management of the cash and investment portfolio. 14. Resource mobilization To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from what sources -- the Bank, bilateral donors, foundations, etc.? To what extent has the program succeeded in diversifying its funding beyond a small number of donors? To what extent are the sources of funding for the program (including donor restrictions on the use of resources) affecting, positively or negatively: · The strategic focus of the program? · The outputs and outcomes of the program? · The governance and management of the program? · The sustainability of the program? 15. Financial management Are there any issues that have emerged during the course of the review in relation to: · The quality of financial management and accounting? · The methods, criteria, and processes for allocating funds among different activities of the program? · Financial management during the early stages of the program? Annex A 42 Evaluation Criteria and Questions Sustainability, risk, and strategy for devolution or exit: Sustainability -- When applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to which the organization or program is likely to continue its operational activities over time. Devolution or exit strategy -- a proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. 16. Sustainability of the benefits of the program's activities What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these changes actually materialize? 17. Sustainability of the program This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its continuity of effective management, and its ability to withstand changing market or other conditions. To what extent is there still a sufficient convergence or accommodation of interests among the major partners to sustain the program financially? To what extent has the program developed institutional capacity such as performance- based management, personnel policies, learning programs, and knowledge management that help to sustain a program? In what areas could the program improve in order to enhance its sustainability, such as better marketing of the program's achievements in order to sustain its reputation? 18. Prospects for continuation and strategies for devolution or exit To what extent should the program be sustained? Is the continuation of the program the best way of sustaining the results achieved? Should the design of the program be modified as a result of changed circumstances, either positive or negative? What other alternatives should be considered to sustain the program's results more cost-effectively, in the light of the previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability: · Reinventing the program with the same governance? · Phasing out the program? · Continuing country or local-level activities with or without devolution of implementation? · Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency on external sources? · "Spinning off" from the host organization? 43 Annex A Annex Table 3. Assessing the Bank's Performance as a Partner in the Program Evaluation Questions 1. Comparative advantage at the global/regional level. To what extent is the Bank playing up to its comparative advantages at the global/regional level -- its global mandate and reach and convening power? To what extent is the Bank's presence as a partner in the program catalyzing other resources and partners for the program? 2. Comparative advantage at the country level. To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the program? To what extent has the Bank's country operations established linkages to the GRPP, where appropriate, to enhance the effectiveness of both? 3. Oversight. To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as appropriate, whether the program is housed in the Bank or externally managed? To what extent is the Bank's oversight independent of the management of the program? To what extent does the Bank's representative on the governing body have a clear terms of reference? 4. Risks and risk management. To what extent have the risks associated with the program been identified and are being effectively managed? For example, IEG identified the following risks in its global review: · Bank bears a disproportionate share of responsibility for governing and managing in-house programs? · Confusion at the country level between global program activities, Bank activities, and Borrower activities? · Representation of NGOs and the commercial private sector on program governing bodies? · Unclear role and application of Bank's safeguards? · Trust-funded consultants and seconded staff representing the Bank on some program governing bodies? 5. Disengagement strategy. To what extent is the Bank engaged at the appropriate level in relation to the Bank's new strategic framework: · Watching brief? · Research and knowledge exchange? · Policy or advocacy network? · Operational platform? To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in relation to the Bank's objectives for its involvement in the program: · The program declares "mission accomplished" and closes? · The program continues and the Bank withdraws from all aspects of its participation? · The program continues and the Bank remains engaged, but the degree of the Bank's engagement in some or all aspects (such as financing) declines over time? Annex B 44 Annex B. Program Goals and Objectives Mission/Objectives Help build financial systems that work for the poor, providing large numbers of people with diverse services through a wide range of organizations. Strategic Priorities Promoting institutional diversity Promoting diverse financial services to a broad range of clients Building financial market infrastructure Fostering sound policy and legal frameworks Improving the effectiveness of microfinance funding Sources: 1. CGAP's Charter as amended in 2004; "CGAP Phase III Strategy, 2003-08," CGAP, January 2003. 45 Annex C Annex C. Program Timeline December 1993 The World Bank President presents the concept of CGAP to systematically increase the resources available to the very poor. July-August 1994 A Donor Working group appointed which recommended establishment of CGAP 1995 The World Bank's Board of Executive Directors approves a special grant of $30 million for CGAP. First Council of Governors meeting held. Nine founding members attended plus 13 observers, 12 of whom later joined CGAP as members. First Executive Director of CGAP appointed. 1998 First external evaluation of CGAP done. The Council of Governors extends CGAP for its Second Phase, FY99­03). 1999 First Executive Director retires and second appointed. 2002 The Council of Governors adopts CGAP's Charter. EXCOM created instead of Policy Advisory Group. 2003 The Council of Governors extends CGAP for Third Phase, FY04­08. 2004 CGAP Charter amended. 2005 CGAP Charter amended. 2006 CGAP Charter amended. 2007 The Council of Governors extends CGAP for its Strategic Directions, FY09­13. CGAP Charter amended. Annex D 46 Annex D. Members of the Governing Bodies54 1. Council of Governors55 A. BILATERAL MEMBERS Australia 1. Ms. Stuart Schaefer Australian Agency for International Development (AusAID) 2. Ms. Christine Groeger Australian Agency for International Development (AusAID) Canada Mr. Jonathan Rothschild Canadian International Development Agency Denmark 1. Ms. Birthe E. Larsen Royal Danish Ministry of Foreign Affairs 2. Mr. Morten Elkjaer Royal Danish Ministry of Foreign Affairs Finland Dr. Mika Vehnamaki Ministry for Foreign Affairs of Finland France 1. Ms. Aude Flogny-Catrisse Agence Française de Développement (AFD) 2. Mr. Jean-Nicolas Béasse Agence Française de Développement (AFD) 3. Ms. Martha Stein-Sochas Agence Française de Développement (AFD) 4. Ms. Florence Lasbennes Ministry of Foreign Affairs Germany 1. Ms. Gabriela Braun 54. As of May 13, 2008. 55. Some members are represented by multiple institutions/individuals, but each member has one vote. 47 Annex D Society for Technical Cooperation (GTZ) 2. Mr. Hanns Martin Hagen Kreditanstalt fur Wiederaufbau (KfW) 3. Ms. Cerstin Sander Kreditanstalt fur Wiederaufbau (KfW) 3. Mr. Matthias Adler Kreditanstalt fur Wiederaufbau (KfW) 4. Mr. Hendrik Denker Federal Ministry for Economic Cooperation and Development (BMZ) 5. Ms. Susanne Dorasil Federal Ministry for Economic Cooperation and Development (BMZ) Italy Mr. Filippo Scammacca del Murgo Ministry of Foreign Affairs Japan 1. Mr. Akihiro Kudo Ministry of Finance 2. Mr.Shintaro Nakagawa Ministry of Foreign Affairs of Japan 3. Mr. Minoru Homma 4. Mr. Yoshito Dobashi Japan Bank for International Cooperation 5. Ms. Mine Sato Japan Bank for International Cooperation Luxembourg 1, Ms. Anouk Agnès Ministry of Finance 2. Mr. Thierry Lippert Ministère des Affaires Etrangères The Netherlands 1. Mr. Hans van der Veen Ministry of Foreign Affairs 2. Mr. Frank Bakx Annex D 48 Netherlands Platform Microfinance Rabobank Foundation 2. Mr. Henk Nijland Netherlands Development Finance Company (FMO) 3. Mr. Maurice A.M. Scheepens Netherlands Development Finance Company (FMO) 4. Mr. Arthur Arnold Netherlands Development Finance Company (FMO) Norway 1. Mr. Parvez Kapoor Norwegian Agency for Development Cooperation (NORAD) 2. Ms. Bente Weisser Ministry of Foreign Affairs of Norway Spain 1. Mr. Manuel Cadalso Galárraga Agencia Española de Cooperación Internacional para el Desarrollo (AECID) 2. Dr. Guadalupe Rubio Agencia Española de Cooperación Internacional para el Desarrollo (AECID) Sweden Mr. Ola Sahlén Swedish International Development Cooperation Agency (Sida) Switzerland 1. Mr. Hans Ramm Swiss Agency for Development and Cooperation United Kingdom Ms. Catherine Martin Department for International Development (DFID) United States 1. Mr. Thomas Kennedy USAID 2. Mr. Conan French USAID B. MULTILATERAL MEMBERS African Development Bank 49 Annex D 1. Ms. Leila Farah Mokadem 2. Mr. Mohamed Alin Asian Development Bank Mr. Nimal Fernando European Commission 1. Mr. David Domes 2. Mr. Amadou Traore European Bank for Reconstruction and Development (EBRD) 1. Ms. Chikako Kuno 2. Ms. Maria Teresa Zappia European Investment Bank (EIB) 1. Mr. Cyrille Arnould 2. Mr. Edvardas Bumsteinas Inter-American Development Bank (IDB) / Multilateral Investment Fund (MIF) 1. Ms. Sandra H. Darville Coordinator Multilateral Investment Fund (MIF) 2. Mr. Dieter Wittkowski Inter-American Development Bank (IDB) International Finance Corporation (IFC) Mr. Martin Holtmann International Fund for Agricultural Development (IFAD) 1. Mr. Michael Hamp 2. Mr. Francesco Rispoli 3. Ms. Jamie Anderson International Labour Office (ILO) 1. Mr. Bernd Balkenhol 2. Mr. Craig Churchill United Nations Capital Development Fund (UNCDF) / UNDP 1. Mr. Henri Dommel 2. Mr. John Tucker Annex D 50 World Bank Mr. Michael Klein, CG Chair C. FOUNDATIONS The Ford Foundation Mr. Frank DeGiovanni Michael & Susan Dell Foundation 1. Ms. Caitlin Baron 2. Ms. Geeta Dutta Goel Bill & Melinda Gates Foundation 1. Ms. Priya Jaisinghani 2. Ms. Amolo Ng'weno 3. Mr. Lawrence Yanovitch 4. Mr. Robert Christen Omidyar Network Ms. Crystal Hutter 2. Executive Committee Council of Governors Representatives 1. Mr. Frank DeGiovanni, Chair The Ford Foundation 2. Ms. Gabriela Braun Society for Technical Cooperation (GTZ) 3. Mr. Martin Holtmann IFC (World Bank Group) 4. Mr. Jonathan Rothschild Canadian International Development Agency 5. Ms. Martha Stein-Sochas Agence Française de Développement (AFD) Members at Large (Industry Representatives) 1. Mr. Fouad Abdelmoumni Association Al Amana 2. Mr. Robert Annibale Citigroup Microfinance Group, UK 3, Ms. Renée Chao-Béroff Centre International de Développement et De Recherche (CIDR) 4. Mr. Vijay Mahajan BASIX 51 Annex D Ex-Officio Ms. Elizabeth Littlefield Executive Director, CGAP 3. Investment Committee 1. Michael Klein, Chair Vice President Financial & Private Sector Development World Bank 2. Simon Bell Sector Manager Finance & Private Sector Development South Asia Region World Bank 3. Deepak Bhattasali Lead Economist Poverty Reduction & Economic Management East Asia Region World Bank 4. Gerard Byam Director Operational Quality & Knowledge Africa Region World Bank 5. Mamta Shah Director International Finance Corporation 6. Antony Thompson Operations Adviser Human Development Network World Bank 7. Laurence Carter Director Business Advisory services International Finance Corporation 8. Junaid Ahmad, Sector Manager Sustainable Development World Bank Annex D 52 9. Tom Duval Chief Counsel World Bank 10. Frank DeGiovanni (EXCOM Observer) Ford Foundation 11. Donald Terry (Council of Governors Observer) Multilateral Investment Fund 53 Annex E Annex E. CGAP: Sources and Uses of Funds ($/000) Total Fiscal Year 2003 2004 2005 2006 2007 Total Phase III Sources of Funds: Contributions from donors (net) 13041 12591 15615 13529 19086 73863 60822 Interest income 626 210 778 1152 2111 4876 4249 Foreign exchange gain 154 154 154 Total Sources 13668 12801 16393 14681 21350 78893 65225 Uses of Funds: Improving the Effectiveness of 556 1300 1169 316 579 3919 3364 Funding Promoting Diverse Financial 6406 870 2129 982 1193 1233 5536 Services Promoting Institutional Diversity 5663 2729 210 280 176 9058 3395 Fostering Sound Policy & legal 547 6525 1755 1860 10687 10687 Frameworks Building Financial Market 426 2045 1778 4483 6311 15043 14617 Infrastructure Communications 1397 1891 2276 1508 1347 8418 7022 Training 2505 4915 90 309 379 8199 5693 Total Initiatives 11416 15556 13030 9843 11885 61730 50314 Other Expenses: Staff salaries and benefits 2616 2491 3009 3382 2729 14227 11611 Office and occupancy costs 884 1233 1145 1698 1311 6271 5387 Evaluation and monitoring 240 240 240 Project preparation and rapid 455 455 455 response CGAP Internship 410 410 410 Representation, member services 1619 123 203 362 455 476 1496 and training CG/Excom meetings 208 143 154 177 275 957 750 Foreign exchange losses 32 32 32 Administration fee 90 163 83 410 531 1276 1186 Total Other Expenses 3921 4232 4753 6154 6427 25486 21565 Total Expenses 15338 19788 17783 15997 18311 87217 71879 EXCESS OF REVENUES OVER -1670 -6987 -1389 -1316 3039 -8324 -6654 EXPENSES Operating Reserves at Beginning of 18263 16593 9606 8217 6901 18263 16593 Period Operating Reserves at End of Period 16593 9606 8217 6901 9940 9940 9939 Annex E 54 Total Fiscal Year 2003 2004 2005 2006 2007 Total Phase III REGIONAL DISTRIBUTION OF INITIATIVES Africa 2686 4371 1980 14 634 9685 6999 ECA 606 145 98 849 849 EAP/SAR 1993 971 498 3462 1469 LCR 90 180 270 270 MNA 85 890 1004 1979 1979 Global 6737 9433 9662 8140 11153 45125 38388 TOTAL 11416 15556 13030 9483 11885 61370 49954 Global % of Total 59% 61% 74% 86% 94% 74% 77% Source: CGAP 55 Annex F Annex F. Persons Consulted Members of CGAP Council of Governors: 1. Jamie Anderson, International Fund for Agricultural Development 2. Bernd Balkenhol, International Labor Office 3. Conan French, United States Agency for International Development 4. Priya Jaisinghani, Gates Foundation 5. Michael Klein, World Bank CGAP EXCOM Members: 1. Brian Branch, World Council of credit Unions 2. Gabriele Braun, Society for technical Cooperation (GTZ), Germany 3. Carlos Cuevas, World Bank 4. Frank DeGiovanni, Ford Foundation 5. Marilou van Golstein Brouwers 6. Carlos Labarthe, Financiera Compartamos 7. Vijay Mahajan, BASIX, India 8. Jonathan Rothschild, Canadian International Development Agency 9. Martha Stein-Sochas, Agence Française de Développement CGAP Investment Committee Members: 1. Simon Bell Sector Manager Finance & Private Sector Development South Asia Region World Bank 2. Tom Duval Chief Counsel World Bank 3. Antony Thompson Operations Adviser Human Development Network World Bank 2007 Evaluation Team: 1. Klaus Maurer 2. Sarah Forster 3. Michael Mithika CGAP Managers & Staff: 1. Elizabeth Littlefield 2. Jennifer Isern 3. Gautam Ivatury 4. Alexia Latortue 5. Tim Lyman Annex F 56 6. Kate McKee 7. Rich Rosenberg 8. Ousa Sananikone 9. Jeanette Thomas 10. Tonia Wright Former CGAP Managers 1. Ira Lieberman (Executive Director) 2. Mohini Malhotra (Manager) World Bank Group Managers & Staff: 1. Sonja Brajovic-Bratanovic Lead Financial Sector Specialist Private/Financial Sector Development Europe & Central Asia Region 2. Lily Chu Sector Manager Finance & Private Sector Latin America & Caribbean Region 3. Asli Demerguc-Kunt Senior Research Manager Development Research Group 4. Aurora Ferrari Senior Private Sector Development Specialist Finance & Private Sector Development South Asia Region 5. Michael Goldberg Senior Private Sector Development Specialist Finance & Private Sector Latin America & Caribbean Region 6. Martin Holtmann Head, Microfinance Group IFC 7. Anjali Kumar Adviser FPD Financial Access Financial & Private Sector Development Vice Presidency 57 Annex F 8. Korotoumou Ouattara Sr. Financial Economist Finance & Private Sector Development Africa Region 9. Rahul Raturi Sector Manager Rural Development, Natural resources & Environment East Asia & Pacific Region 10. Tunc Uyanik Sector Manager Financial & Private Sector East Asia & Pacific Region 11. Richard Zechter Senior Partnership Specialist Global Programs & Partnerships Annex G 58 Annex G. Response of the Program to IEG's Global Program Review Dear Mark Sundberg, First of all, thank you very much for sharing with us this first draft of IEG's Global Program Review of CGAP. We want to thank both the IEG team and Khalid Siraj for the thoroughness, dedication, and collegial spirit with which he conducted this review. We have reviewed the report and appreciate its recognition of CGAP's strong performance as a global resource center for the microfinance industry. We are proud of what we have accomplished and it is an important boost to the CGAP members and operational team to have our work commended by IEG. The report also raises important questions about CGAP's role and impact -- questions which we ourselves have been working on addressing over the years. There are two main issues we would like to raise in this response to the evaluation: (1) CGAP's aid effectiveness work, and (2) our monitoring and evaluation system. Please note that our Excom has not yet been able to complete their review and you will be receiving a short note from our Excom Chair next week as well. Improving the Effectiveness of Microfinance Funding. We feel the report does not adequately capture the rationale, evolution, and impact of CGAP's aid effectiveness work. What the report denotes as "rapid changes in [aid effectiveness] activities" has been, in fact, a careful sequencing of engagements and outputs based on demand from our member donors. Following the completion of the Donor Peer Reviews, CGAP's engagements with its member donors -- from diagnostic reviews to tailored advisory services -- have been selective and results-focused, building on CGAP's deepening relationship with the agency's microfinance focal point, senior management and field staff. A number of aid agency leaders have provided powerful testimonials on the impact of CGAP's work with their agencies, such as specific changes in microfinance policies and procedures. The report also states, incorrectly, that "CGAP has been somewhat optimistic with respect to the demand for its institutional work". In reality, CGAP receives many more requests for assistance than it can accommodate, and these requests are carefully screened for strategic importance and potential for impact. CGAP indeed is forced to turn down many requests or refer them to consultants; for example, we declined a grant from IFAD to advise their MENA microfinance team. We have attached two documents that may shed light on this matter: one is a two page list of testimonials from agency leaders about CGAP's Peer Reviews and Country Level Reviews. The second is a zip file with individual agency reports tracking concrete results resulting from CGAP's work. Monitoring & Evaluation. We fully agree with the report that CGAP needs to make greater efforts to develop a Monitoring & Evaluation system that assesses outcomes and not just outputs. We appreciate this recommendation and are now working to identify practical and meaningful indicators to implement it. We're still in the middle of that process, and will have 59 Annex G a clearer idea of what's feasible when we get further along. But think it's important to keep expectations realistic. Given that CGAP's role is based on advocacy, standard setting, and knowledge creation and dissemination, rather than implementation on the ground, these outcomes will be best evidenced by results at the intermediate level -- policies changed, advice adopted, etc. This evidence of such positive outcomes can be supplemented by testimonials, client surveys and evaluations. However, even then attribution will be very hard to determine. It may not be possible, nor even appropriate, to attempt to measure impact with attribution when multiple intermediary parties are involved in achieving the results that will serve the final poor client. We also hesitate to focus too much on proving and claiming attribution since much of what we do is through partnerships and contributing to public goods. We would welcome any concrete and feasible recommendations that IEG or the evaluators might have as to indicators or methods that would address their call for "an in-depth analysis of the causal relationship between CGAP outputs and developments in the industry" (see summary, page xii). We ourselves have not been able to point to any other organizations similar to CGAP, or to CGAP's think-tank function, that have found it possible to measure their contribution to global, or even country, change; that said, we would welcome any suggestions you might have as to other institutions that might have models we could apply. By focusing on intermediate level outcomes and qualitative assessments, we believe we can and will develop a more meaningful and systematic monitoring and evaluation program that is both truthful as well as useful and practical. In line with the report's recommendations, we are working on a results framework that will guide the implementation of our new five-year strategy; track, monitor and evaluate our ongoing work; and collect output as well as outcome data that is useful to management and funders and can be easily used for multiple stakeholders and reporting requirements. The new results framework will clearly delineate between broad industry-level goals and the more specific CGAP-level objectives and outcomes. In addition, we plan to conduct annual client surveys and external evaluations of CGAP activities in each of the four priority areas. CGAP is committed to implementing a pragmatic M&E system that provides a solid basis against which success can be measured over the next five years. Thank you again for the very thoughtful and valuable evaluation. Best regards, Elizabeth Littlefield CGAP Director The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor GPR Established in 1995, the Consultative Group to Assist the Poor (CGAP) is a consortium of public and private agencies located in the World Bank with the mission of expanding access to financial services for the poor. During its third phase (2004­08), CGAP pursued five strategic priorities: (a) promoting institutional diversity, (b) promoting diverse financial services to a broad range of clients, (c) building financial market infrastructure, (d) fostering sound policy and legal frameworks, and (e) improving the effectiveness of microfinance funding. Its activities focused on providing advisory services, developing and setting stan- dards, advancing knowledge, and training and capacity building. An external evaluation, con- ducted in 2006­07, concluded that CGAP is a powerful and pivotal force in the microfinance field, playing a critical role in helping to build inclusive financial systems. IEG's Global Program Review (GPR) also found that CGAP has strong relevance and effective gover- nance. CGAP's achievements in terms of activities and outputs were also impressive during 2004­08, but there is little systematic evidence relating to its achievements at the outcome level, or their attribution to CGAP's activities due to weaknesses in its monitoring and report- ing system. Therefore, both the external evaluation and the GPR concluded that CGAP needs to do more with regard to measuring, monitoring, and evaluating its outcomes. CGAP's Council of Governors has recently approved a new strategic plan for 2009­13, including improvements to its monitoring system.