MALI Project to Support Grassroots Initiatives to Fight Hunger and Poverty Report No. 123411 FEBRUARY 14, 2018 © 2017 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2017. Mali—Project to Support accuracy of the data included in this work. The Any queries on rights and licenses, including Grassroots Initiatives to Fight Hunger and boundaries, colors, denominations, and other subsidiary rights, should be addressed to Poverty. Independent Evaluation Group, information shown on any map in this work do World Bank Publications, The World Bank Project Performance Assessment Report not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC 123411. Washington, DC: World Bank. World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 123411 PROJECT PERFORMANCE ASSESSMENT REPORT MALI PROJECT TO SUPPORT GRASSROOTS INITIATIVES TO FIGHT HUNGER AND POVERTY (CN-037; CP957) February 14, 2018 Financial, Private Sector, and Sustainable Development Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = CFAF 1998 US$1.00 CFAF 562 1999 US$1.00 CFAF 650 2000 US$1.00 CFAF 705 2001 US$1.00 CFAF 744 2002 US$1.00 CFAF 633 2003 US$1.00 CFAF 525 2004 US$1.00 CFAF 482 2005 US$1.00 CFAF 550 2006 US$1.00 CFAF 550 2007 US$1.00 CFAF 498 2008 US$1.00 CFAF 452 2009 US$1.00 CFAF 458 2010 US$1.00 CFAF 458 2011 US$1.00 CFAF 490 2012 US$1.00 CFAF 507 2013 US$1.00 CFAF 496 2014 US$1.00 CFAF 477 Abbreviations and Acronyms ANICT National Investment Agency for Territorial Collectivities (of Mali) CAS country assistance strategy CDD community-driven development ICR Implementation Completion and Results Report IDA International Development Association IEG Independent Evaluation Group M&E monitoring and evaluation NGO nongovernmental organization PACR Projet d’Appui aux Communautés Rurales (Rural Community Development Project) PAIB Grassroots Hunger and Poverty Initiative Project PMU Project Management Unit PPAR Project Performance Assessment Report All dollar amounts are U.S. dollars unless otherwise indicated. Fiscal Year Government: January 1–December 31 Director-General, Independent Evaluation Ms. Caroline Heider Director, Financial, Private Sector, and Sustainable Development Mr. José Carbajo Martínez Manager, Sustainable Development Ms. Midori Makino Task Manager Ms. Lauren Kelly iii Contents Principal Ratings ....................................................................................................................... v Key Staff Responsible............................................................................................................... v Preface..................................................................................................................................... vii Summary ................................................................................................................................ viii 1. Background and Context....................................................................................................... 1 2. Objective, Design, and their Relevance ................................................................................ 2 Relevance of Objective ......................................................................................................... 2 Project Design ....................................................................................................................... 3 Relevance of Project Design ................................................................................................. 3 Implementation ..................................................................................................................... 4 3. Efficacy ................................................................................................................................. 5 4. Efficiency .............................................................................................................................. 9 5. Ratings ................................................................................................................................ 11 Outcome .............................................................................................................................. 11 Risk to Development Outcome ........................................................................................... 11 World Bank Performance ................................................................................................... 13 Quality at Entry ............................................................................................................... 13 Quality of Supervision .................................................................................................... 14 Borrower Performance ........................................................................................................ 14 Government Performance ............................................................................................... 14 Implementing Agency Performance ............................................................................... 15 6. Lessons ................................................................................................................................ 15 References ............................................................................................................................... 17 Tables Table 3.1. Public and Private Services Financed by PAIB ....................................................... 6 Table 3.2. Net Attendance Rates in Mopti, 2001 and 2006 ...................................................... 7 Table 3.3. Literacy Rates in Mopti, 2001 and 2006.................................................................. 8 Table 3.4. Production Increases from Small-Scale Irrigation Subprojects ............................... 9 Table 4.1. Average Yearly Operating Accounts for Selected Physical Investments .............. 10 This report was prepared by Lauren Kelly, Senior Evaluation Officer and Amber Stewart who assessed the project in March 2017. The report was peer reviewed by Dr. Leif Brottem, Assistant Professor at Grinnell College, and panel reviewed by Christopher Nelson, Senior Evaluation Officer, IEG. Vibhuti Khanna and Jean- Jacques Ahouansou provided administrative support. iv Figure Figure 3.1. Birth Assistance in Mopti ....................................................................................... 8 Appendixes Appendix A. Basic Data Sheet................................................................................................ 19 Appendix B. List of Persons Met ............................................................................................ 21 Appendix C. Borrower Comments ......................................................................................... 23 v Principal Ratings ICR* ICR Review* PPAR Outcome Satisfactory Moderately Moderately Unsatisfactory Satisfactory Risk to Likely Significant High Development Outcome (*Sustainability) Institutional Substantial Modest N/A Development Impact World Bank Satisfactory Unsatisfactory Moderately Performance Unsatisfactory Borrower Satisfactory Satisfactory Satisfactory Performance * The Implementation Completion and Results (ICR) report is a self-evaluation by the responsible World Bank global practice. The ICR Review is an intermediate IEG product that seeks to independently validate the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Adriana de Leva Roger Sullivan Hasan Tuluy Completion Daniel Moreau Mary Barton-Dock David Craig vi IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About this Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank country management unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the World Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, and Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. World Bank Performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for World Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. vii Preface This is a project performance review of the Grassroots Hunger and Poverty Initiative Project (PAIB) financed by the International Development Association (IDA) and implemented between 1998 and 2004 across two regions of Mali (Mopti and Tombouctou). Original financing was anticipated to be $23 million, including a $21.5 million IDA credit and $1.5 million borrower contribution. Actual costs were $23.2 million. The project sought to improve the living conditions of disadvantaged targeted rural communities, responding to their priority needs by strengthening the capacity of communities in identifying and ranking their priority needs and in planning, implementing, and supervising actions to respond to those needs in partnership with nongovernmental organizations (NGOs) and local authorities. In parallel, it also sought to strengthen institutional and policy-making capacity at the local and national levels in the fight against hunger and poverty. Methodology. This assessment was based on a review of World Bank project documentation, supplemented by several sources of primary and secondary data collected during a field mission to Mali conducted between May 8 and May 30, 2017. PAIB was a pilot project that tested new participatory implementation modalities for service delivery in two regions of Mali before decentralization structures were fully put in place. Implemented more than a decade ago, this review was conducted to identify lessons about how the World Bank contributed to the development of Mali’s broader decentralized service delivery process as it was formalized and rolled out. This assessment faced severe limitations in accessing project sites: travel to both project regions was prohibited at the time of the mission because of active conflict in those areas. As such, the evaluation team worked with local field agents who had access to these areas to conduct a series of key informant interviews with NGOs that implemented the project and that continue to support service delivery in Mopti and Tombouctou today. Lauren Kelly, Senior Evaluation Officer, Independent Evaluation Group (IEG) Sustainable Development, prepared the report with support from Amber Stewart, consultant. Moussa Sacko, local consultant, supported the fieldwork in Mopti. Leif V. Brottem, Assistant Professor of Global Development Studies at Grinnell College, was the peer reviewer for the report, and Chris Nelson, Senior Evaluation Officer, IEG, was the panel reviewer. The mission is grateful to Maimouna Abdoulaye Dite Koura Diarra and Moussa Sidibe for their excellent country office support. Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback but no comments were received. viii Summary The incidence of poverty in Mali fell from 55.6 percent in 2001 to 43.6 percent in 2010, but it relapsed to 45 percent in 2013 after heavy conflict in 2012 and a series of droughts (World Bank 2016). A conflation of factors has perpetuated poverty in Mali and its relative stagnation, including conflict, a dependence on subsistence agriculture complicated by climatic conditions, and a low level of educational attainment. Almost 60 percent of Mali’s population is uneducated, and in the northern regions of Mopti, Tombouctou, and Kidal, more than 80 percent of the population has not attended school (World Bank 2015). Mali has shown some improvement in its overall economic and social welfare since the World Bank piloted its first community-driven development (CDD) there (the Grassroots Hunger Campaign from 1998–2004), but most of Mali’s rural population still lacks access to affordable and sustainable basic services. The World Bank introduced a CDD approach just after the country held its first democratic election in 1992, formulated its decentralization strategy, and established 682 new rural communes. The Grassroots Hunger Campaign sought to influence the decentralization strategy by modeling a participatory approach to local decision making. Relevance of project objectives is Substantial. Specifically, the project sought to accomplish the following: (i) improve the living conditions of disadvantaged targeted rural communities by responding to their priority needs; (ii) strengthen the capacity of communities in identifying and ranking their priority needs and in planning, implementing, and supervising actions to respond to those needs in partnership with nongovernmental organizations (NGOs) and local authorities; and (iii) strengthen institutional and policy-making capacity at the local and national levels. The objective of improving service delivery through participatory planning was and still is a substantially relevant objective. Relevance of project design is Modest. The Grassroots Hunger Campaign was designed to support a more participatory, NGO-led model of service delivery aimed at expanding rural services to more remote areas. The project was implemented alongside a wave of decentralization reforms characterized by sharp shifts in the legal and institutional landscape. During project implementation, the government established local communal seats of government and subsequently transferred to them all responsibility for the delivery of health, education, and water, although the transfer of responsibility significantly lacked (and still lacks) sustainable sources of financing. Also during this time, the government of Mali set up the National Investment Agency for Territorial Collectivities (ANICT) to oversee the construction of communal infrastructure. Thus, although the pilot was relevantly developed to test new modalities for local service, by the end of the project, the relevance of project design (for example, the NGO-led service delivery model) was weakened because of the roles the national investment agency and communes were expected to play. A follow-on World Bank project, the Rural Community Development Project (2006–14), adopted some of the private investment mechanisms, but the public service design was altered radically with the launch of ANICT and its allocation criteria. The swiftly changing landscape undermined the relevance of a pilot for testing and scaling new service modalities. Design was also overambitious: it sought to influence the national collection and use of poverty data, ix which would have required different incentives and policy support than those in project design. Results. The project modestly improved the living conditions of disadvantaged targeted rural communities through the provision of public infrastructure and small private agricultural investments. Although this assessment recognizes the critical welfare benefits of the provision of basic public services (including water and health care), the lack of a results measurement system undermines the ability of this assessment to validate welfare gains attributable to the project investments. The project used a single indicator to measure improvement in living conditions: the percent of persons satisfied with the outcome of subprojects. The project reported a 100 percent rate of satisfaction. It is not clear how many of the 170,000 people the project reportedly reached were surveyed or how a 100 percent satisfaction rate was obtained. At the output level, the project surpassed its subproject goal. It financed 287 socioeconomic investments—more than the target of 260 investments. Most of the investments were made in public services (69 percent), including potable water, education, and health. The rest of the investments were for private productive infrastructure: small-scale irrigation and agricultural equipment, plus a few in market construction and feeder roads. Public Services. Potable water was the most frequently occurring communal investment. This assessment was unable to validate the durability of infrastructure outcomes because of security restrictions. Therefore, it relied on field studies conducted by WaterAid Mali of a representative sample of water investments in the World Bank’s project area to provide an approximate estimate of the durability of the World Bank’s investments. WaterAid’s field studies in Mopti reveal that the installation of water-pumping technologies would have improved water quality more than open wells would (open wells are highly vulnerable to contamination). The studies found that over time, hand pumps are more durable than motorized or solar schemes because of the complex management needs of the latter (for example, parts, repair, and finance). However, the World Bank’s pilot directed two-thirds of its water investment to solar pumping systems. WaterAid estimates that the lifespan of solar pumping technologies is two to three years. Therefore, it is unlikely that the World Bank’s investments are yielding sustained outcomes. For all public investments, project reporting is clear that although communities might have been able to bear the operating costs of the communal infrastructure, they would have been unable to cover the costs of amortization, including for basic equipment repair. Private Investments. Production increases resulted from the productive investments, and these are attributable to a combination of land expansion and increased water availability. However, the gains are marginal, and few people would have benefited from these investments. Forty- seven investments were directed toward small-scale irrigation programs for activities such as the expansion of vegetable gardens. Based on a sample, the economic rate of return analysis shows that for each irrigated plot of 1–2.5 hectares, 66–126 people were expected to benefit. Efficiency is Modest. Projected net earnings on all investments except for hand pumps are estimated to be negative because of communities’ inability to pay for amortization costs. The amount of cofinancing communities could afford or were willing to pay was overestimated, undermining the viability and financial sustainability of the services put in place. x World Bank performance is Moderately Unsatisfactory. With this highly innovative pilot project, the World Bank helped to introduce a participatory planning process just as the country rolled out its decentralization reforms. Although these efforts are noteworthy, the World Bank gave less attention to the financial and economic viability of the rural socioeconomic investments that were scaled up in a second phase. The transfer of task team leaders and World Bank staff, along with transferring the project from one department to another, caused delays. A midterm correction that adjusted the project’s focus from socially oriented one to one focused on rural investments did not include a change in the monitoring and evaluation (M&E) system to support more relevant data gathering. With a renewed focus on economic activities, including irrigation and other agricultural activities, the safeguard category should have been revised to reflect these increased environmental risks. Borrower performance is Satisfactory. The government of Mali showed a strong commitment to expanding rural service delivery, including through participatory planning. However, delays in the disbursement of the government’s contribution caused some financial difficulty for the Project Management Unit (PMU) and delays in appointing teachers and health professionals to staff newly built infrastructure. The PMU helped bridge the divide between the newly created communes and the NGO-led activities that the project supported. The government also showed overall compliance with the World Bank’s fiduciary and procurement procedures. Risk to development outcomes is High. In Mali, protracted and structural constraints are associated with rural service delivery. Rural institutions in Mali require a higher level of financial assistance from the central government, and the ANICT transfer mechanism has been disappointing in this regard. A new law (Loi 2017-052) stipulating that 30 percent of public finances will be transferred to communes is encouraging, but it has not been implemented. Even so, those risks will not be mitigated fully until Mali tackles its regressive local tax structure whereby local authorities collect a nominal flat rate for each member of a household. Some development partners like the International Monetary Fund are calling for tax reform, and until such reform takes place, communes will be financially constrained and dependent on unpredictable forms of external aid. Risk was also rated high because of the ongoing conflict in the north, which has interrupted service delivery in the project areas. The UN Office for the Coordination of Humanitarian Affairs estimates that 501 schools have closed in and around the project areas, resulting in a minimum of 150,000 children leaving school. Insecurity has spread and taken hold in the Mopti Region since IEG conducted this assessment, causing its public institutions to stop functioning in many communes. It will likely take years for those communities to recover when the situation stabilizes. However, the crisis led to the signing of the Algiers Peace Accord, which was the impetus behind the latest decentralization measures, including the passage of Law 2017-052. Social risks, including exclusion and a lack of voice in local development planning and priority setting, are also high. Many of the social innovations will not endure without an institutionalized process for negotiation and a voice in the identification of needs, budgeting, and planning. A key example is the expiration of many of the participatory Plan de Développement Economique et Social de la Commune (Local Economic and Social Development Plans), and the lack of follow-on financing to update them. xi Lessons • Projects that seek to “improve living conditions” need to define, benchmark, and measure a project’s attributable contribution to changes in human welfare. Measuring access to assets is an insufficient metric. One tool that could be utilized by World Bank rural development projects is the Aga Khan Foundation’s Quality of Life Assessment, which has been rolled out and used in their rural development programs. • Private productive investments should not be relied on to finance the operations and maintenance of core public services provided by World Bank–financed projects when local revenue recovery is not feasible. The returns on investment for micro and small enterprise development are not likely to contribute to significant autonomy of public finance in rural areas in the short to medium term without major advances in productivity, market connectivity, and consumer demand. Project objectives should focus on the marginal but important household income and revenue benefits that can accrue while acknowledging that financially sustainable public infrastructure requires a more conducive policy environment at higher levels (region and central government). • A pilot project should be designed to generate lessons: its M&E system should therefore double as a learning lab to test and adapt interventions within the country and sector context. PAIB was a pilot project, but its M&E system was limited mainly to output indicators. There were no “tests” or efforts to adapt the project based on learning along the way, and no parallel or special studies conducted to learn why these innovations worked or didn’t work, with the aim of scaling what worked in a second phase. José Carbajo Martínez Director, Financial, Private Sector, and Sustainable Development Department 1 1. Background and Context 1.1 The incidence of poverty in Mali fell from 55.6 percent in 2001 to 43.6 percent in 2010, but after heavy conflict in 2012 and a series of droughts during the past decade, it relapsed to 45 percent in 2013 (World Bank 2016). Poverty in Mali and its relative stagnation has been perpetuated by a conflation of factors, including the following: • Mali’s rural population is dependent on subsistence agriculture, with little to no investment in mechanization. Three-quarters of Mali’s population are rural, and 90 percent of this population depends on the agriculture sector. Changing climatic conditions, including more frequent drought and erratic rainfall, have complicated farm investment decisions. • Conflict, especially in the north, has had a negative impact on hunger, nutrition, welfare, and health. The most recent conflict in Mali, sparked by the Tuareg Rebellion of 2012 and the ensuing al-Qaeda insurgency, followed a third consecutive regional drought. Combined, these adverse conditions have since taken a heavy toll on food security. • Mali has a low level of educational attainment: Almost 60 percent of the population ages six and above has no education at all, approximately 35 percent has primary education, and less than 8 percent has secondary education or higher (World Bank 2015). In the northern regions of Mopti, Kidal, and Tombouctou, more than 80 percent of the population has not attended school at all (World Bank 2015). 1.2 Although Mali has shown some improvement in its overall economic and social welfare since the World Bank piloted a Grassroots Hunger Campaign in the late 1990s, current data continue to paint a dire picture of the need for more effective service delivery to Mali’s most extreme poor, rural, and conflict-affected areas. Set against a backdrop of political firsts—including a democratic election (1992), the formulation of a decentralization strategy, 1 and the establishment of 682 new rural communes (1999)—the Grassroots Hunger Campaign was designed to test new service delivery modalities to reduce poverty, in the areas of health, education, water, and to a local extent, local economic investment. 1.3 Mali officially transferred all responsibility for health, education, and water to the municipal level just before the project closed. A follow-on World Bank project, the Rural Community Development Project (2006–14), adapted some of the pilot’s modalities and implemented a similar community driven development (CDD) initiative at scale. However, both projects were implemented during a period characterized by swiftly changing intuitional responses to the newly established decentralization process. As such, elements of the pilot were adapted and scaled, but the follow-on project directed a substantial amount of its finance for social services through country systems that were tested through this pilot project, and this prevented a broader analysis of the effectiveness of the World Bank–financed nongovernmental organization (NGO) model over time. 2 2. Objective, Design, and their Relevance 2.1 Project Development Objective. The project had one main objective and two subordinate objectives. The project’s main objective was to improve the living conditions of disadvantaged targeted rural communities by responding to their priority needs. The project objective also included a statement on how this objective would be achieved. It would (i) strengthen the capacity of communities in identifying and ranking their priority needs and in planning, implementing, and supervising actions to respond to those needs in partnership with NGOs and local authorities, and (ii) strengthen institutional and policy-making capacity at the local and national levels in the fight against hunger and poverty. 2.2 Theory of Change. The project’s theory of change posited that greater community ownership of local development—along with increased collaboration between empowered citizens, local authorities, NGOs, and a better-informed government—can generate more relevant, cost-effective, and sustainable local investments that lead to increased access to services and improved living conditions than does top-down development. In this way, the project’s main overarching objective was an improvement in living conditions, which the two subobjectives of community development and strengthened institutional and policy-making capacity support. This theory of change was based on four guiding principles: • Community participation is essential to make sustained progress in poverty and hunger alleviation. • Stronger relationships and increased dialogue between the Malian government, NGOs, and communities can contribute to this goal. • Capacity building must focus on the community and involve all project stakeholders. • The project’s impact must be measured by both process and performance indicators. Relevance of Objective 2.3 The provision of basic decentralized services to rural areas—and increased citizen engagement and ownership of their selection and maintenance—was a key theme of the Malian government’s development objectives as embodied in the country assistance strategies (CAS) at the time of project design. These CASs highlighted the need for improved service provision (health and education), increased agricultural production, and support for capacity building and decentralization. Enhanced dialogue between the government and civil society was also a key element of the CASs. Decentralized service delivery was a pillar of the 2002 poverty reduction strategy. Attention to strengthened service delivery was still prominent in the CAS at closing (FY04–06). However, as subsequent CASs have indicated, despite project aims to improve living conditions through more meaningful citizen participation in rural services and productive investments, a lack of attention to accountability (both in objectives and design) have constrained achievements in this area. As of 2018, the objective of improving rural living conditions was still substantially relevant. It is aligned with Mali’s Systematic Country Diagnostic (2015) and Country Partnership Framework (2015), both of which stress the criticality of improving rural livelihoods for sustainable poverty reduction. 3 2.4 Although the overall objective was relevantly in line with the goals articulated by the country’s development strategies, it was written in a way that unnecessarily overcomplicates the project aim. The objective to strengthen institutional and policy-making capacity at the local and national level was a lofty goal in a pilot CDD project that was introducing participatory processes for the first time. CDD programming needs transitory steps, including incremental mechanisms to increase the government’s role over time—a longer-term aim. As the Implementation Completion and Results Report (ICR) points out, a policy support operation may have been a better place for this goal. The relevance of the project objective is rated Substantial. Project Design 2.5 The project had two main components supplemented by a third that supported operating costs: • Component 1 directed $15.5 million toward community development. This component financed subproject identification and construction, including by conducting participatory needs assessments and the hiring of local firms. The administration of this component and the project was complex. The project was implemented through a Project Management Unit (PMU) and was executed by an international NGO that oversaw the work of local NGOs in charge of community support services. The local NGOs supplied community development agents to villages within the targeted regions of Mopti and Tombouctou. The project provided training for communities, NGOs, and technical service providers, though there is very little information on this aspect of the project. • Component 2 directed $2.2 million toward policy and institutional development. This part of the project was designed to strengthen local and national decision making on poverty issues. It sought to do so by financing and improving the quality and relevance of poverty data and data analysis. It also sought to achieve better data integration between NGOs and the government. • Component 3 provided $3.3 million for operating costs, project preparation, refinancing, and physical and price contingencies. 2.6 Project Dates and Costs. The project was implemented between 1998 and 2004 at a total cost of $23.2 million. The actual IDA credit was lower than at appraisal because of currency exchange rate fluctuations. The government contribution increased from $1.5 million to $2.7 million. The project was not subject to restructuring or extension. Relevance of Project Design 2.7 This was the first World Bank project in Mali to support a participatory local development process explicitly. Because of a lack of effective local institutions, the project sought to test a more relevant and efficient service delivery system by pairing communities with nonstate actors, and also by involving the local government and state. The theory was that direct participation in local planning would lead to local investment that is more relevant and that the communities would own, and therefore use and maintain better. The theory, in 4 line with the wider aims of decentralized service delivery, also sought to use the campaign to achieve broader state presence and to enhance the legitimacy of fledgling local institutions while directing resources to areas in need. 2.8 The overall project architecture was sensible: Without state support for service delivery and institutions capable of delivering this, the project supported a grassroots campaign to connect each of the key constituents responsible for service delivery to each other under a grant financing mechanism. The project employed local development agents who were embedded at the village level and who conducted participatory needs assessments to inform project targeting. 2.9 The pilot project was designed to test new participatory approaches to service delivery; however, the project design was somewhat overcomplicated. In addition to community-level mobilization, it also sought to influence policy and institutional development at the national level. This aim was ill designed (defined solely by the achievement of outputs such as a database and geographic information systems training) and ill placed within a first-stage, pilot CDD project introduced within a fragile environment. The design lacked mechanisms to incentivize the integration of the poverty-related data into decision making and causal inferences as to how the poverty-related data and skills developed would be leveraged to achieve welfare impacts. 2.10 The project also overestimated the ability or willingness of the beneficiaries to provide cofinance, both because of the level of poverty, and because it takes time to build trust between the new entities (NGOs and private contractors, for example) that were introduced to the communities. 2.11 The relevance of project design is Modest. Implementation 2.12 The project was implemented by the Association d’Appui aux Initiatives de base, which contracted services from four international and 10 national NGOs (maîtres d’ouvrage délégués) to oversee the identification, selection, and construction of subprojects. These intermediaries recruited 25 community development groups that placed embedded development agents in the target villages. These agents oversaw all aspects of the project— from sensitization, to capacity building and the creation of community development plans, to the preparation, implementation, and management of community investments. At the village level, the community development agents contracted and oversaw the work of 94 technical service providers. This was a grassroots campaign, and the availability of skilled technicians willing and able to work and relocate to rural poor remote areas challenged the implementation. Seasonal and permanent migration, which affected the mobilization of labor for the subprojects and induced delays, was also challenging. 2.13 Targeting. A somewhat perverse set of incentives associated with the project’s indicator choices also challenged the aim of improving the living conditions of disadvantaged targeted rural communities during the project’s implementation. The project set a high population target, which created an incentive for project teams to work in villages with the 5 largest population size. The project reported reaching 170,000 beneficiaries in 185 villages in Mopti and Tombouctou. However, this number exceeds the average village size (most villages in Mali have between 150 and 600 residents). In Mali, the village chief typically resides in the most populous village; thus the largest villages tend to have, ex ante, better service access. 2 2.14 Swiftly evolving institutional landscape. The project was launched amid a series of decentralization reforms. Around its midterm review, in 1999, the government adopted a new regulation for decentralization that created new rural entities (communes) and later on delegated the responsibility for managing education, health, and water issues to communes. Sector policy reforms were also introduced that included participatory approaches and a set new rules for cost recovery and community fee levels. This was introduced through two large-scale national education national programs; through the revision of the agricultural and rural development plan, and through the adoption of a new water access strategy. 3. Efficacy 3.1 Evaluability (M&E Design, Implementation, and Use). As outlined in the project appraisal document, the management information system was theoretically strong: it included participatory monitoring and iterative feedback on living conditions in the project areas. However, the indicators (outputs) and methods (perceptions) used were too limited to measure change in welfare—the project’s objective. 3.2 The project used a single indicator to measure improvement in living conditions that were attributable to the project. The indicator was the percent of people satisfied with the outcome of the subprojects. The project reported that 100 percent of beneficiaries were satisfied with the project outcome. These data were derived from an external assessment that used beneficiary perception as the main indicator and data collection method. It is not clear from the project how many people were interviewed or what their relationship was to the subprojects (the project reported reaching 170,000 people). However, a 100 percent outcome on a satisfaction survey is implausible. None of the participatory methods for data collection built into the design appear to have been used for outcome-level reporting. This was a major shortcoming of the project’s M&E protocol. 3.3 The project should have developed a tool to conduct an analysis of the attributable welfare impacts associated with the water, health, and education project subinvestments that were cofinanced by the communities. A tool like the Aga Khan Development Network’s Quality of Life Assessment could have been relevantly applied. 3 Similarly, for the economic investments (gardens, irrigation works, and the like), metrics were needed to ascertain the net effects of the investments on the living conditions of the disadvantaged communities targeted by the project. 3.4 This type of more granular data—relatively easy to collect had it been included in project design—could have shed light on household welfare impacts and important intracommunity dynamics. For example, most rural communities in Mali are highly stratified by lineage (that is, chieftain lineage is typically elite), caste, and household assets. It is widely shown how village elites can capture development projects to the detriment of poor 6 households (Bierschenk, Chauveau, and Olivier de Sardan 2000; Schroeder 1999). A focus on poor, disadvantaged communities is not necessarily inappropriate, but it might mask these equally important intracommunity dynamics concerning rural poverty reduction. 3.5 Because of the lack of more granular data, this assessment uses the theory of change to draw causal inferences about project outcomes, considering the effects achieved under similar conditions in other World Bank–financed CDD projects implemented under similar conditions. It is designed to determine, through contribution analysis, whether the reported outputs could have plausibly contributed to the improved living conditions of disadvantaged rural communities. As such, it uses the Demographic and Health Surveys conducted in Mali for the region of Mopti over two, and in some cases three, periods (1995, 2001, and 2006) to determine the overall direction of welfare trends in the project area (and to situate the project, through contribution analysis, within this evolving landscape). Objective: Improve the living conditions of disadvantaged targeted rural communities by responding to their priority needs 3.6 Outputs. The project surpassed its subproject investment goal. It financed 287 socioeconomic investments compared with a target of 260 investments. 4 Most of the investments were made to obtain increased access to public services (69 percent), including for potable water, education, and health. About one-third of the investments (31 percent) were made for the acquisition of private assets, of which half were for small-scale irrigation programs, followed by agricultural equipment, market construction, and feeder roads (table 3.1). Table 3.1. Public and Private Services Financed by PAIB Investment Quantity (Share of Total, %) PUBLIC INFRASTRUCTURE 198 (69) Water (boreholes, solar pumps, manual pumps) 98 (34) Education (schools and literacy centers) 80 (27) Health (community health centers) 19 (7) Miscellaneous (latrines) 1 (0) PRODUCTIVE INFRASTRUCTURE 89 (31) Small-Scale Irrigation Schemess (vegetable gardens, micro-dams) 47 (16) Agricultural Equipment (mills) 30 (10) Miscellaneous (vaccination parks, markets, feeder roads) 12 (4) Total 287 Note: PAIB = Grassroots Hunger and Poverty Initiative Project. 3.7 IEG interviews of seven national NGOs that supported the implementation of this project confirmed that the targeted beneficiaries regarded the subprojects as relevant and useful. Local NGOs reported that target villages reacted very favorably to this first participatory process. The external assessment reported that based on the reported data obtained from the NGOs, 80 percent of subprojects were well executed, 15 percent had average execution, and 4 percent had poor execution. Although it is plausible that 80 percent 7 of the subprojects were executed well, the assessment does not provide information on their use or their quality. Public Infrastructure and Services 3.8 Water Supply. The participatory process led many villages to choose an investment in potable water (implemented in 98 villages). Through the external assessment, these investments were reported to have improved water quality and achieved time savings, especially for women. This assessment was unable to field-validate these findings because of travel restrictions, but an interview with WaterAid Mali (working in Mopti) provided further confirmation that the installation of water-pumping technologies would have improved water quality more than would open wells, which are highly vulnerable to contamination. Hand pumps were confirmed to be a better option than piped or motorized schemes (or solar), which require the skills, finances, and management that Mali’s remote rural areas lack. However, the project’s financing was inverted toward solar systems rather than the installation or environmental upgrading of wells: two-thirds of the water financing was directed toward solar systems versus one-tenth of financing toward hand pumps. If a field validation had been possible, it likely would have found that these solar pumps were not well maintained and therefore were probably not generating welfare impacts. 3.9 The installation of water points also lacked a water ecosystem approach, including behavioral training and sustained communication and advocacy about the benefits of water quality in addition to reliability and accessibility. 3.10 Education. The second most favored investment was the construction of schools. The project financed 80 education subprojects of which 64 were schools that supported the additional attendance of 6,000 pupils. It also supported literacy programs that reportedly trained 4,500 people. The project reported that reduced time for fetching water contributed to girls’ school attendance. Without indicators to measure education outcomes attributable to the project, this assessment uses data from Demographic and Health Surveys conducted in Mali’s Mopti region over two periods (2001 and 2006). The data show that although the overall primary and secondary attendance rate for males in Mopti was about the same for both periods, the attendance rate for females for both primary and secondary increased significantly (table 3.2). These results are not directly attributable to the project, but the data suggest that the schools constructed contributed to increased attendance and thus were likely to have marginally contributed to the positive trends reported through the Demographic and Health Surveys. Table 3.2. Net Attendance Rates in Mopti, 2001 and 2006 Mopti Region Net Attendance Rate, Primary (%) Net Attendance Rate, Secondary (%) 2001 2006 2001 2006 Male 27.4 27.3 10.3 11.6 Female 19.1 32.8 3.6 19.2 8 3.11 Literacy rates obtained from the Demographic and Health Surveys conducted in Mali’s Mopti region over the same periods (2001 and 2006) also show an increase in literacy for females, but at a rate much lower than overall attendance (table 3.3). Male literacy also increased significantly. Although these results are not directly attributable to the project, the data suggests that the project was likely to have contributed to this positive trend. Table 3.3. Literacy Rates in Mopti, 2001 and 2006 Mopti Region 2001 (%) 2006 (%) Male 6.7 26.2 Female 3.4 9.7 3.12 Health. Reported achievements related to the construction of small- Figure 3.1. Birth Assistance in Mopti scale health centers implemented in 19 40 . villages are not plausible. According 30 to the project, the health centers 33.7 20 29.5 reportedly reached 100,000 people, 21 20 10 16.5 18.1 implying that a single health center 0 could service 5,263 people. It is Percentage of births in a Percentage of births assisted plausible that the 19 health centers health facility by a trained professional increased access to basic health 1995 2001 2006 services (such as assisted births) for hundreds of villagers and thus contributed marginally to improved living conditions. However, because of the paucity of investment in this area, the contribution can be judged only marginal. At the same time, project documentation does not provide any information about the quality of the services provided, including the availability of equipment or trained birthing assistants. 3.13 Overall, data obtained from the Demographic and Health Surveys in Mopti for the pre-, during, and post-project implementation periods show improvement for several health- related indicators, including distance to a health center and assisted births. According to data collected through these surveys, the percentage of women who indicated they have difficulty accessing health care because of the distance to a health center decreased from 58.7 percent to 45.6 percent between 2001 and 2006. The percentage of births in a health facility increased from 16.5 percent to 18.1 percent between 1995 and 2001, and then to 29.5 percent between 2001 and 2006 (figure 3.1). The percentage of births assisted by a trained professional decreased from 21 percent in 1995 to 20 percent in 2001, but increased overall to 33.7 percent in 2006 (figure 3.1). Private Assets 3.14 Productive Infrastructure. The project financed the construction of 47 small-scale irrigation schemes and 30 mills, and made 12 investments in market-related infrastructure, including feeder roads, vaccination parks, and the construction of markets. According to the external assessment, the small-scale irrigation schemes enabled an expansion in cultivated 9 land and an additional planting season for vegetable gardens (table 3.4). Agricultural processing equipment reduced the time and labor requirements to transform crops. However, no data were collected on consumption, sales, revenues, or related welfare effects. 3.15 Table 3.4 documents the production increases associated with the project investments. The data, taken from a sample, reveal that production increased because of a combination of land expansion and increased water availability. However, the gains are only marginal for the vegetable gardens, which are the most frequently occurring investment. In five of the six cases cited, 66–126 people are reported to have benefited from increased production on a single hectare of land. Table 3.4. Production Increases from Small-Scale Irrigation Subprojects 3.16 Overall, efficacy is rated Modest. This assessment recognizes the critical welfare benefits of the provision of basic public services (including water and health care), but the lack of a results measurement system undermines this assessment’s ability to credibly validate welfare gains attributable to the project-financed investments. Technology choices for the most frequently occurring public investment were misaligned with the operating environment, reported health gains are not plausible, and only modest gains were made through the private productive investments (for example, in vegetable gardens). There is evidence that communities have paid over time for the basic operating costs of public services, but that their overall maintenance—including for equipment—is unaffordable. Shifts in the institutional landscape undermined the aim, as a pilot project, of testing and integrating lessons into the follow-on project. For example, in the follow-on project, the participatory approaches could not be replicated for public service provision because of ANICT’s launch. 4. Efficiency 4.1 Economic Rate of Return. An economic rate of return for the project as a whole was not calculated at appraisal because of the project’s qualitative and demand-driven nature, nor was it calculated at project close, but a basic financial analysis of net earnings from physical investments was presented. All investment types are reported to have produced positive net 10 earnings. However, these net earnings quickly turn negative when the costs associated with the maintenance of equipment and works are factored into the analysis (table 4.1). This underscores the need to focus on community financial capacity to ensure that subprojects reach their full economic potential (increased productivity, reduced transportation costs, and so on). The ICR suggested that additional interventions could have increased returns on subprojects, including diversifying the use of agricultural processing equipment (especially for cash crops), and providing microcredits to allow beneficiaries to profit from saved time. However, a review of the more diversified activities in the Projet d’Appui aux Communautés Rurales (Rural Community Development Project; PACR), the follow-on project, revealed that these activities lacked market integration and required high upfront capital costs, and that beneficiaries often incurred familial debt to undertake them. In PACR, the more complicated productive investments were the least profitable. Table 4.1. Average Yearly Operating Accounts for Selected Physical Investments 4.2 Cost Effectiveness. Overall, costs were kept under targets. Administrative costs were less than 10 percent of total costs, and the average subproject cost was just under $70,000, including preparation and associated soft investments like community mobilization and organization, capacity building, and NGO fees (table 5.3). 5 Community financial contributions represented, on average, 5.7 percent (under the 10 percent target); communities also contributed in kind by using local materials and labor. 4.3 The pilot operation was cost effective, but efficiency is rated Modest because the projected net earnings on all investments except for hand pumps are negative. Furthermore, according to studies conducted by Water Aid, hand pumps are unlikely to have the reported return because they have an average life of two years before requiring maintenance. 11 5. Ratings Outcome 5.1 The project’s overall outcome is rated Moderately Unsatisfactory. The overall objective was relevantly in line with the country’s development strategies. Its design at approval was sensible: without state support for service delivery and institutions capable of delivering this, the project supported a grassroots campaign to connect rural citizens with local service providers in a way that best matched their needs. However, the relevance of design was undermined in a swiftly changing institutional landscape, and parallel design elements to influence broader policy and institutional reforms related to poverty were ill suited for a pilot CDD project. Although this project delivered essential services to rural areas—including by increasing access to health, education, and water—the lack of a results measurement system undermines the ability of this assessment to credibly validate welfare gains attributable to the project-financed investments. Efficiency is modest because the projected returns on all investment choices except for hand pumps are negative because of the lack of sustainable financing to maintain the service delivery systems provided by the project. Risk to Development Outcome 5.2 Financial Risks. Rural institutions in Mali require a higher level of financial assistance from the central government. As noted in this Project Performance Assessment Report, the ANICT transfer mechanism has been disappointing in this regard. A law passed on October 2, 2017 (Loi 2017-052) stipulates that 30 percent of public finances be transferred to the communes beginning in 2018. This has been the kind of substantial fiscal transfer that local authorities have demanded for years, so if it is implemented, programs such as the Grassroots Hunger and Poverty Initiative Project (PAIB) will have a much better chance of achieving long-term results. The same law stipulates that a share of the revenue from natural resource extraction and other kinds of economic activities, which addresses another longstanding deficiency in Mali’s decentralization program. Again, it is not yet known whether the law will be implemented. Finally, it is always notable that Mali has a regressive local tax structure in which local authorities collect a nominal, flat rate for each member of a given household. Until there is tax reform, local governments will be financially constrained and dependent on NGOs. Some development partners, like the International Monetary Fund, are calling for such tax reform. 5.3 O&M Risks. Many of the basic services provided by the World Bank in Mali’s rural areas have substantial risk of not being operated or maintained effectively. At the time of project close, it was clear that “collected fees for schools and health centers could barely cover their basic operating and maintenance costs” (World Bank 2004, 9). Arrangements with public institutions to cover these costs had not been solidified, and most communities lack the financial resources to manage these costs on their own. During and at the time of the pilot closure, the average annual transfer for basic services in Mopti was $1.70 per capita. Although the Malian government has increased resources in the past few years for the 12 provision of basic services, the 2012 coup interrupted this, and civil unrest continues to hinder progress. 5.4 Data that are more recent, provided by Water Aid Mali for the Mopti region, also find that donor-financed hand pumps in Mopti tend to be abandoned, but hand-dug wells remain. WaterAid’s research shows the need to invest in a sustained information and behavioral campaign if health outcomes associated with water quality versus water accessibility are intended to be achieved. 5.5 Conflict-Related Risks. The risk that project gains could erode because of violence and insecurity is high. Ongoing conflict since the 2012 coup has interrupted service delivery in the project areas. Data provided by the UN UN Office for the Coordination of Humanitarian Affairs show that many schools have been closed in the targeted regions: 501 schools in the Gao, Ménaka, Mopti, Ségou, and Timbuktu have closed, resulting in at least 150,000 children leaving school. More than half of these school closings (266 schools) have occurred in the Mopti region. 5.6 Since IEG conducted this assessment, insecurity has spread and taken hold in the Mopti region, causing its public institutions to stop functioning in many communes. It will likely take years for those communities to recover when the situation eventually stabilizes. However, the crisis led to the signing of the Algiers Peace Accord, which has been the impetus behind the latest decentralization measures, including the passage of Law 2017-052 previously referenced. Currently, there is mixed evidence that elite politics are improving in Mali, and it is likely that, on balance, it is not improving substantially or quickly enough. Without a well-functioning central government, these reforms and others that aim to improve institutions, services, and livelihoods in the rest of the country will continue to be an immense challenge. 5.7 Social/Institutional Risks. The project had a strong focus on social organization and management capacity building, including forming and supporting community service committees. The project established management committees for each investment, teacher- parent associations for education, and community associations for health. All members received training specific to each category of investment and each type of responsibility, as well as literacy training geared toward their management responsibilities. IEG’s field mission to Mali in the spring of 2017 found these committees still intact and respected by the commune councils. The mission also observed a large amount of pride shown by committee members and built knowledge. The committees’ financing, however, was unclear. 5.8 Citizens’ representation and participation in priority setting and municipal-level development planning were found to be at risk. This risk was present at the closure of PAIB, and was reconfirmed by IEG’s assessment of the follow-on project, PACR. IEG found that in the lack of an institutionalized process for negotiation and a voice in needs identification, budgeting, and planning, many of the social innovations (participatory planning and priority setting) will likely be discarded. A key example is how the communes have engaged in the development of their Plan de Développement Economique et Social de la Commune (Local Economic and Social Development Plans). IEG’s mission found that although the follow-on project supported a participatory process to enhance representation of the identification of 13 felt needs, in all but one of the dozen communes visited, there has been no financing for these plans since they were developed in 2009. Interviews with several mayors reinforced this finding, that communes are relying on NGOs to help them take inventory of their village- level services and to prioritize needs. 5.9 The project used an NGO model to support basic rural service delivery, a model that the establishment of ANICT (originally funded by the European Commission) has since replaced. IEG found that the subsequent country systems require a significant level of capacity building, especially in supporting fair and transparent resource allocation to those areas most in need. IEG’s assessment found instances of collusion between ANICT and select commune heads and between ANICT and entrepreneurs, which at times has resulted in inefficient choices or poor-quality infrastructure (building materials, for example) in the former project areas. 5.10 Risk to development outcome is rated High. World Bank Performance QUALITY AT ENTRY 5.11 Quality at Entry. The preparation process and early phase of the project development were uniquely innovative because they worked to introduce a participatory planning process for the first time in a World Bank project in Mali. Although fully in line with the decentralization aims of the government, especially shortly after the introduction of democratic elections in the country, the project was designed in a vacuum, without a decentralization spine including structures and mechanisms for financial mobilization and adequate transfer. To build the project, the World Bank engaged in a highly consultative process. It also pre-piloted activities in the field to test institutional arrangements and implementation procedures. The World Bank then conducted an early in-depth analysis to assess the viability of project design and adjust it based on the different lessons learned. These lessons included increasing communities’ involvement in decision making, standardizing implementation and bidding documents to simplify procedures for field actors (NGOs and communities), targeting villages as the focal or entry point for intervention, and so on. This phase placed a premium on participation and inclusion. 5.12 Although this approach is noteworthy, it placed less attention on understanding the financial and economic viability of the physical investments. One of the main challenges with this project was its overestimation of the technical and financial capacity of communities to manage the productive investments and to afford to operate and sustain the communal investments. Training was needed, but because of the lack of human resources and the target areas’ remoteness, the project relied on a trainers-of-trainers approach. This approach, as expected, resulted in a lack of high-quality information distribution along the chain. Another weakness was the project’s overambitious aim to affect policy and institutional reform (as discussed in the Relevance of Design section). Furthermore, the results frame was not designed to test project assumptions along a causal chain, which erodes much of a pilot’s potential to inform the development of subsequent phases. 14 5.13 Quality at entry is rated Moderately Unsatisfactory. QUALITY OF SUPERVISION 5.14 National project staff appreciated the frequency, quality, and frankness of World Bank supervision missions. However, the turnover of task team leaders and World Bank staff, as well as the transfer of the project from one department to another, caused implementation delays. The project’s nature changed from social to economic at midterm in an effort to achieve the main objective of improved living standards. However, supervision did not refine or upgrade the M&E system, whose project development objective indicators were limited to socially oriented output indicators (and were incapable of properly measuring the main objective), and it did not update the safeguard category. There is a lack of evidence, though, that supervision engaged with the pilot to learn lessons though trial and error throughout the implementation process and to document these lessons to inform the scale up of the pilot in subsequent phases. Unanswered questions, like that of affordability, plagued the project’s second phase, which IEG also evaluated. 5.15 Safeguards. The project (approved in 1998) was classified as category C, considering the nature of the subprojects to be financed. The justification was that the subprojects would have very limited, if any, negative impact on the environment, and that providing training on environmental management would reduce any impact. This project, if approved today, would be a category B, like most other CDD projects with multisectoral investments. At midterm, the project shifted its focus from social to economic investments, including a significant level of investment in irrigation and land cultivation (including flooded areas). This assessment could not find evidence that the project revisited the safeguard rating after midterm because of the agricultural activities. 5.16 Quality of supervision is rated Moderately Unsatisfactory. 5.17 Together, these ratings lead to an overall World Bank performance rating of Moderately Unsatisfactory. Borrower Performance GOVERNMENT PERFORMANCE 5.18 The government of Mali showed a strong level of commitment to the PAIB pilot, which at the time was in line with its renewed support for decentralization, including for more effective service delivery. It adopted institutional changes that supported the project’s community-driven nature and its focus on public services. The institutional changes included a 1999 regulation on decentralization that created new rural entities (communes); a 2002 law transferring education, health, and water management to these communes; and sectoral reforms in education, health, agriculture and rural development, and water access that aligned with PAIB’s participatory approach. However, delays in the disbursement of the government’s contribution caused some financial difficulty for the PMU, along with delays in appointing teachers and health professionals to staff newly built infrastructure. 15 5.19 Government performance is rated Satisfactory. IMPLEMENTING AGENCY PERFORMANCE 5.20 This assessment generally concurs with the rating the ICR assigned to the implementing agency, but it deviates from the highly satisfactory rating assigned. For this assessment, “implementing agency” refers to the full suite of project implementers, including the PMU, and the international and local NGOs involved in project execution. The PMU highly supported the project model, and it worked effectively to bridge the interests of local government, NGOs, and the communities at a time when local rural commune councils had just been created. The implementation model embedded members of local NGOs, in the role of community development agents, into rural remote villages far away from home and family. In doing so, it ensured the effective selection and execution of the communal investments. PMU oversight also supported effective compliance with the World Bank’s financial management and procurement procedures. The PMU was organized well, though a bit understaffed. 5.21 Implementing agency performance is rated Satisfactory. 5.22 Together, these ratings lead to an overall rating of borrower performance of Satisfactory. 6. Lessons 6.1 Projects that seek to “improve living conditions” need to define, benchmark, and measure a project’s attributable contribution to changes in human welfare. Measuring access to assets is an insufficient metric. One tool that could be utilized by World Bank rural development projects is the Aga Khan Foundation’s Quality of Life Assessment, which has been rolled out and used in their rural development programs. 6.2 Private productive investments should not be relied on to finance the operations and maintenance of core public services provided by World Bank–financed projects whenever local revenue recovery is not feasible. The returns on investment for micro and small enterprise development are not likely to contribute to significant autonomy of public finance in rural areas in the short to medium term without major advances in productivity, market connectivity, and consumer demand. Project objectives should focus on the marginal but important household income and revenue benefits that can accrue, while acknowledging that financially sustainable public infrastructure requires a more conducive policy environment at higher levels (region and central government). 6.3 A pilot project should be designed to generate lessons: its M&E System should therefore double as a learning lab to test and adapt interventions within the country and sector context. PAIB was a pilot project, but its M&E system was limited mainly to output indicators. There were no “tests” or efforts to adapt the project based on learning along the way, and no parallel or special studies conducted to learn why these innovations worked or didn’t work, with the aim of scaling what worked in a second phase. 16 1 On the agenda since 1977, decentralization in Mali gained new momentum with democratization in the 1990s. Decentralization provided a platform for the new civilian regime to redefine the nature of the state after decades of military rule and dictatorship. Its purpose was also to bolster state legitimacy in the north after the Tuareg Rebellion of 1990. Building on traditions that go back to the Malian Empire, decentralization was framed as a political solution to the state’s security and development challenges (Baudais 2006; Whitfield 2009). 2 With a reported 170,000 beneficiaries in 185 villages, the average project village had a population of about 920. This village would be larger than average; most villages in Mali have between 150 and 600 residents. 3 Visit http://www.akdn.org/news/akdn-quality-life-assessment-programme-brochure to download the Aga Khan Development Network’s Quality of Life Assessment brochure. 4 This differs from the number reported in the external assessment, which include an additional 39 “latrines ADC” that appear to have been built for the community agents that resided in villages in support of PAIB. 5 Estimated at CFAF 37.5 million, U.S. dollars calculated using an exchange rate of $1 = CFAF 547. Distribution of costs was estimated at “2.5 percent for preparation (initial survey, participatory diagnosis, and feasibility study), 30.5 percent for ‘soft’ investment, that is, community mobilization, social organization, capacity building activities, and NGOs’ honorariums, and 60 percent for the physical realization” (World Bank 2004, 9). 17 References Bebbington, Anthony, Michael Woolcock, Scott E. Guggenheim, and Elizabeth A. Olson. 2006. The Search for Empowerment: Social Capital as Idea and Practice at the World Bank. Bloomfield, CT: Kumarian Press, Inc. Bierschenk, Thomas, Jean-Pierre Chauveau, and Jean-Pierre Olivier de Sardan. 2000. Courtiers en développement: les villages africains en quête de projets. Paris: Karthala. de Regt, Jacomina, Shruti Majumdar, and Janmejay Singh. 2013. “Designing Community-Driven Development Operations in Fragile and Conflict-Affected Situations: Lessons from a Stocktaking.” Working paper 83022, World Bank, Washington, DC. de Sardan, Jean-Pierre Olivier. 2005. Anthropology and Development: Understanding Contemporary Social Change. London: Zed Books Ltd. Grootaert, Christiaan, and Thierry van Bastelaer, eds. 2002. Understanding and Measuring Social Capital: A Multidisciplinary Tool for Practitioners. Washington, DC: World Bank. Hoogeveen, Johannes G., Mariacristina Rossi, and Dario Sansone. 2017. “Leaving, Staying, or Coming Back? Migration Decisions during the Northern Mali Conflict.” Policy Research Working Paper WPS8012, World Bank, Washington, DC. Öjendal, Joakim, and Anki Dellnäs. 2013. The Imperative of Good Local Governance: Challenges for the Next Decade of Decentralization. New York: United Nations University Press. Matsumoto-Izadifar, Yoshiko. 2009. “Mali: Beyond Cotton, Searching for ‘Green Gold’.” OECD Journal: General Papers 6 (2): 33–51. Pattison, C. and S. Wong. 2016. “Taking Stock of Community-Driven Development in Fragile and Conflict Situations: Lessons Learned from the First Generation.” Paper prepared for the World Bank workshop “Service Delivery and State Legitimacy in Fragile and Conflict- Affected States,” Washington, DC December 5. Schroeder, R. A. 1999. Shady Practices: Agroforestry and Gender Politics in The Gambia. Berkeley: University of California Press. UNICEF (United Nations Children’s Fund) and UCW (Understanding Children’s Work). 2009. Comprendre le travail des enfants au Mali: Rapport sur le travail des enfants. Rome: UNICEF and UCW. World Bank 2004. “Mali— Project to Support Grassroots Initiatives to Fight Hunger and Poverty.” Implementation Completion and Results Report 28408, World Bank, Washington, DC. ———. 2005. “Mali—Rural Community Development Project.” Project Appraisal Document 31205- ML, World Bank, Washington, DC. ———. 2013. “Mali—Reconstruction and Economic Recovery Project.” Project Appraisal Document. 81084-ML, World Bank, Washington, DC. ———. 2014. “Nigeria—Second National Fadama Development Project.” Project Performance Assessment Report 88958, World Bank, Washington, DC. ———. 2015. “Mali—Geography of Poverty in Mali.” Poverty Assessment 88880-ML, World Bank, Washington, DC. ———. 2016. “Can a Nation Build its Future if it Cannot Feed its Children? Five Policy Actions to Transform Crop and Livestock Farming in Mali.” Who We Are/News, December 9. 18 http://www.worldbank.org/en/news/feature/2016/12/09/can-a-nation-build-its-future-if-it- cannot-feed-its-children-five-policy-actions-to-transform-crop-and-livestock-farming-in-mali. 19 Appendix A. Basic Data Sheet MALI RURAL COMMUNITY DEVELOPMENT PROJECT Key Project Data (amounts in US$, millions) Appraisal Actual or Actual as % of Estimate Current Estimate Appraisal Estimate Total project costs 23 23.2 100.86 Grant amount 21.5 21.01 97.72 Cumulative Estimated and Actual Disbursements FY99 FY00 FY01 FY02 FY03 FY04 Appraisal estimate ($, M) 4.58 8.49 13.89 19.68 21.43 21.43 Actual ($, M) 3.24 5.66 10.01 15.18 20.68 21.01 Actual as % of appraisal 70.74 66.66 72.06 77.13 96.5 98.04 Date of Final Disbursement 2004 Project Dates Original Actual Concept Review 08/15/1995 Board approval 04/07/1998 Signing 04/09/1998 Effectiveness 10/01/98 10/01/1998 Closing date 01/31/2004 01/31/2004 20 APPENDIX A Staff Time and Cost Staff Time and Cost (World Bank Budget only) Stage of Project Cycle No. of Staff Weeks $, thousands (including travel and consultant costs) Lending Identification/Preparation 210 453.1 Appraisal/Negotiation 37 79.4 Supervision 102 218.3 ICR 8 11.5 Total 147 762.3 21 Appendix B. List of Persons Met Government of Mali Cisse, Modibe Director General National Agency for Collective Territorial Investments Cissouma, Edouard Head of M&E National Agency for Collective Territorial Investments Macky, Alpha Deputy National Agency for Collective Territorial Investments Implementing Nongovernmental Organizations (NGOs) Mama Sankare Project Officer MOD” CARE International Mali, MOPTI Salmana Traore Agent de développement Aga Khan Foundation communautaire Seydou Traore Finance Officer Afar Abdoulaye Tembely President Afar Chaka Sidibe Project Officer in charge of Afar infrastructures Amba Ouelogueme M&E Officer Afar Seydou Traore Project Officer Afar United Nations, Bilateral and Regional Development Banks Dentice, Allesandra Deputy Representative UNICEF Mistycki, Veronique Resource Mobilization UNICEF Specialist Kollies, Ute Chef de Bureau UN Office for the Coordination of Humanitarian Affairs Seid, Fatouma Representative FAO Bilaterals Boutroux, Thierry Assistant Director Agence Francaise de Developpement (AFD) Neri, Beatrice Head of Section, Rural Delegation of the European Union in Mali Development Bambara, Moussa Director, Peace, Democracy U.S. Agency for International Development (USAID) Mali and Governance Camara, Amadou Resilience Program USAID Coordinator Clark, Donald Special Adviser USAID Acting Bureau Chief USAID Dioum,Macky Amadou Principle Agro-Economist African Development Bank Fredette, Marc-Andre Ambassador Embassy of Canada Other NGOs Douvan, Yawo Mission Director CARE Mali APPENDIX B 22 World Bank Rogy, Michel Program Leader, Bamako, Mali CMU Sustainable Development, World Bank Guinea, CAR, Chad, Mali, Niger (AFCW3) Um, Paul Noumba Former Country Director Current Country Director Mali AFCS1, World Bank Yamouri, Najat Senior Social Development Washington, DC Specialist World Bank Durand, Olivier Task Manager, PACR Tashkent, Uzbekistan World Bank Ehoue, Nicaise Task Manager, PACR Washington, DC World Bank Hans Hoogevan Washington, DC World Bank International NGOs Douvon, Yawo Country Director CARE International Mali Diallo, Mamadou Diarafa Country Director Water Aid Mali 23 Appendix C. Borrower Comments No comments were received from the Borrower.