Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Report Number: ICRR0021804 1. Project Data Project ID Project Name P129565 West Africa Agric. Productivity Prog. 2A Country Practice Area(Lead) Western Africa Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-51350,IDA-51360,IDA-52860,IDA- 31-Dec-2017 186,527,792.38 58110 Bank Approval Date Closing Date (Actual) 22-May-2012 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 120,000,000.00 0.00 Revised Commitment 199,816,092.82 0.00 Actual 186,626,543.73 0.00 Prepared by Reviewed by ICR Review Coordinator Group Keith Robert A. Oblitas Ridley Nelson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives To scale-up the generation, dissemination and adoption of improved technologies in the Participating Countries’ priority agricultural commodity areas. (Source: Financing Agreement, Republic of Senegal, August 24, 2012) Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) The Project Objectives in the Financing Agreements for the Ghana and Mali operations are identical. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project – “the Second Phase of the Support Program for the West Africa Agricultural Productivity Program (WAAPP-2A)” was the second phase of an Adaptable Program Loan for Ghana, Senegal and Mali and a cross-country coordinating entity for the programs of these countries. It was part of the multi-country overall WAAPP program involving WAAPP-1B (2007-2016: Burkina Faso, Cote d’Ivoire and Nigeria); and WAAPP-1C 2018-2019: Benin, Gambia, Guinea, Liberia, Niger, Sierra Leone and Togo. Component 1: Enabling Conditions for Regional Cooperation in the Generation, Dissemination and Adoption of Agricultural Technologies. (Estimated costs at appraisal: US$ 10.9 million; actual costs US$ 8.8 million) Strengthening mechanisms and procedures for the exchange of technologies between participating countries including: common regulations on genetic materials, pesticides, fertilizer and veterinary products; development or improvement of regional strategies into national action plans, including knowledge management, information and communication systems; mainstreaming of gender and climate change activities; sustained mechanisms for financing agricultural research and development of intellectual property rights; and including environmental and social management safeguards. Component 2: National Centers of Specialization (Estimated costs at appraisal: US$ 42.3 million; actual costs US$ 40.8 million) Strengthening regional cooperation in technology generation by aligning national priorities with regional ones; upgrading national research to Regional Centers of Excellence; strengthening national research agencies; developing synergies with the CGIAR (Consultative Group on International Agricultural Research, and other research institution; capacity building grants for the West and Central African Council for Agricultural Research and Development (CORAF) for research planning and workshops; and regional policy development and market integration to improve the sustainability of Research and Development. Component 3: Support to Demand-driven Technology Generation, Dissemination and Adoption (Estimated costs at appraisal: US$ 131.1; actual costs US$ 120.0 million) Strengthening priority focused, demand-driven agricultural research, scaling up technology dissemination and adoption, facilitating access to genetic material, including strengthening the Competitive Agricultural Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Research Grant Scheme in each of the participating countries, strengthening technology delivery systems and cooperating with other development activities and agencies. Component 4: Project Coordination, Management and Monitoring and Evaluation. ((Estimated costs at appraisal: US$ 14.7 million; actual costs US$ 17.0 million.) Coordination and Management of the overall WAAP-2A was handled by CORAF at the regional level, and by each state’s national coordinating body at country levels. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs Project costs estimated at Appraisal were US$ 200 million and actual costs at project closing were US$ 186.6 million, 93 percent of the appraisal estimate. The shortfall (while implementation targets were generally exceeded) was due to use for some project expenditures of national funding rather than regional IDA sources. The ICR does not provide a break-down of funding by source. The cost reduction for Component 3 (demand-driven technology generation, dissemination and adoption) which fell from the US$ 131.1 million at appraisal to US$ 120 million at project completion, was the primary factor in the project’s overall cost reduction. There are some discrepancies between the cost figures in the ICR’s paras 14 to 18 and the project costs in the ICR’s Annex 3. As Annex 3 provides more complete data, including costs estimated at appraisal as well as actual costs, Annex 3 data is used throughout this ICRR. Financing Each of the three participating countries (Senegal, Ghana and Mali) received an initial IDA Credit of US$ 60.00 million. Senegal received a further US$ 20 million to expand component 3’s program to include a value-added chain for groundnuts. The borrower contribution was US$ 15 million. Key dates The project was approved on May 22, 2012 and became effective on 12 December, 2012.The mid-term review was in October 2015. Project closure was planned to be 31 December, 2017, but was extended by one year to 31 December, 2018 to provide time for implementing the Mali program, which had started late due to security concerns. The Ghana program was closed on December 31, 2017, as planned at Appraisal. The Senegal program continued during the extra year, but primarily to finance development of the value chain for groundnuts. Reallocations Three adjustments were made during project implementation comprising two reallocations of funds (on January 27, 2017 and August 22, 2017) and the extension of the project by one year on January 11, 2018. There was no change to the project objectives and project components. The inclusion of the groundnut value chain for Senegal was the same design as for the original project components under objective 3. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) 3. Relevance of Objectives Rationale WAAPP-2A was responsive to the agricultural development needs of three participating countries – Senegal, Ghana and Mali – and also supported the rural development strategies in regional and Bank policies. The central problem was the agricultural sector’s low productivity: agricultural growth in the three participating countries was below potential and dominated by low income subsistence farming; farmers, typically smallholders, had limited access to new technologies and extension advice; there was a dearth in sharing of new technologies across countries; Government interventions such as subsidies were inefficient; the private sector had a limited presence; and Research and Extension was not adequately addressing the needs and productive potential of women. There was also an increase in climate extremes such as unusually high precipitation, droughts, and floods. The consequences of such weaknesses were considerable. Regionally, agriculture contributes about 30 percent of GDP, 28 percent of exports, and two- thirds of employment; and improved agricultural growth is key to boosting all three of these. Policies proposed to address such problems were broadly the same in Government, Bank and other stakeholder assessments. Regionally and nationally, the ECOWAP (Regional Agricultural Policy), subscribed to by all of the participating countries, provides a regional action plan supporting each country’s National Agricultural Investment Program, and cross-country research is also allied to this plan coordinated by the regional research agency, CORAF, a primary implementer of agricultural research across the region, and the coordinator of WAAPP-2A. The project was also consistent with Bank policy for the countries and region. The Regional Integration Strategy for Sub-Saharan Africa, updated in 2011, supports cross-country collaboration for regional development and for growth in agricultural productivity. The Country Assistance Strategies for the participating countries promote improving agricultural productivity and sustainability, and support of regional integration. For Ghana, the Country Partnership Strategy 2013-2016, extended to 2018, targets improving competitiveness (the essence of the WAAPP program) and job creation. In Mali, the FY14-15 Interim Strategy note emphasizes strengthening capacities and delivering basic services. The project approach is also supported by Senegal. WAAPP was, effectively, the operational manifestation of much of Government’s and the Bank’s strategy. The project - WAAPP-2A – was an integral part of the overall WAAPP program. WAAPP was financed through a ten-year Adaptable Program Loan in two tranches intended to be five years each. This project was the second and final tranche of the APL. The project’s objectives to improve the (i) generation, (ii) dissemination and (iii) adoption of agricultural technologies responded directly to the agricultural sector’s central need – the goal of improving productivity – thereby impacting positively on both GDP growth and on rural employment and social welfare. Improved agricultural technology was articulated through the objectives enabling a more focused project design. The project description shows a generally good reflection of the project objectives in the project components and grouping of actions, and the Theory of Change chart at page 5 of the ICR provides a logical progression from objectives to components. Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) What is missing, however, is a connection between the objectives (and their expression through the activities in the project components) to the presumed primary goal of the project which, according to the national strategies, was to increase agricultural productivity. The gap here is reflected in other sections of the ICR (e.g. Efficacy, Bank Performance, M&E, Monitorable Indicators, Results Framework) where there is limited reference to impacts and yields. For instance, the quantity and quality of seed produced says nothing about yields and incomes. The exception is in the economic analysis which has a selection of gross margins, but there is limited indication that project impacts were a major consideration - a results- based approach, with more focus on the outcome end of the logical chain, would have been better. However, on balance, taking account of the substantial strategic need for improved technologies in the agriculture sector, the focused objectives chosen, and the project’s design to meet these objectives, the Relevance of WAAPP -2A was Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To scale-up the generation of improved technologies in the Participating Countries priority agricultural commodity areas. (Note that, in this Section 4, WAAPP-2’s three objectives are reviewed separately as: – to scale up the Generation (Objective 1); Dissemination (Objective 2); and Adoption (Objective 3). Rationale Outputs  National agricultural research centers were upgraded to become Regional Centers of Excellence: Achieved in Ghana and Senegal but not in Mali.  Linkages were established or significantly strengthened with international research organizations such as the CGIAR (the Consultative Group on International Agricultural Research).  National laboratories were internationally certified by the International Seed Testing Association and the International Organization for Standardization). Actual - 5 labs certified (target 12).  Action plans for gender inclusion and climate change were prepared for all countries.  A participatory process for transparent approval of research grants coordinated by the Competitive Agricultural Grant Scheme was established to identify relevant applied research needs and for approval of grants for applied research proposals. Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) The technologies promoted covered a wide range of enterprise activities mostly related to crops (80 percent), with 12 percent of other grants going to livestock and eight percent to fundamental research. Technologies promoted included grants for developing new crop varieties (the dominant grant activities), cropping technologies, livestock husbandry, soil management, processing, marketing, weed and pesticide management, agricultural mechanization, agroforestry, and climate-smart agricultural practices.  Demand-driven multi-country research proposals were submitted to the national research grants scheme: Proposals submitted 218 (target 197).  75 technologies with International Property Rights were registered (target 78)  96 new crop varieties were recorded in the Regional Catalogue (target 77)  1,198 Pesticide products and genetic materials were registered by national committees compared with a baseline of 409 and an original target of 810.  Generated/released technologies with at least 15 percent productivity increase over control sample: 112 actual. (target 59). In country comparisons there are some differences in achievement of individual project activities between Ghana and Senegal but for both countries, performance in nearly all activities was at or above targets. In the case of Mali, affected by delayed commencement due to country security conditions, performance was better than might have been expected and for most indicators close to the achievements of Ghana and Senegal: e.g. The number of beneficiaries was 900,000 in Mali compared with 959,000 beneficiaries in Senegal and 875,000 in Ghana. Outcomes and Intermediate Outcomes The main achievement was the creation of institutions and capacity in the countries and across countries through the establishment or strengthening of an institutional and technical foundation for generation of improved agricultural technologies (see outputs above). Particular impact on generation of technologies was through the strengthening of laboratories at national and regional levels, and of other infrastructure for seed generation. Linkages were also established with international centers of excellence, fostering the sophisticated processes and technologies required for seed testing, production, and standardization in the seed production process. The development of a transparent selection and funding process for approval of technology grants was also a significant achievement. Together, and in coordination with the agricultural extension system and the web-based agricultural information system (refer actions under objectives 2 and 3) 112 new technologies with at least a 15 percent productivity increase against controls were generated (the target was 59 technologies). More significant, however, was the upgrading of technical and institutional capacity, nationally, cross-country and internationally for generating future technologies. In broader terms the ultimate desired outcome would be to contribute to the governments' rural strategies to increase rural productivity, for which the project objectives could be considered as intermediate objectives. The Efficacy of the objective to Scale Up the Generation of New Technologies was Substantial. Rating Substantial Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) OBJECTIVE 2 Objective To scale-up the dissemination of improved technologies in the participating countries' priority agricultural commodity areas. Rationale Outputs The primary means to improve dissemination of new technologies was through three thrusts: (i) the extension service; (ii) a web-based information system; and (iii) use of demonstration plots. These three approaches were applied on a large scale. At a higher level, exchange visits with other countries, grants for university degrees and association with scientists from international centers of expertise provided the knowledge base for establishing and expanding the program. The relative weight of the three dissemination approaches varied between the countries reflecting different strategies for reaching producers. Ghana developed a web-based- information system far larger than the other countries, but Mali had 52,000 demonstration plots, nearly the same as Ghana, with Senegal achieving lower numbers.  Demonstration plots established: 119,000 plots established; target 80,000 plots.  Producers in project areas with knowledge of new generated technologies. Actual 83 percent of producers; Target 80 percent.  Beneficiaries using technologies generated in other countries: 297,000 (target: 420,000).  Project technologies demonstrated in at least 3 ECOWAS countries: actual 30; target 18.  A Market of Innovations and Agricultural Technologies was set up.  Scientific exchange visits with other countries) Actual: visits 114; target: 104  Studies sponsored (university degrees): actual 505; target 213.  Establishment and expansion of a web-based information system on agricultural technologies.  The public extensions systems were aligned with the WAPPAA-2A to provide a link between the research stations and much of the technology generation activities. Dissemination included substantial use of demonstration plots and farmer training, and close linkage of extension with the web, especially in Ghana.  Use of the web-based agricultural information system reached nearly 1 million ‘hits,’ although the great bulk of this was Ghana’s program (880,000 hits), Mali and Senegal making limited progress. Outcomes and Intermediate Outcomes Although some activities, especially the web-information system, had uneven performance, most other actions in the three countries (e.g. number of demonstration plots, proportion of farmers with knowledge of the new technologies) achieved their targets, and the scale of the demonstration and extension programs could reasonably be expected to improve dissemination of agricultural technologies. The Efficacy of this objective of scaling up dissemination is rated Substantial. Rating Substantial Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) OBJECTIVE 3 Objective To scale-up the adoption of improved technologies in the participating countries' priority agricultural commodity areas. Rationale Outputs  Direct project beneficiaries 2,735,000 persons (22 percent above targeted 2,250,000) (targets achieved in all three countries.)  Female beneficiaries (in percent): actual 42 percent; target 40 percent.  Area under new technologies (ha): 1,850,000; target 1,700,000) (all countries reached target)  Beneficiaries who have adopted at least one new technology 1,619,000 (target 1,240,000)  Producers/processors who have adopted at least one new technology: 1.6 million actual compared with 1.24 million targeted. (all countries achieved their target).  Certified foundation seeds produced (metric tons): Maize: actual 6,000 (target 3350); rice: actual 3,600 (target: 4,600); sorghum: actual 2,007 (target 300). Outcomes and Intermediate Outcomes The outputs reported above, some of which could arguably be designated intermediate outcomes, indicate that the project’s objective to scale-up the adoption of improved agricultural technologies was mostly successful. Particularly indicative are the number of beneficiaries of new technologies and the farm area using new technologies – both increased beyond targets. A beneficiary satisfaction survey, one measure of outcome, showed that 67 percent of beneficiaries were “satisfied” with the extension services, close to the target of 70 percent. The Efficacy of the objective to Scale-up the Adoption of Improved Agricultural Technologies is rated Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project was generally successful in achieving its objectives. The number of new technologies exceeded targets, and, importantly, the project created or improved an institutional structure and established the processes and institutional support for future development of new technologies (Objective 1). Dissemination Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) (Objective 2) targets (such as the number of demonstrations, number of farmers, training, linking the extension system to the web to enhance outreach to farmers) was achieved. Adoption rates (Objective 3), as assessed by the number of beneficiaries and the size of the area using new technologies, exceeded targets. Although there is some variation in achievements, each of the three countries had significant gains in institutional capacity for technology development and adoption. The efficacy of all three objectives was substantial. The Overall Efficacy is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Viability The project’s economic rate of return was estimated in the ICR at 15.0 percent based on farm-models with data collection supervised by the project’s coordination units. Findings from independent surveys were also used. The Gross Margin Analysis indicates significant increases in profitability of farmers who adopt new technologies – most increases being between 50 and 100 percent: e.g. Senegal rice 63 percent increase in profits; Mali irrigated rice 66 percent increase; Mali sorghum 44 percent increase; and Ghana cassava 57 percent increase. The ERR estimate is moderately robust to changes in costs and benefits - the ERR falls to 13.5 percent when benefits decrease by 20 percent; and to 13.75 percent if costs increase by 20 percent: both scenarios are still viable. (The ICR reports the long term opportunity cost of capital in agricultural production as 12%, although the source of this is not given.) However, there is a methodological question on the extent to which project management and overhead costs were included in the ICR analysis. The ICR is unclear on this, the data and discussion referring primarily to gross margins of individual technologies. On the other hand, the ICR notes that the analysis did adjust for spillover effects between countries but applied a range of low and high (more optimistic) scenarios. Cost Effectiveness The project was completed with most implementation activities at or above targets. For two key monitoring indicators of project achievement – the number of beneficiaries (users of the new technologies) and the area using the new technologies, the project, achievements were 21% and 9% respectively above appraisal targets. On the cost side, at completion actual cost of the overall project (US$ 186.6 million) was 9 percent lower than the appraisal estimate of US$ 200 million. So achievements were greater than targeted while costs were lower. Implementation Efficiency The original project period of 5 ½ years was extended during project implementation by one year to 6 ½ years. The extension was primarily to provide sufficient time for the Mali program to complete due to unforeseen conflict leading the Bank to stop processing new lending. Effectively, Mali lost over a year of the original 5 ½ year project period. During the extra year, a value chain for groundnuts was added to the Senegal program but without changing project components. The Ghana program was completed and closed on the closure date Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) scheduled at appraisal. Overall, given that the security related impediments for Mali were exogenous to the project, and that the continued activities in Senegal were additional to the original program, project implementation is assessed as efficient. Overall Efficiency The project is assessed as economically viable, although, as noted above, the treatment of overhead project costs is not entirely clear. The incomes of beneficiaries increased by over 50 percent. With lower than anticipated project costs and achievements generally higher than anticipated. The project was cost-effective; and, while the project implementation period was extended by one year, this was primarily due to the late start of the Mali program due to circumstances beyond the project’s control. Overall, the Efficiency of WAAPP-2A is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 100.00 ICR Estimate  15.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project’s Relevance was Substantial in that it responded to an important need in all three countries as expressed in the strategies to increase agricultural productivity, a fundamental need for poverty reduction, export earnings and economic growth. The use of cross-regional partnership and international linkages in research and innovations was an effective and efficient way of improving agricultural technologies, the core need for improving productivity; and both Bank and Government policies encompassed the strategy that the project was embracing. Efficacy was also Substantial, forged through the development of national and regional laboratories, links with international centers of expertise, improved dissemination through demonstration plots; the extension service; and a web-based information system; all contributing to an adoption of technologies greater than targeted. With significant increases in farm incomes, a viable economic rate of return, and costs held to below appraisal estimates, Efficiency, on balance, is rated Substantial. With relevance, efficacy and efficiency Substantial, the overall Outcome of the project was Satisfactory. a. Outcome Rating Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Satisfactory 7. Risk to Development Outcome Inadequate financing to sustain the developments is the primary risk. During project implementation, discussion during supervision of the financial provisions needed from each government, stimulated provision of the the funds required by CORAF and the three participating countries. With the project over, such regular reviews are not guaranteed and there are no line items exclusive to the WAAPP program in the country budgets. Donor funding, including a follow-on Bank project will likely alleviate financial constraints in the short and medium term. But this is not a longer-term solution, and the medium and long-term commitment of the governments involved cannot be assumed, hence compounding the risk of unreliable funding. A guaranteed financing mechanism, such as self-generated funds from sale of seeds or other services, would be better, as recognized by the governments, CORAF and the Bank. See below. While not significantly covered in the ICR, the task team has advised IEG of a promising development towards self-sufficiency in all three countries and CORAF. This is the initiation of institutions and procedures for self-generated funding. Ghana is the most advanced, and is expected to pass a law based on the Ivorian self-sufficiency model, which will allow for collection of funds from producer organizations to finance research. Achieving self-sufficiency, however, will likely be a gradual process. Most farmers are poor smallholders, whereas financial contributions are easier from commercialized producers or producer organizations. Nevertheless, long term-prospects for self-sufficiency of the research/technology sector appear promising. 8. Assessment of Bank Performance a. Quality-at-Entry The project was well designed, conceptually and in its operational features. The Task Team learned from the experience of WAAPP-1A, aimed to strengthen CORAF, the operation’s lead player, and developed linkages with international centers of excellence such as CGIAR. The necessary close linkages between the three countries and CORAF were fostered through integrated partnership between the institutions involved. The preparation team also arranged for Integration of the WAAPP program within national budgets and development programs. Overall, project design was sufficiently focused and detailed to establish an operational base for project implementation. However, project formulation needed a more results-based approach, and the limited attention to this appears to have affected, in particular, the M&E program (section 9). Implementation achievements (e.g. number of varieties generated, number of demonstration plots, number of beneficiaries) provide important information but a broad assessment of impacts (e.g. yields, farmer incomes) would have helped focus on the ultimate goal - to increase agricultural productivity. Also, consideration of how the program could have eventually become financially self-sufficient from self-generated funds was not assessed at appraisal, yet such considerations might have affected institutional aspects of how the project was formulated. For these reasons, Bank performance in Project Design is rated Moderately Satisfactory Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Supervision missions were regular, benefitting from a largely Africa-based regional staffing, and the proximity of staff to the three countries helped in resolving the difficulties of implementing a multi-country program requiring complex integration between countries, and national and international centers of excellence. Fostering the institutional and operational adjustments under the project required adaptability and perseverance. Performance benchmarking was one tool used to instill competition between programs and generated healthy competition between the countries. Nearly all operational targets were met or exceeded, and Bank supervision played a key role in these achievements. The limited focus on results in project design remained largely unchanged throughout project implementation. The ICR has limited discussion on prospects for financial self-sufficiency through self-generated funds, but IEG discussion with the TTLs found some initial progress towards this which the Task Team has fostered. However, there were some significant fiduciary issues towards the end of the project. The ICR reports (para 78, Bank Performance, Quality of Supervision) - "Several fiduciary issues have been recorded (in Mali) at the end of the project with unpaid balance to suppliers and civil works not complete due to lack of resources. The overall Quality of Supervision is rated Moderately Satisfactory. Overall Performance of the Bank The Bank performed well in most aspects of project design, enabling a strategically important and innovative program to go forwards, despite the complex, multi-country, nature of the program. But the Bank could have placed more focus in design and M&E on true outcomes in terms of production and profitability. Project supervision was generally sound despite the challenge of a multi-country operation, but there were fiduciary issues towards the final stages of the project. Long-term financial self-sufficiency could have received greater attention. The overall Performance of the Bank was Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) a. M&E Design M&E design built on the earlier WAAPP operation, making some improvements in assessing the theory of change, and other aspects. The six monitorable indicators for achievement of WAAPP-2A’s objectives were supported by generally well-chosen intermediate results indictors and most of these are quantitative in nature (e.g. new crop varieties registered in the regional catalogue), hence measurable. M&E Operational Manuals were prepared at regional and country levels describing data collection methods, responsibilities, and definitions. One gap was the limited disaggregation of results and background on how results were attained. Thus, measurement of the “number of technologies generated” conveys no information of what broad technology types were introduced, whether these were mainly new seed varieties and agronomic improvements. A shortfall in the design of the M&E program and the core monitorable indicators was the limited focus on results. For instance, the "number of beneficiaries" of the project conveys achievement of the actions that contributed to the number of beneficiaries, but not the results in terms of yields and income. Yet the overall goal in the Government's and Bank' rural sector strategy is increasing productivity; Several "results- based" indicators could have been chosen to include consideration of benefits as well as the physical indicators already included in the Results Framework (ICR Results Framework at Annex 1). b. M&E Implementation Each country had M&E units and the M&E coordinator in CORAF provided backup including workshops and support missions. Considerable human and financial resources were applied to the M&E programs. A regional web-based platform for data collation at country and regional levels was created. c. M&E Utilization M&E data was used as an operational management tool by CORAF, by each participating country, and during Bank Implementation Support Missions. Results were communicated transparently and formed a benchmarking base used for countries to adjust aspects of their programs. CORAF and each country M&E specialist investigated weaker performance areas to identify needed operational adjustments. The M&E system was also used during Bank missions to gauge project progress and to identify areas for improvement. Overall Quality of M&E The overall quality of the M&E program was good in terms of: the outcome and intermediate project implementation indicators used; the creation of an institutional base for the M&E program; and the organization of data collection, and overall, could outweigh the gap in results monitoring. The Quality of M&E is, thus assessed Satisfactory. M&E Quality Rating Substantial Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) 10. Other Issues a. Safeguards The project was classified as category B, but the safeguards triggered are not specifically listed although the text of the ICR (paras 72 and 73) provides some indication of which safeguard issues were addressed. The PAD also does not specify the safeguards triggered. The comments below include clarifications provided by the task team. Environment: A regional strategy and individual strategies for each country; and country action plans were implemented according to the strategies. Each country also appointed focal points to coordinate implementation using existing institutions to this purpose. Integrated Pest Management was practiced as appropriate. A Climate Change Adaptation program was carried out by CORAF, but was limited to assessing yield gaps and developing a predictive model for yields, planting and harvesting dates. Social: The primary social activity described in the ICR is a gender program. In particular, the project promoted technologies favored by women (e.g. marketing, small scale livestock). And the ICR advises that the project “mainstreamed gender considerations in technology generation and adoption. The ICR reports that there was no Involuntary Resettlement. b. Fiduciary Compliance Financial: A financial specialist was located in each country and in CORAF. Audit reports were sometimes delayed. Reviews of financial management and procurement were conducted by the Bank during project preparation and in all three countries fiduciary arrangements were considered adequate to meet the Bank’s minimum requirements. The ICR refers to "several fiduciary issues" at the end of the project with unpaid balances to suppliers and civil works not complete due to lack of resources (size of balances and actions taken not provided). Procurement: The ICR advises that Annual Procurement Plans were produced and reviewed during Implementation Support missions. No further details are provided, but the timely implementation of the project in two countries (Ghana and Senegal), in a relatively short project period, indicates expeditious procurement processes (though not necessarily an indication of quality). The Mali, part of the construction program was incomplete at project closure. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The project's implementation and management experience yield the following lessons, of which lessons 1 and 4 are adaptations from the ICR: 1. A regional (multi-country) approach for a technology development program enables better access to international expertise, economies of scale in researching technologies, cross-border exchanges of new technologies, and potential for constructive competition between participating countries. WAAPP-2A, supporting a combination of countries, was large enough to get technical support from international centers of excellence, whereas each country alone would have attracted less support. Likewise, research and dissemination of some technologies Is too costly for any one of the countries acting alone. In this case, performance benchmarking encouraged constructive competition between the countries. 2. .A technology development program needs to be results-based. In this case, the ICR reviews the development and adoption of technologies without articulating the results of such technologies. The focus in the results framework is on numbers of participants, area under new technologies, number of technologies, etc. The ultimate outcome of a project such as WAAPP-2A would be increased productivity and incomes. Without reviewing results (yields, net incomes, and, as relevant, labor savings, soil water retention, etc.) the ultimate outcomes from new technologies are less evident. 3. An integrated combination of agricultural research and extension supported by a media agricultural information system can be an effective way of transmitting knowledge and encouraging adoption of new technologies. In this case, WAAPP-2A linked agricultural extension and research. Ghana in particular developed a web-based agricultural information system with 880,000 “hits” per annum. Ghana used the web as a core part of farmer outreach. The extension staff used web information in their contact with farmers, leveraging its impact. Importantly, the research program was applied research focused on agricultural needs and contained a grant program to finance independent activities to generate technologies. Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) 4. A sustainable funding mechanism is essential to assure long-term sustainability of a regional and cross-country technology development program. In this case, the overall WAPPAA program relies on the political will of participating countries for funding of the national and regional programs. This is based on each country’s annual budgeting process with year by year prioritization between sectors and within sectors influenced by political considerations which may provide insufficient or unreliable funding. As a promising initiative, the Ghana WAPPAA -2A program has initiated a mechanism for producer organizations to contribute funding for the agricultural research and technology development programs. Agricultural commercialization is expected to make such financing easier. 5. Building the institutions, expertise and processes for development of agricultural technologies is a long-term process requiring continued financial and technical support. This project was an Adaptable Program Loan, a more suitable long/medium-term financing vehicle than a conventional loan, and an instrument that could be increasingly adaptive as experience is gained. 6. Safeguards need clear articulation in the PAD and clear corresponding assessment in the ICR, including review during project implementation enabling adaptation as needed. The ICR’s discussion of environmental and social safeguards, and, to a lesser extent, fiduciary matters, provided few details on issues and actions, and the safeguards themselves were not tabulated. The PAD had similar shortcomings. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain Assessment of WAAPP-2A as situated within the overall WAAPP program would provide learning. The performance and issues of each participating country would be valuable to draw lessons from the cross- country regional experience and the successes and issues encountered as the program developed. Also, better understanding of the outcomes of the enhanced adoption is needed and of the financial sustainability prospects of the program. 14. Comments on Quality of ICR The ICR is a thoughtful and forward-looking review and constructively addresses issues. It identifies problems and shortfalls and offers suggestions on future strategy. The lessons section discusses possible operational improvements. The section on efficacy reflects this operational focus but it analyzes achievements by component rather than by Development Objective, which is the guideline approach for ICRs. Analysis by Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review West Africa Agric. Productivity Prog. 2A (P129565) Development Objective, would have given a greater strategic orientation in this section and elsewhere, particularly on the broader agricultural sector need to enhance productivity and incomes. And, with productivity in mind, the effectiveness of the new technologies – e.g. measured impacts of improved seeds technologies would have added understanding of the farmer adoptions made under the project, and offered directions for the future. Other areas of improvement would have been: (i) the section on safeguards (environment, social and fiduciary) is unstructured, does not clearly state what safeguards were triggered, and provides insufficient information to assess issues; (ii) information on what the broad categories of technologies were, their relative importance in project implementation, and typical productive impact of technology types (since provided to IEG by the task team) and yields; (iii) the promising initial actions towards self-sufficiency from self-generated funding could have been discussed; (iv) the ERR analysis was not clear on whether overheard costs were included; and (v) fiduciary issues are referred to but not clearly explained. Nevertheless, given the ICR’s overall strength in discussing operational improvements, the Quality of the ICR is rated Substantial overall. a. Quality of ICR Rating Substantial Page 17 of 17