Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Report Number: ICRR0021684 1. Project Data Project ID Project Name P126440 RW: Third Rural Sector Support Project Country Practice Area(Lead) Rwanda Agriculture L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-50640,IDA-54030 30-Oct-2017 90,522,713.22 Bank Approval Date Closing Date (Actual) 01-Mar-2012 30-Oct-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 80,000,000.00 0.00 Revised Commitment 95,900,000.00 0.00 Actual 90,522,713.22 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives This project was the third and final phase of an Adaptable Program Loan (APL) supporting the Rural Sector in Rwanda since 2001. Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) The Project Development Objective (PDO) stated in the Project Appraisal Document (PAD, p. 4) and the PDO in the Financing Agreement (p. 5) were identical and aimed to: "(i) Increase the agricultural productivity of organized farmers in the marshlands and hillsides of sub- watersheds targeted for development in an environmentally sustainable manner; and (ii) Strengthen the participation of women and men beneficiaries in market-based value chains." b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components The PDO was supported by three components: 1. Infrastructure for Marshland, Hillside and Commodity Chain Development (appraisal Cost US$65.4 million, actual cost: US$74.5 million). This component aimed to expand irrigation in cultivated marshlands through rehabilitation and development; promote sustainable land management practices on associated hillsides; and improve economic infrastructure in support of commodity chain development. The component has three sub-components: 1.1. Marshland Rehabilitation and Development. The sub-component would finance rehabilitation and development of selected irrigation schemes in marshlands totaling 6,000 ha with high potential for commercialized agricultural production. It would finance preliminary, detailed feasibility and participatory design studies (some of which have already been completed or are on-going), construction and construction supervision. Investments would be demand-driven and a clear selection framework would be applied. 1.2. Sustainable Land Management on Hillsides. This sub-component would aim at the development of economically interesting sustainable land management (SLM) on the hillsides. Drawing from and adapting the successful approaches of the LWH as well as the best practices developed under first and second phases, the project would finance investments in improving productivity on 17,000 ha of those hillsides directly adjacent to the marshland irrigation schemes to be developed by the Project. Activities supported by the Project would include: promotion of sustainable land management on hillsides immediately adjacent to the marshlands where irrigation investments would be done, using the 3:1 area ratio used in phase 2; and (b) promotion of cost-effective soil moisture retaining technologies on these hillsides for agricultural production. Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) 1.3. Rural Investments for Economic Infrastructure. This sub-component would invest in the construction of economic infrastructure for developed marshlands and hillsides to support the integration of organized farmers in diversified value chains. These commercial infrastructure investments would support the economic activities handled by cooperatives or small farmer groups. A Community-Driven Development (CDD) approach would be used and be directly linked to business plans developed by cooperatives with support from sub-component 2.3. Selection criteria have been developed to help prioritize the hillside cooperatives that could best benefit from similar investments. 2. Capacity for Marshland, Hillside and Commodity Chain Development (appraisal cost: US$7.5 million, actual cost: US$5.4 million). This component aimed to provide multi level capacity needed to maximize beneficiary gains from the infrastructure investments and to ensure the sustainability of Project objectives beyond the life of the APL. It includes three sub-components: 2.1. Capacity Building for Farmer Organizations and Cooperatives. This sub-component would support group formation where necessary (e.g., hillsides), and would strengthen Water User Associations (WUAs) and cooperatives to improve their governance and management capacity to deliver quality services to their members. 2.2. Capacity Building for Improved Production Technologies. This sub-component would support activities to improve production and productivity in the marshlands and hillsides adjacent to marshlands. In line with Government policy for extension and in collaboration with Rwanda Agricultural Board (RAB), the Project would support the upscaling of the Farmers Field Schools (FFS). The FFS approach builds capacity and empowers farmers to use improved and economically viable practices for sustainable soil, water and pest management with a view of increasing agricultural productivity and profitability. 2.3. Capacity Building for Value Chain Development. This sub-component would aim at building the capacity of farmers for value chain development through enhancing their understanding of agribusiness principles. This sub-component would build the capacity of farmers for market-oriented farming at three levels: the producer, cooperative and the agribusiness center level including half bulk markets. 3. Project Coordination and Support (appraisal cost US$5.5 million, actual cost: US$10.5 million). This component would finance the implementation and management of the project activities. This component covered Project management costs, such as staff salaries, equipment, operational expenses, as well as implementation of environment and social safeguards and of monitoring and evaluation. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total cost of the project was estimated at appraisal to be US$85.00 million. The actual cost reported by the ICR Annex 3 was US$90.40million which represented about 113% of the appraisal estimate. Financing. The project was financed through an IDA Credit worth US$80.00 million. The project received an additional IDA Credit worth US$15.9 million. The total disbursed amount according to the ICR (p. 2) was Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) US$90.52 million including US$76.01 million from the original loan and US$14.51 million from the additional financing. Borrower Contribution. The Government of Rwanda was expected to provide US$5.00 million of counterpart funds mainly for the payment of resettlement costs. The ICR did not provide details on the actual Government contribution. Dates. The project was approved on March 1, 2012 and became effective on June 20, 2012. The Mid-Term Review was conducted on November 2, 2015, about three years and five months into implementation, no date was set in the PAD for the MTR. The project closed on October 30, 2018 compared to an original closing date on October 30, 2017. The closing date was extended by 12 months to allow enough time to implement the project activities after receiving additional financing in 2014. The project was restructured twice, both level 2 restructurings.  The first was on April 3, 2014, when the amount disbursed was 46% of the budget (the Restructuring Paper did not give an exact amount), following the processing of an additional IDA credit to the Republic of Rwanda in the amount of US$ 15.9 million. Three changes were made: (i) the Project closing date was extended by 12 months, through October 30, 2018; (ii) the results framework was revised to increase the targets for four indicators to reflect the additional resources available; and (iii) resources were reallocated across disbursement categories.  The second restructuring was on December 5, 2017, when the amount disbursed was US$89.83 million, in order to reflect a transfer of the Project implementation responsibilities from the Ministry of Agriculture and Animal Resources (MINAGRI) to the Rwanda Agriculture and Animal Resources Development Board (RAB). "The transfer was part of a Government-wide reform aiming at increasing the efficiency of public administration, by focusing the role of line ministries on policy- making and monitoring, while tasking designated agencies with the implementation of policies and programs (ICR, p. 10, para 19)." 3. Relevance of Objectives Rationale The agricultural sector in Rwanda accounts for about 36% of Gross Domestic Product (GDP), 80% of employment, and in 2010, 45% of foreign exchange earnings. It also provides 90% of the country’s food needs. Key challenges facing the agriculture sector include: (i) a binding land constraint that rules out expanding land area for agricultural growth, (ii) small average land holdings (0.3 ha); (iii) poor water management (uneven rainfall and ensuing variability in production) resulting from very low levels of irrigation (15,000 ha nationwide); (iv) the need for greater (public and private) capacity from the district to Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) the national levels and the weak extension services for farmers; and (v) limited commercial orientation constrained by poor access to output and financial markets (PAD, p. 1, para 2). A. Relevance of the Objectives of Phase 3. At project appraisal, objectives were in line with the Government’s strategic planning for the agriculture sector. The project started at the transition between Rwanda’s Strategic Plan for the Transformation of Agriculture Phase II (PSTA II), covering 2009-12, and the Plan’s third phase, PSTA III, covering 2013-17. Objectives were in line with PSTA III which explicitly outlined a vision for agriculture development based on intensification and commercialization, which were also the two pillars driving the project. Objective were also in line with the Bank’s Country Assistance Strategy (CAS) for Rwanda for FY09-12. Specifically, it was designed to contribute to its Outcome 1.1. (sustainably raising agricultural production, particularly of food crops). The CAS reflected two strategic themes: (a) promote economic transformation and growth, and (b) reduce social vulnerability. Agriculture production, with an emphasis on sustainable approaches, was one of the four components of the CAS’s first strategic theme. Objectives were also in line with the World Bank’s Africa Strategy, “Africa’s Future and the World Bank’s Support to It” (March 2011). It supported competitiveness and employment, vulnerability and resilience, and governance and public sector capacity. At project completion, objectives continued to be in line with the Government vision and strategy reflected in the Rwanda’s Vision 2050. The National Strategy for Transformation 2017-24 lays out the economic, social and governance pillars that would support achieving Vision 2050; through increasing agriculture productivity and production, as well as sustainably exploiting natural resources remain key objectives of the economic transformation. Objectives were also in line with PSTA IV which coincided with the time of project completion. PSTA IV continues and accelerates changes begun under PSTA III to enhance productivity and profitability and encourage private investment. Objectives were also in line with the Bank's Country Partnership Strategy (CPS, FY14-18) for Rwanda. The CPS included three themes, and agriculture was a key focus sector under the second theme. The CPS (p. 32, para 87) was explicit in stating that intensifying agricultural productivity would remain central in the IDA program, and that investments would equally facilitate transition to more commercial farming practices. The CPS (p. 33, para 90) also stated that IFC would focus on improving advisory services on horticulture in Rwanda and agri-business investments, thus complementing and reinforcing the effects of the project interventions. The PAD (p.4) described the project beneficiaries as "female and male farmers in the selected marshlands and adjacent hillsides, as well as community members receiving project support in small groups for value chain activities, either upstream or downstream." The first part of the PDO was clear and focused. However, using the term "women and men beneficiaries" in the second part of the PDO implied a target group although according to the PAD there was none. Based on the above-mentioned information, relevance of objectives is rated Substantial. B. Relevance of the Program Objectives. Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) The long-term programmatic objective of the Rural Sector Support Project (RSSP) APL series (2001 to 2019) was "to help the Government of Rwanda achieve its strategic goal of unlocking rural growth in order to increase incomes and reduce poverty" (PAD, p. 5, para 11). Relevance of objectives of phase 1 (P064965) and phase 2 (P105176) of the program were both rated High by IEG. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective PDO1: to increase the agricultural productivity of organized farmers in the marshlands and hillsides of sub- watersheds targeted for development in an environmentally sustainable manner. Rationale Theory of Change. The theory of change is a theory about the relationships between activities, outputs from those activities and the final outcome. In this case change was facilitated by a critical element such as the continued capacity of water user associations (WUAs) to effectively provide services to their users, while being able to recover their water fees, and the ability of cooperatives and other aggregators to find market outlets for their farmers’ produce. Activities and expected outputs. The project was designed to focus on improving agribusiness and marketing skills among its beneficiaries, improving quality and reducing post-harvest losses through better post-harvest technology and handling. These activities were expected to result in training lead farmers and extension agents on improved farming methods and post harvest handling. Also, post-harvest infrastructure would be built and equipment provided through project support. Irrigation infrastructure would be rehabilitated and small-scale irrigation technologies provided to beneficiaries, and Water User Associations would be set up, trained and formally registered. Expected outcomes. Farmers were expected to adopt sustainable land management practices which would lead to developing hillside areas sustainably. Farmers would also adopt improved production technologies and cooperatives would produce certified seeds in areas with improved irrigation and drainage. These activities combined would lead to increasing productivity in marshland and adjacent hillside areas in an environmentally sustainable manner. Expected long term outcomes. Sustainable management of rural natural resources and greater income diversification in rural communities. Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Based on the above, the causal links in the theory of change are evident, and the suggested activities were relevant and adequate to achieve the PDO. Outputs The information below was reported in the ICR (Annex 3) except when otherwise referenced. Sustainable Land Management activities:  4,117,720 agro-forestry trees planted for soil erosion control. 1,657,192 trees planted for ecosystem rehabilitation.  149,000 trees and 184,440 shrubs planted as part of the silt trap zone for dam protection.  9,312 ha benefited from improvements through radical terracing (ICR, p. 12, para 28).  2,676.8 ha were covered with forest, while 3,283 ha of embankment were protected with grasses and another 2,398 with agroforestry trees (ICR, p. 13, para 29).  1,172 ha of marginal land in the marshlands and hillsides were brought into productive use (ICR, p. 13, para 29).  29,780 beneficiaries (of which 14,579 women) were trained in sustainable land management practices.  10,464 farmers trained in compost making Improving Irrigation and Drainage services:  70,932 male beneficiaries were provided with improved drainage and irrigation services compared to a baseline of: 33,473, original target of: 63,473 and a revised target of: 64,973. Also, 30,707 female beneficiaries were provided with improved drainage and irrigation services compared to a baseline of: 13,389 original target of: 25,389 and a revised target of: 26,019.  6 new small dams/reservoirs built and 3 sites developed with river weirs  17 new WUAs were set up and the project strengthened the capacity of 42 WUAs (target: 38). 1,923 water users (farmers) were trained. Improving Productivity and Yields:  1,936 lead farmers (hillsides) and 60 lead farmers (marshlands) trained in seed production  19 rice varieties tested in 9 marshlands  25 farmer field school plots set up in the marshlands, and 2,500 farmers trained on various issues including pest and disease control as well as rice harvesting  850 farmer field school plots set up in the hillsides  2,180 lead farmers trained on lead farmer extension approach  28 co-operatives were involved in certified seed production compared to a baseline of: 7 and a target of: 17. Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Outcome This outcome is assessed through three elements: (a) increasing the agricultural productivity in irrigated marshlands, (b) increasing the agricultural productivity in the marshlands and hillsides of sub-watersheds, and (c) promoting environmental sustainability. (a) Agricultural productivity in irrigated marshlands.  The evidence provided in the ICR showed that agricultural productivity in irrigated marshlands increased by 208% from a baseline of US$662 per ha to US$2,865 per ha exceeding the end target of US$1,375. Taking the 30% devaluation in the Rwandan currency from 2012 to 2019 into account, the end-of-Project value for marshland productivity would be US$ 1,955/ha, which was 3 times more than the baseline.  Rice, the main crop cultivated in the irrigated marshlands, saw yields increase by 25% from about 4 tons/ha, at baseline, to about 5 tons/ha at project closing. While the national average was 3 tons/ha at appraisal and 3.4 tons/ha at project closing. These increments in productivity/yields resulted from 4,103 ha that were newly developed, and 3,194 ha were rehabilitated. Also, through more reliable irrigation water availability to areas previously deprived of water, benefiting nearly 71,000 users.  In addition, the use of better quality certified seeds where according to the ICR (p. 12, para 27) the use of improved seeds increased from 5% in 2013 to 80% in 2018. The project-supported farmers field schools enabled beneficiaries to improve their knowledge and skills and provided them with relevant training to use better technology and/or management practices. (b) Agricultural productivity in the marshlands and hillsides of sub-watersheds.  The ICR (p. 11 table 1) reported that the agricultural productivity in non-irrigated marshlands and hillsides of sub-watersheds increased by 109% form a baseline of US$470 per ha to US$1,133 per ha by project completion, exceeding the target of US$1,038 per ha.  Yields of different crops cultivated in the hillsides also increased between the start of the Project and its closing, for example, in the project areas maize yield increased from 1.5 tons/ha to 3.5 tons/ha, against a national average of 1.8 tons/ha; bush beans from 0.3 tons/ha to 1.3 tons/ha, against a national average of 0.8 tons/ha; climbing beans from 0.8 tons/ha to 2.6 tons/ha, against a national average of 1.0 tons/ha, and Irish potatoes from 3 tons/ha to 21 tons/ha, against a national average of 8.6 tons/ha (ICR, pages 12 & 13, para 28).  These improvements were a result of activities promoted and supported by the project including: improved land management practices, adoption of improved technologies and practices and better farmers’ knowledge and skills through training and farmer field schools. (c) Promoting environmental sustainability.  The project promoted a number of sustainable land management practices and rain water harvesting technologies including: grass strips, contour bunding and improved radical terracing; pasture improvement through trees and grass planting; dam and canal buffer zone protection; afforestation of Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) critical hillside ecosystems unsuitable for intensive agriculture and animal production and shallow soils, and construction of bench terraces and ditches for soil erosion control.  Also, other techniques such as liming and organic materials application were introduced, while crop rotation, double cropping and contour cultivation and mulching were encouraged. According to the ICR (page 11, table 1), the share of farmers adopting sustainable land management practices, both in the hillsides and in the marshlands, increased to 95.5% for women and 96.5% for men from a baseline of 32% and 36% for women and men, respectively, and against a target of 90% for both.  Further, to curb soil erosion the Government adopted an environmentally-friendly zero grazing policy. This was possible after the project supported the protection of embankments with grasses, and the areas planted agroforestry trees were used for fodder production for livestock (ICR, p. 13, para 29). It is plausible to assume that marshlands were better protected against soil erosion as a result of infrastructure and better water management supported by the project, despite the lack of evidence on this point in the ICR. Based on the above-reported information, the evidence provided points to the success of the project in increasing agricultural productivity of organized farmers in both the marshlands and hillsides of sub- watersheds targeted for development. There is also evidence that this was done in an environmentally sustainable manner. However, more time is needed to assess the environmental impact of the project activities since benefits are slow to materialize. This review concludes that the efficacy of the achievement of this objective is Substantial. Rating Substantial OBJECTIVE 2 Objective PDO2: to strengthen the participation of women and men beneficiaries in market-based value chains. Rationale Theory of Change. The theory of change is a theory about the relationships in this project between activities, outputs from those activities and the final outcome. In this case change is facilitated by a critical element such as the ability of cooperatives and other aggregators to enable the market integration of women and men in targeted marshland and hillside areas by intensifying production, promoting diversification into agricultural value addition or upstream markets, and expanding access to markets. Activities and expected outputs. The project would support mobilization and formation of new cooperatives. Also, the project will support training of farmers in business planning and agribusiness development. These activities would result in more produce sold through cooperatives, and the trained cooperatives would have better access to finance. Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Expected outcomes. Project-supported co-operatives were expected to see their revenues increase by more than 50% which is a sign of increased share of commercialized output, and increased participation of targeted beneficiaries in market-based value chains. Also, more women and men beneficiaries would participate in up- stream and down-stream value chain activities. Expected long term outcomes. Greater professionalization in agriculture and greater income diversification in rural communities. Outputs The information below was reported in the ICR (Annex 3) except when otherwise referenced. Development of Market-Based Value Chains.  21 storage facilities, 62 rice drying grounds and 9 maize dryers and collection centers were built.  4,100 palettes, 18 moisture meters, 30 weighing machines, 16 combined maize threshers and winnowers, 11 maize shelters, 25 collapsible dryers distributed or built for demonstration.  The Project facilitated the setting up of 3,382 Self-Help Groups (SHGs), each counting 20 to 30 members; the SHGs were further organized into 355 zones, and then into 51 cooperatives, 15 in the hillsides and 36 in the marshlands (ICR, p. 13, para 30).  40 cooperatives (baseline: 6, target: 30) out of 51 were implementing projects with funding from commercial banks or other financial institutions at project closing.  1,117 farmers (54% women) received refresher training on post-harvest and handling, entrepreneurship and business plans. Participation of Women and Men in Markets  9,695 farmers (of which 4,957 women) were trained on community mobilization and cooperative/group governance.  31 study tours were organized for 1,047 beneficiaries (33% women) on how to improve coop management, crop production and marketing, access and use of financial services, women’s empowerment, business creation. Outcome  To strengthen the participation of women and men beneficiaries in market-based value chains, the project supported capacity building activities, including group formation and mobilization of farmers as well as activities to improve post-harvest handling and marketing. As a result of these activities, women and men farmers improved their skills to market their products and improved their ability to attract funding. The share of commercialized agricultural products, both in the marshlands and in the Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) hillsides, as well as the number of beneficiaries linked to upstream and downstream value chain activities increased.  The ICR (p. 11, Table 1) reported that the share of commercialized agricultural products in marshlands (measured as total value of crops sold over total value of crops produced) increased from a baseline of 43% (women), 45% (men) to end target of 93% for both exceeding the target of 90%.  Also, the share of commercialized agricultural products in hillsides (measured as total value of crops sold over total value of crops produced) increased from a baseline of 43% (women) 45% (men) to an end target of 78% (women) and 79% (men) compared to target of 60% for both. Also, by project completion, 33,973 project beneficiaries were involved in up- and downstream value chain activities compared to target of 6,206 and a baseline of 896.  While there is ample evidence reported in the ICR on the participation of women and men in different activities, there was no reference in the ICR to the relative importance of these activities to either women or men. Therefore, while the PDO refers clearly to women and men, this review looked at women and men as an aggregate due to the absence of any information in the ICR on the relative importance of activities to each gender. Based on the above-mentioned information, efficacy of this objective is rated Substantial. Rating Substantial OVERALL EFF TBL OLD Rationale PDO1: to increase the agricultural productivity of organized farmers in the marshlands and hillsides of sub-watersheds targeted for development in an environmentally sustainable manner. Substantial. The evidence provided point to the success of the project in increasing agricultural productivity of organized farmers in both the marshlands and hillsides of sub-watersheds targeted for development. There is also evidence that this was done in an environmentally sustainable manner. However, more time is needed to assess the environmental impact. PDO2: to strengthen the participation of women and men beneficiaries in market-based value chains. Substantial. Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) As a result of project support, the share of commercialized agricultural products for women and men farmers, both in the marshlands and in the hillsides, as well as the number of beneficiaries linked to upstream and downstream value chain activities increased exceeding their end of project targets. Overall efficacy of the project is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Analysis (EFA) ex ante  The ex-ante EFA in the PAD estimated an economic internal rate of return (EIRR) of 55% for marshlands, 785% for hillsides and 112% for the entire project when using a stochastic model. The stochastic variables consisted of: capital investment costs on marshlands and hillsides; crop yields; crop output prices; labor unit costs; planting materials and unit costs; fertilizer use and unit costs (manure, NPK inorganic fertilizer, and urea); crop chemical costs; rice transport unit costs; irrigation operation and management costs; economic price of carbon sequestration; and capital and operating costs for infrastructure investments.  For an assumed pro0ject life of 21 years and a discount rate of 12% the financial net present value (FNPV) and economic net present value (ENPV) were estimated to be US$218 million and US$228 million, respectively for the entire project. An estimated 15,878 ha (5,769 ha of marshland and 10,109 ha of hillsides) were foreseen to benefit from the project. ex post  The ex-post EFA in the ICR estimated a Financial Internal Rate of Return (FIRR) of 96% and an EIRR of 88%, and a Financial Net Present Value (FNPV) of US$1.3 billion and an Economic Net Present Value (ENPV) of US$1.5 billion for the entire project. The financial and economic benefit-cost ratios (BCRs) were estimated to be 14.9 and 14.1, respectively.  The number of actual area benefited by the project was 15,560 ha which was close to the above- mentioned planned area of 15,878 ha. The assumed adoption rate for new farming practices and technologies was about 40%. The ICR (p. 14, para 34) noted that the low adoption rate assumed for the Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) analysis in the ICR was expected to be surpassed given the "broad nature of the program within the Rwandan context."  The analysis used the FAO EX-ACT tool to estimate total CO2 emissions reductions. The reduction equaled 443,149 tons over a twenty-year period. Environmental carbon reduction benefits as a percentage of total benefits equaled 4.5%, 21% and 35% for market, low and high shadow prices for carbon, respectively. This analysis used the World Bank Guidance notes on shadow price of carbon (September, 2017).  A sensitivity analysis assessed the impact of the main risks for the project results and the adverse situations that may arise in terms of benefits and costs, without environmental benefits. The analysis reported in the ICR revealed, inter alia, that under a severe scenario, using a technology adoption rate of 40%, assuming that cost overruns are 20%, benefits and output prices decrease by 20% and input prices increase by 20%, the EIRR would be 65% and (assuming a discount rate of 17%) the BCR would be 8.6 (ICR, Annex 4, paragraph 9). Administrative and Institutional efficiency The project closed on October 30, 2018 compared to an original closing date on October 30, 2017. The closing date was extended by 12 months to allow for enough time to implement the project activities after receiving additional financing in 2014. During the first few years of the project, there were some delays in the completion of design studies, which impacted on the planned dam and irrigation construction works. There were also some procurement related delays that stemmed from the lack of capacity of local contractors to undertake contracts to construct dams and irrigation works, lack of realistic procurement plan, low efficiency of internal tender committee and inadequate contract monitoring (ICR, p. 23, para 72). The actual cost of Project Coordination and Support (component 3) was US$10.5 million or 190% of the appraisal estimate of US$5.5 million. In a communication with IEG, the World Bank project team explained that several factors contributed to a cost increase relative to appraisal: first the project received additional financing of US$15.9 million, which allowed scaling up its activities and geographic scope (note that the Single Project Implementation Unit had district level as well as central level staff); second, the project got a 1 year extension; and third the pay scale for Government projects was revised upwards relatively early during project implementation, to address SPIU staff turnover. As a result, the project’s coordination and support costs increased from 6.5% of the total project cost at appraisal to 11.6% of actual total project costs at completion. Based on the above-mentioned information, efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  112.00  Not Applicable 100.00 ICR Estimate  88.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives and overall Efficacy are both rated Substantial. The evidence provided in the ICR point to the success of the project in increasing agricultural productivity of organized farmers in both the marshlands and hillsides of sub-watersheds targeted for development. There is also evidence that this was done in an environmentally sustainable manner. However, more time will be needed to assess the environmental impact. Also, as a result of project support, the share of commercialized agricultural products, both in the marshlands and in the hillsides, as well as the number of beneficiaries linked to upstream and downstream value chain activities (involving women as well as men) increased exceeding their end of project targets. Efficiency is rated Substantial. Based on a Substantial rating for each of Relevance, Efficacy and Efficiency, Outcome is rated Satisfactory. Overall Program Outcome. The outcomes of the first phase and second phases of the program were rated by IEG as Satisfactory and Highly Satisfactory, respectively. These ratings combined with a Satisfactory outcome rating for the third and last phase confirm the success of the program in achieving its stated objectives which were to: help the Government of Rwanda achieve its strategic goal of unlocking rural growth in order to increase incomes and reduce poverty. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (p. 24, para 78) rated the risk to development outcome (RDO) as low. The following points were discussed in the ICR with potential impact on RDO:  Financial sustainability of farmer organizations. 51 cooperatives were registered with the Rwanda Cooperative Agency (RCA) and all of them received the needed skill training regarding governance, financial management and agribusiness. However, cooperatives established late in the implementation cycle (including most in the hillsides) would require additional capacity building for 2-3 Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) years before achieving financial sustainability. Some cooperatives had limited resources to pay their staff and cover operational costs. Continuing the efforts to link with potential buyers, such as Africa Improved Food (AIF), Rwanda Grain and Cereals Corporation (RGCC) would help address these concerns.  Sustainability of irrigation infrastructure. A tripartite Irrigation Management Transfer Agreement (IMTAs) was signed between WUAs, the districts and the Rwanda Agriculture and Animal Resources Development Board for all the project-financed irrigation infrastructure. The IMTAs detailed responsibilities among the different agencies for the management of the irrigation infrastructure as well as water cost recovery charges. At project closing, cost recovery for the irrigated areas reached about 95%. Collected water use fees for each scheme were composed of operation and maintenance of an irrigation scheme, reserve fund for major water infrastructure repairs and an irrigation trust fund for contribution to other government irrigation programs. WUAs as well as districts noted the need for additional engineering and other skills required for system maintenance. WUAs would further benefit from continuing capacity building regarding the operation and maintenance of project-developed irrigation infrastructure. The ICR (p. 24, para 80) noted that capacity building efforts would be supported through the on-going Sustainable Agriculture Intensification and Food Security Project.  Environmental sustainability. The implementation of irrigation and drainage systems allowed the exploitation of marshlands and contributed among other things to improve rice-cropping intensification. To improve environmental sustainability, the project put in place buffer zones to handle the drainage water. Continued monitoring of water quality in the buffer zones is important to ensure that levels of pollutants are kept in check. 8. Assessment of Bank Performance a. Quality-at-Entry  It is noteworthy that this project was the third and final phase of an Adaptable Program Loan (APL) supporting the Rural Sector in Rwanda since 2001.  Apart from the reference to "women and men" in the second part of the PDO, objectives were clear and focused and in-line with the Government priorities for the agricultural sector.  Design was double focused on increasing productivity and commercialization of marshland and adjacent hillside agriculture. Design benefitted from the experience and lessons learned from phase 1 and 2 of the program. Four key lessons were reflected in the design of the third phase: (i) improving farmers’ business skills through the promotion of participatory value chain approaches, (ii) enhancing farmers’ capacity for the operation and maintenance through close coordination between the infrastructure development and capacity building, (iii) providing adequate resources for the capacity building activities, and (iv) establishing adequate budget with higher contingencies to reduce the risk of cost over-runs during project implementation. Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440)  Six risks were identified at appraisal, three were rated moderate and three were rated low. While currency depreciation risk may have been underestimated, "it was well managed using project contingency funds and did not affect the project outcomes (ICR, p. 23, para 75)."  According to the ICR (p. 23, para 75) the project had clear institutional arrangements and an adequate financial management system in place.  M&E suffered from minor design shortcomings, but overall the M&E system was sound (see section 9 for more details).  Based on the above-mentioned information, Quality at Entry is rated Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank conducted regular missions to support the project. However, the ICR did not report the exact number of supervision mission conducted. According to the ICR (p. 23, para 76) the Bank missions had adequate resources and included relevant expertise to assess progress along all project components and activities. Mission also included a specialist to cover the implementation of environmental and social safeguards aspects of the project. The project supervision missions were frequently combined with the supervision of the Land Husbandry, Water Harvesting and Hillside Irrigation project. This facilitated efficiency gains and synergies between both projects (ICR, p. 23, para 76). The Mid-Term Review (MTR) flagged the need for a budget reallocation between components, and the currency exchange loss-which was resolved by tapping into the project's contingency funds (ICR, p. 23, para 76). The project team dropped the activity that aimed to develop irrigation with shallow groundwater after detailed studies revealed that it was an unfeasible approach in the selected sites. In a further communication, the project team explained that "at appraisal it was decided that this technology was going to be piloted on only 200- 300 ha out of the targeted 6,000 ha (later 7,000 ha) of irrigated area supported by the project. However, the feasibility studies were completed by project MTR, and showed that on some sites there wasn’t enough ground water to allow the use of the shallow aquifer technology (technically not feasible), while on other sites it was too expensive to use (economically not feasible). The government and the Bank agreed, at project MTR, to drop this technology for these reasons. Given the initial small footprint of this technology, the decision did not have any significant impact on either component costs or on the ability to meet irrigation specific targets." The project's FM system was adequate throughout implementation (see section 10 for more details). Based on the above-mentioned information, Quality of Supervision is rated Satisfactory. With Quality at Entry and Quality of Supervision both rated Satisfactory, overall, the Bank Performance is rated Satisfactory despite some minor shortcomings regarding the M&E framework. Quality of Supervision Rating Satisfactory Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The PAD did not include a Theory of Change which was not required at that time of appraisal. Nonetheless, the ICR (pages 6 & 7) included a detailed Theory of Change that reflected the key assumptions and the relation between the project inputs, outputs and expected outcomes. The M&E system for phase 3 was built on that of phase 2, and was fully aligned with the Ministry of Agriculture and Animal Resources M&E system. The M&E units of phase 3 and that of the Land Husbandry, Water Harvesting and Hillside Irrigation project were merged to streamline processes and to increase efficiency at project and sector levels. The merged unit benefited from the extensive experience of Rural Sector Support Project on one hand and from the specific experience of Land Husbandry, Water Harvesting and Hillside Irrigation project working on the hillsides on the other. The achievements of the PDO were to be assessed through four PDO level indicators. These were relevant and measurable and included baselines at the appraisal stage. However, the ICR (p. 20, para 58) reported that "there were some methodological shortcomings primarily in the definition of the indicators for agricultural productivity in irrigated marshlands and non-irrigated hillsides." These were resolved during implementation. The Results Framework also included ten intermediate outcome indicators to assess the achievements under the project's components. These indicators were linked to the project activities, measurable and had baselines where relevant. Overall M&E design was sound and benefitted from the experience built under the previous two phases of the program. b. M&E Implementation The Project relied on several data collection methods. For example, production and marketing related data were collected through Project cooperatives’ seasonal reports, complemented by bi-annual surveys. Validation of data collected from cooperatives was done through field visits in a sample of plots. Infrastructure and land husbandry related data was collected by district level M&E specialists and triangulated with contractors’ reports. Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Overall, monitoring of progress across all project components was detailed and timely (ICR, p. 20, para 59). Except for adjusting targets upwards following the additional financing, there were no changes in indicators or M&E methodology during project implementation. Using the same M&E unit to monitor both projects (Rural Sector Support Project 3 and the Land Husbandry, Water Harvesting and Hillside Irrigation project) was efficient since both projects relied on the same guidelines and M&E manuals as well as the same data collection process. c. M&E Utilization The monitored indicators generated time series and comparisons across the project beneficiaries and sites, which according to the ICR (p. 20, para59) were solid and reliable. Beneficiary surveys were extensively used (biannually, for several indicators). According to the ICR (p. 21, para 61) the information generated by the project’s M&E system was routinely used for operational decisions throughout project implementation; it was also readily available for review and action during World Bank missions." The data generated by the project's M&E system was also used in the preparing the ICR. Overall, the Quality of M&E is rated Substantial. Design was sound with some minor shortcomings regarding definition of indicators, implementation was timely and generated detailed data, and utilization benefitted from the generated data to inform operational decisions. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was categorized as an Environmental Screening Category B – Partial Assessment. At the appraisal stage, the project triggered the following six safeguard policies: OP 4.01- Environmental Assessment, OP 4.04 – Natural Habitats, OP 4.09 – Pest Management, OP 4.11 – Physical Cultural Resources, OP 4.37 – Safety of Dams, and OP 7.50 – International Waterways. Environmental impacts were likely to be short term, site-specific, non-sensitive or irreversible, and in every case, mitigation measures could be designed to reduce the negative impacts. An Environmental and Social Management Framework (ESMF) was prepared by the implementation team based on the earlier ESMF prepared for the phase 2 project. The project team also updated the phase 2 Pest Management Plan (PMP) to cover the project's focus on a wider range of crops. The updated ESMF and PMP were reviewed by the Bank and found to be satisfactory. Both the ESMF and PMP were disclosed in the World Bank InfoShop (on November 28, 2011). According to the ICR (p. 22, para 66) "awareness raising campaigns and safeguards training was conducted for all implementation partners, before implementation of sub-projects began." Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) The project later triggered the policy on Involuntary Resettlement (OP/BP 4.12) and consequently 16 Resettlement Action Plans (RAPs) were prepared. There were 4,737 Project Affected Households (PAHs), of which 2 required relocation and were compensated in cash; there were 171 vulnerable project affected persons (PAPs) who were offered additional assistance. According to the ICR (p. 22, para 68) "all the project affected households (PAHs) were compensated as per the Rwanda expropriation law and World Bank policy on involuntary resettlement." The total compensation amount used for the project affected households was US$0.88 million. In total, 113.9 ha of private land were acquired for the project activities. The Project established and operationalized Grievance Redress Committees (GRCs) in all subproject sites. Ninety two common grievances were recorded, and all were resolved (ICR, p. 22, para 70). The ICR did not include an explicit statement of compliance with the triggered safeguard policies. b. Fiduciary Compliance Financial Management. According to the ICR (p. 22, para 71) the financial management (FM) system for the project was adequate. The FM benefitted from low staff turnover as this helped build and maintain capacity within the implementation unit. The quality of the financial reports improved overtime, and reports were submitted on time to the Bank with unqualified audit opinion on financial statement. The ICR (p. 23, paras 71) also reported that FM was rated satisfactory throughout the implementation period. Procurement. Procurement suffered from delays during the first few years of the project. This impacted the planned construction of some works. Initial procurement delays stemmed from "non-responsive bids due to lack of capacity of local contractors to (undertake contracts for the construction of) dams and irrigation (works), lack of realistic procurement plan, low efficiency of internal tender committee and inadequate contract monitoring function" (ICR, p. 23, para 72). Procurement benefitted from the support of the Bank procurement specialist. The implementation unit also worked with the Bank team to improve procurement performance by following the Bank's recommendations. Procurement function improved starting mid-2015. Procurement operations were carried out in accordance with the Bank's procurement rules and guidelines (ICR, p. 23, para 72). In 2017, the procurement unit suffered from understaffing and procurement rating was downgraded to moderately satisfactory. That said, the ICR (p. 23, para 73) reported that despite the staffing issue, the existing staff managed to handle the work load because the project was approaching its closing. c. Unintended impacts (Positive or Negative) Positive. Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Supplemental income and environmental benefits from composting. The Project introduced composting and trained farmers accordingly mainly to improve soil fertility on the hillsides. Farmers grouped in SHGs collected a total income of about US$588,454 from the surplus of compost sold. This benefited mostly landless rural dwellers. At the same time, composting helped combat climate change by sequestering carbon, through converting waste materials into organic fertilizers. Strengthening of the rice milling industry in Rwanda. By increasing rice production throughout the country and by better organizing its harvest and collection, the project has greatly contributed to the strengthening of the rice milling industry in Rwanda. With demand growing over the years and the project helping boost the rice supply, the rice milling sector has grown considerably, to employ thousands of traders and millers. This shows the maturity of the institutions that were supported by the project, and the potential for future sustainability. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR included three lessons. The following are emphasized with some adaptation of language:  Important impacts can be achieved when a strategically relevant program/approach is continued and perfected over time. The project experience emphasized the importance of continuity and strategic articulation. This is what happened during the three phases of the project, which built one upon another. Despite its difficult start, the first phase managed to turn around and provide valuable lessons and direction to the next two phases. The Government’s long-term vision for the agriculture sector, articulated in the Vision documents and in the successive Strategic Plans for the Transformation of Agriculture (PSTA) generations, was equally a determining success factor. It enabled pursuing a coherent policy in the sector and government commitment to stay on course, across over a decade of the program interventions, under a stable regulatory and institutional environment.  Successful agriculture-food value chain development requires the following elements: sensitization, technology dissemination, aggregation, value chain partnerships, Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) complementary infrastructure, and soft-skill development. The project focused on sensitization, demonstration plots and farmer field schools to demonstrate the achievable benefits. This facilitated technology dissemination which ensured both a reduction in the input costs and high adoption rates among producers. Aggregation of farmer organizations empowered small agricultural producers to establish themselves up-and down- stream along their respective value chains. Through supporting a holistic approach to value chain development, the project ensured that no gaps were left on the path from farm to market. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain An assessment will be useful to further validate the impact and sustainability of the three-phase program of agricultural transformation. An assessment would also be useful, inter alia, in generating relevant lessons on what worked. 14. Comments on Quality of ICR The ICR is well written. It provided thorough coverage on the project activities and candidly reported on shortcomings. Discussion of outcomes was logical and relied on what the project achieved on the ground. The ICR included a detailed ex post EFA that provided a good justification for the feasibility of the project investments. The ICR also included several data rich annexes that helped document the project achievements and implementation experience. A weakness was the absence of any explicit statements regarding compliance of project management with World Bank safeguards. Overall, the Quality of the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RW: Third Rural Sector Support Project (P126440) Page 22 of 22