Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Report Number: ICRR0021768 1. Project Data Project ID Project Name P112033 IN: UP Sodic III Country Practice Area(Lead) India Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-46400,TF-98748 31-Dec-2015 173,291,726.74 Bank Approval Date Closing Date (Actual) 30-Jun-2009 29-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 197,000,000.00 200,000.00 Revised Commitment 197,200,000.00 200,000.00 Actual 174,829,818.79 200,000.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) stated in the Project Appraisal Document (PAD, p. 5, para 14) and the PDO in the Financing Agreement (p. 4) were identical and aimed to: Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) "increase agricultural productivity of degraded lands in selected areas of Uttar Pradesh by: (i) reversing water-induced land degradation; (ii) enhancing soil fertility; and (iii) improving the provision of agricultural support services." b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The PDO was supported by five components: 1. On-Farm Development and Land Treatment (appraisal estimate: US$170.7 million, actual cost: US$183.60 million). This component aimed to sustainably reverse water-induced land degradation- salinization, sodification, and water-logging-through carefully sequenced technical interventions. Activities to be financed under this component include (a) mobilization of village communities; (b) detailed mapping and classification of sodic lands; (c) formation of water user groups (WUGs); (d) on-farm development through land-leveling, bunding, and linking field drains to link and main drains; (e) provision of shallow tube-wells to help in reclamation operations and provide irrigation; and (f) application of chemical/organic amendments and plant nutrients to the soil; and (g) cultivation of rice-wheat-green manure crop. About 130,000 ha of sodic lands covering about 25 districts would be reclaimed under this component. This project would operate within the existing land titling and land administration framework of the state. It would strengthen tenure security by helping to clarify and confirm land titles holdings as part of pre-reclamation activities. Also, a pilot on ravine reclamation covering an area of about 5,000 ha would be included under this component. The pilot would follow a watershed development approach, focusing on in situ moisture conservation, local water harvesting, reducing soil erosion, improving natural vegetation, and enhancing crop and livestock productivity. The expected results from this component would be: (a) improved soil quality; (b) increased productivity; and (c) higher cropping intensity. 2. Improvement of Drainage System (appraisal estimate: US$39.2 million, actual cost: US$23.05 million). This component aimed to improve the drainage networks in the project area to remove/leach effluents, excess rain, and irrigation water from reclaimed and adjoining areas. The activities to be financed included: (a) rehabilitation of main drains; (b) maintenance of main drains; (c) training and capacity building of Uttar Pradesh Bhumi Sundar Nigam (UPBSN) and Irrigation Department (ID) staff on technical and management aspects, and training of WUGs for appropriate O&M of drainage network. The expected results from this component would be: (a) improved drainage capacity of the drainage network; and (b) reduced water-logging in adjacent land area (with resultant improvement in land productivity). For reclamation of about 130,000 ha of sodic lands under Component A, about 5,700 km of the drainage network would require rehabilitation. Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) 3. Agriculture Support Service (appraisal estimate: US$9.4 million, actual cost: US$10.22 million). This component aimed to increase agricultural productivity through introduction of improved technology, better agronomic practices, and more effective provision of key support services. The activities to be financed included: (a) training farmers in effective land and water management practices; (b) dissemination of improved agricultural technology and production practices through on-farm demonstrations; (c) support for livestock production, including dairy development and small ruminants as appropriate; (d) exposure visits, farmer fairs, animal health camps and other “means” for rural communication and outreach; and (e) training and capacity building of line department staff and other relevant providers of support services to farmers. The expected results would be: (i) increased productivity; (ii) greater cropping intensity; and (iii) agriculture diversification (crops, vegetables and livestock). 4. Institutional Strengthening and Capacity Building for Market Access (appraisal estimate: US$9.7 million, actual cost: US$4.28 million). This component aimed to improve the profitability of farm production and enhance livelihoods of the poor. This would be achieved through better input-output market linkages, more efficient and effective delivery of key support services and augmentation of community level capacities as well as provision of some productive infrastructure. The activities to be financed include: (a) mobilization and capacity building of community-based institutions like Self Help Groups (SHGs) and producer groups (PGs); (b) support to cluster level producer groups for productive assets; (c) investment support for productive assets (d) improving rural market infrastructure; and (e) organization of innovation forums. The expected results would be: (i) increased share of produce marketed (by project area farmers); (ii) access to non-traditional markets; and (iii) mobilization of landless families to form SHGs for livelihood activities. 5. Project Management (appraisal estimate: US$22.9 million, actual cost: US$25.08 million). This component aimed to ensure smooth implementation of all project activities, monitoring of project implementation progress and outputs/outcomes, and learning from project experience. This would be achieved through ensuring fully functional project units at the state and district levels, providing technical assistance and training to improve implementation, ensuring adequate monitoring and evaluation as well as documentation of project experience and its dissemination to the wider development community, and liaising with project partner organizations, support organizations and external professional agencies, Activities to be financed include: (a) establishing and supporting project units at the state and district levels; (b) specialized support services relating to key activities such as independent external M&E’, external audit, financial accounting, procurement and social audit; (c) technical and other training of staff involved in project implementation; and (d) learning from and dissemination of project experience. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total cost of the project was estimated at appraisal to be US$272 million including US$2.7 million for taxes and duties. The actual cost reported by the ICR Annex 3 was US$246.23 million which was about 91.75% of the appraisal estimate. The contingencies cost was US$20.1 million which was not added to the total cost. Financing. The project was financed through an IDA Specific Investment Loan (SIL) worth US$197 million equivalent with a maturity of 35 years including a grace period of 10 years. The actual amount disbursed according to the ICR (p. 2) was US$174.81 million. In a further communication the project team explained Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) that "at the time of the ICR submission, the disbursement grace period was extended due to election related delays.The undisbursed XDR 9,565,455.91 had been canceled. The remaining difference between the appraisal and disbursement values in US$ is due to exchange rate losses and gains. The ICR template needs to be updated to capture the final disbursement amount. As the final disbursement was unknown, these clarifications were not provided in ICR." Borrower Contribution. The Borrower and the beneficiaries were expected to contribute US$49.2 million and US$25.8 million, respectively. The actual amounts contributed were US$49.2 million and US$27.69 million for the Borrower and the beneficiaries, respectively. Dates. The project was approved on June 30, 2009 and became effective two and half months later on September 18, 2009. The Mid-Term Review was carried out on April 29, 2013, seven months later than the date set at appraisal on September 30, 2012. The project closed on December 29, 2018, which was three years later than the expected closing date on December 31, 2015. According to the ICR (p. 10, para 23), the extension of the closing date by 36 months was needed to accommodate the limited availability of soil amendments required to meet annual targets on reclaimed sodic lands, to address quality issues that had arisen in the on-farm development elements of "the sodic land reclamation process" , and "to scale up ravine reclamation, provide technical support to newly established Farmers' Producer Organizations (FPOs), and consolidate achievements of the three phases by revisiting and improving rehabilitated drainage systems and re-treating earlier reclaimed patches where, for a number of reasons, sodicity had recurred." The project was restructured twice, both Level 2 Restructurings. The first was on August 28, 2014, when the amount disbursed was US$88.24 million, in order to extend the project closing date by 24 months from December 31, 2015 to December 29, 2017. The second was on August 3, 2017, when the amount disbursed was US$147.24 million, in order to extend the project closing date by 12 months from December 29, 2017 to December 29, 2018. 3. Relevance of Objectives Rationale Uttar Pradesh (UP) is the most populous state in India with a population of about 199 million (2011 census). Nearly 80% of the population in UP lives in the rural areas, and about two-thirds (66%) of them are dependent on agriculture for their livelihoods. Over the years, the share of agriculture in the state domestic product (SDP) has declined from about 33% in 1999-2000 to 26% in 2007-08. The performance of agriculture in UP is strained by the following issues: low land productivity, land degradation, small land holdings, weak dissemination of agricultural technology, and weak systems for delivering agricultural credit. The project aimed to addresses these core challenges through sustainably reclaiming 130,000 ha of Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) predominantly barren and low productivity sodic lands, which would improve household food security through increased productivity and cropping intensity. At project appraisal, objectives were in line with the Government of Uttar Pradesh vision to accelerate agricultural growth. Objectives were also in line with the central vision of India’s Eleventh Five Year Plan that aimed to promote inclusive growth so that the benefits were shared by all people. Objectives were also in line with the Bank's Country Assistance Strategy (CAS FY09-12) for India that focused on providing support for programs that: address rising inequality, ensure sustainable development, and enhance access to services by the poor. More specifically, objectives were in line with the CAS’ pillar of “ensuring sustainable development” under which the Bank sought to support India to reduce the burden of land degradation on the population. At project completion, objectives were in line with the Government of India priority to double farmers’ incomes by 2022, partly through enhancing agricultural productivity. Objectives were also in line with the Bank’s 2018-2022 Country Partnership Framework (CPF) for India. The CPF, among other things, emphasized that for India to sustain its growth trajectory and spread more equitably the gains of economic growth, it needs to pivot to a more resource efficient, inclusive, and diversified growth in the rural sector. This includes increasing productivity while improving efficiency in the use of land resources in agriculture. The statement of objectives was clear and focused. However, it lacked a connection to higher level objectives, namely, reducing inequality and poverty. The connection to higher level objectives was made clear in the ICR (paras 5 and 6, page 7), in stating that improving agricultural productivity on degraded lands was expected to raise incomes of the poor, contribute to addressing income inequality and foster inclusive growth (para 5) and that increased crop productivity would lead to increased rural income thus contributing to narrowing the income inequality in the long-term. This was not, however, reflected in the PDO formulation of the project. Based on the above-mentioned information, Relevance of Objectives is rated Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Objective PDO: to increase agricultural productivity of degraded lands in selected areas of Uttar Pradesh. Rationale Theory of Change. The theory of change is the relationship between activities, outputs from those activities and the final outcome (increase agricultural productivity). In this case, change is facilitated by a number of critical elements including: (i) timely availability and supply of key inputs (e.g. gypsum and press mud) necessary for effective land reclamation of sodic lands; (ii) interest and willingness of farmers to adopt new technology; and (iii) willingness of farmers to organize for cooperative action, both in production and output marketing. The project aimed to increase agricultural productivity on degraded lands through addressing three key issues: land degradation, low productivity and low profitability. Activities. The PDO was expected to be accomplished through: (i) Land treatment to reclaim degraded lands. The project would amend the sodic soils with adequate treatments, rehabilitate and maintain drains to improve drainage and reduce water logging, support forming water user groups (WUGs) and provide irrigation through tube wells. These activities were expected to increase the area of reclaimed lands and improve soil quality and reverse degradation. (ii) Provision of agriculture support services. The project would provide farmer training and demonstration of improved technologies, and fund exposure visits. Training also would be provided to the Government staff. These activities were assumed to result in farmers adopting improved crop management technologies. (iii) Strengthening and building capacities of institutions for market access. The project would support organizing farmers collectives and self help groups (SHGs), and improve market infrastructure. These activities were expected to improve access to markets and provide alternative livelihoods for landless families. While most of the stated activities were logical and had clear causal links to the PDO, the activities supporting markets were not directly linked to increasing productivity. That said, experience from similar projects have showed that improving market access had a positive effect on the sustainability of outcomes. Projects that sought to improve productivity and ignored marketing could face declines n crop prices due to excess production and limited marketing opportunities. Outcomes. The above-mentioned activities would result in increased agricultural productivity on degraded lands. This increased productivity would benefit from improved market access. Long term impacts. Increased crop productivity and improved market access were expected to lead to increased rural income. Thus, contributing to narrowing the income inequality in the long-term. Outputs The information reported below is from the ICR (Annex 1) unless referenced otherwise.  On-Farm Development and Land Treatment. Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) o 142,677 ha of sodic lands were reclaimed against a target of 130,000 ha (109% achievement rate). o 23,943 ha of ravine land were reclaimed against the target of 18,300 ha (target exceeded) in 158 watersheds (ICR, p. 12, para 27). o 1,248 ha were reclaimed through press mud treatments against a target of 6,000 ha (20.8% achievement rate, target not achieved). o Soil quality: pH level of 7.70 against a target of 8.40 (exceeded), EC level reached 0.3 ds/m against a target of less than 4 ds/m (exceeded), organic carbon in reclaimed lands reached 0.31% against a target of 0.30% (exceeded).  Improvement of Irrigation and Drainage System, and market infrastructure. o 3,235 km of main drains were rehabilitated, against the target of 5,700 km (target not achieved) reflecting lower actual needs than projected (ICR, p. 12, para 27). o 29,436 km of main drains were maintained (no targets provided). o 69,496 ha were under assured irrigation against a target of 98,000 ha (70% achievement rate, target not achieved). The ICR (p. 29) claimed that this reflected lower actual needs that originally projected. o 34,883 Water User Groups were formed (target: 33,000) and trained in maintenance of field drains (target exceeded, ICR, p. 12, para 27). o 102 rural markets known as rural haats (RH) received improvements in infrastructure against a target of 125 (target not achieved, ICR, p. 13, para 29).  Agriculture Support Service. o 19,996 demonstrations (148% of the target) were carried out on Integrated Plant Nutrient Management (IPNM), 7,537 on Nadep compost (80% of the target), 7,625 on vermi-compost (90% of the target), 1,656 on Integrated Pest Management (IPM, 92% of the target), 13,888 on crop diversification (163% of the target), and 137 on Sodic Kisan Mela (137% of the target - ICR, p. 21, para 28). o IPM practices were adopted on a total of 5,284 ha against the target of 4,000 ha (target exceeded). o 8.8% of sodic GCA were under oil seeds and pulses against a target of 7.5% ( target exceeded). o 2.9% of sodic GCA were under vegetables against a target of 2.5% (target exceeded). o Average milk productivity increased to 8.55 liters per animal per year for buffalos against an end of project target of 7 liters per animal per year and 6.33 liters per animal per year for cows against an end of project target of 5 liters per animal per year, both targets exceeded (ICR, p. 12, para 28). Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033)  Institutional Strengthening and capacity building o 120 farmers producer groups (FPOs) were formed against a target of 100 (120% achievement rate). o 82 FPOs were qualified to receive project resources as planned under the Community Investment Fund (CIF) for working capital and basic infrastructure o 20 FPOs secured statutory licenses enabling them to operate in regulated wholesale markets for agricultural produce (no target provided). o 69 FPOs registered with e-NAM, a pan-India online trading platform for agricultural commodities, to enable them access markets in different states. o 57 FPOs were linked to the Small Farmers’ Agribusiness Consortium (SFAC) from where they accessed grant and credit financing for their business needs (ICR, pages 12 & 13, para 29). o 15 FPOs entered into binding supply contracts with several flour mills and had delivered over 46,000 tons of paddy and wheat before project closing (no target provided). o 73,257 women were organized in 6,555 Self Help Groups (SHGs) against a target of 5,500 (119% achievement rate). Women were trained in income generating skills and supported by the project through the Livelihood Support Fund (LSF) grants to engage in multiple livelihood activities including, but not limited to beekeeping, goat rearing, jewelry making, tailoring, and packaging (ICR, p. 13, para 30). Outcome The project aimed to increase agricultural productivity on degraded lands. To achieve this objective, the project featured a combination of reclamation and improved provision of agriculture support services as explained in the theory of change above. As a result of these activities, pH and Electrical Conductivity (EC) levels (two critical parameters for suitability of land to grow crops) for reclaimed lands dropped to within ranges appropriate for crop production (ICR, p. 12, para 27). Also, the rehabilitation and maintenance of the main drains reduced waterlogging in the project areas by 95% which, further improved soil quality for crop production purposes. The achievement of the project objective was assessed by four PDO indicators. As a result of the project- supported activities, the annual cereal (wheat and paddy) productivity increased from 4.09 t/ha to 6.02 t/ha, which represented a 94% achievement of the target of 6.4 t/ha, target almost achieved. Also, cropping intensity in project areas increased from 45% to 211% which exceeded the target of 200%. Crop income per unit area of land also increased by 76% exceeding the target of 71%. Finally, the market value of reclaimed C class lands increased from Rs. 200,000 to Rs. 648,279 per ha, an almost 99% achievement of the target. While it is plausible to assume that reclaimed lands could potentially appreciate in price, it is worth mentioning that the price of land could appreciate due to other factors beyond the project control, such as urbanization, for example. Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) The project also strengthened FPOs, which had a positive impact on marketing of crops and reduced input costs by 10% (p. 13, para 29). According to the ICR (p. 13, para 29), 37% of paddy and 32% of wheat producing farmers supported by the project sold their produce through the FPOs. At project closing, cumulative FPO business transactions were estimated at Rs.1.2 billion, of which 60% were accrued from agricultural produce marketing, 38% from agriculture inputs trade, and the rest from provision of agro- services. However, the ICR did not provide earlier figures to better assess the impact of the project on FPOs. Based on the aforementioned information, it is evident that the project fully achieved its targets for two of the four outcome indicators. The achievement for the other two indicators (“increase in productivity of wheat and paddy” and “increase in C class land value”) were substantially achieved, with 94% and 99% achievement rates, respectively. The ICR (p.12, para 27) also reported that a sustainability assessment conducted at project closing, found that 88% of reclaimed sodic lands were still being used for agricultural purposes. This compared favorably well with levels observed at the completion stage in other land reclamation projects funded by the Bank, for example, 78% in the Heilongjiang Land Reclamation Project in China; 82% in the Land Husbandry and Water Harvesting and Hillside Irrigation Project in Rwanda; and the 85% percent in the Loess Plateau II Project in China. The ICR (p. 12, para 27) noted that the agricultural usage was 93% in the second phase of the project. It attributed the lower rate in the third phase to "absentee landlords" who owned most of the unutilized lots. The evidence provided point to the success of the project in increasing agricultural productivity in selected areas of Uttar Pradesh. Therefore, Efficacy of the achievement of this outcome is rated Substantial, despite that the project fell short on meeting some output/intermediate outcome targets. However, there are concerns on the quality of M&E data and the absence of counterfactuals (see section 9 for more details). Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall efficacy is rated Substantial. The evidence provided point to the success of the project in increasing agricultural productivity in selected areas of Uttar Pradesh. However, there are concerns on the quality of M&E data and the absence of counterfactuals (see section 9 for more details). Overall Efficacy Rating Substantial 5. Efficiency Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Economic and Financial Efficiency ex ante  At project appraisal, an economic and financial analysis (EFA) estimated the economic rate of return (ERR) on the project investments at 25.1%, and an economic net present value (ENPV) of Rs.9.5 billion. These returns were assessed for a 25 year period (including five years of implementation), at 2009 prices and a 12% discount rate.  Estimated benefits were expected to derive from improved crop productivity, intensification and diversification on reclaimed sodic lands and non-sodic lands benefitting from improved drainage, agriculture support services and market access, and improved livestock productivity resulting from improved extension and animal breeding services.  However, the appraisal returns were based on an optimistic implementation timeframe that did not materialize, and correspondingly overestimated benefit accumulation timelines and incremental returns. ex post  The ex post EFA ERR was etimated at 19.7% and the ENPV was Rs.4.0 billion. The project returns were re-estimated following the appraisal methodology except that actual and revised future costs and benefits were used. The returns were assessed for 29 years, including 9 years of implementation, at 2009 prices and a 12% discount rate.  The ICR (page 13 & 14, para 32) attributed the reduction in the ex post ERR compared to appraisal (19.7% compared to 25.1%) to the overestimation of benefits at appraisal based on unrealistic assumption of the project timeframe; delayed benefit realization by three years; lower than expected areas of both Class C land and utilization of reclaimed lands.  The analysis estimated benefits from: sodic land reclamation, ravine reclamation, improving drainage systems, crop diversification, livestock benefits and rural market development.  The ex post EFA could have benefited from the inclusion of a sensitivity analysis to reflect the impact of different scenarios that the project could potentially experience on the estimated ERR. Administrative and Institutional Efficiency Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) The project closed on December 29, 2018, which was three years later than the expected closing date on December 31, 2015. Implementation delays reduced the overall project efficiency because realization of benefits took longer than expected. Implementation delays resulted from not knowing the eligibility and selection criteria for the sodic lands, and the lack of details on the implementation arrangements for the market access activities (ICR p. 16, para 45). Also, the availability of soil amendments in the required quantities to meet annual reclamation targets was a challenge. Overall, Efficiency is rated Substantial because "the cost per ha of reclaimed land remained within the projected ranges"(ICR, p. 37, para 14) despite implementation delays and a lower ex-post ERR. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  25.10  Not Applicable 100.00 ICR Estimate  19.70  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives is rated Substantial. Overall efficacy is rated Substantial. The evidence provided point to the success of the project in increasing agricultural productivity in selected areas of Uttar Pradesh, and the project fully achieved two of the PDO indicators and substantially achieved the other two. However, there are concerns on the quality of M&E data and the absence of counterfactuals makes it difficult to capture general trends that are not attributable to the project (see section 9 for more details). Efficiency is rated Substantial because "the cost per ha of reclaimed land remained within the projected ranges" (ICR, p. 37, para 14) despite implementation delays and a lower ex-post ERR. Based on a Substantial rating for all three elements (relevance of objectives, efficacy and efficiency) of the outcome assessment, Outcome is rated Satisfactory. a. Outcome Rating Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Satisfactory 7. Risk to Development Outcome The project aimed to increase agricultural productivity on degraded sodic lands. The ICR (p. 21, para 63) discussed several issues that could potentially impact the sustainability of productivity gains achieved under the project :  Adequate drainage of the reclaimed sodic land. The deterioration of the drainage network will result in the return of waterlogging conditions-a key driver of degradation. The ICR explained that the irrigation department is responsible for maintenance of main drains, and has adequate resources through the state budget to perform this job. The maintenance of the field drains falls under the responsibility of the Water User Groups (WUG), which generally have enough funds and technical competence to maintain. However, WUGs would need to be regularly supervised and guided by the irrigation department.  Continued cultivation of the degraded land following a well-defined cropping cycle. Reclaimed degraded lands need to follow a crop cycle that ensure leaching of salts. The failure of following such a cropping pattern could result in salt accumulation and deterioration of productivity on these lands. Land scarcity in Uttar Pradesh combined with continued government support should ensure that the majority of reclaimed lands continue to be cultivated (ICR, p. 21, para 63).  Continued use of good land and water management practices by farmers. Reclaimed lands require the availability of adequate amounts of irrigation water to support crop growth and meet salt leaching requirements. Shortage of irrigation water will result in quick degradation of the reclaimed lands.  Continued effective functioning of producer collectives. Farmers' collectives played a positive role in ensuring market access and in reducing input costs. This enabled farmers to successfully cultivate reclaimed lands and ensure marketing of their produce. If farmers' collectives become dysfunctional, this could result in higher input costs and limited marketing opportunities, which would strain farming in these reclaimed areas. Linking farmers' collectives to the Small Farmers Agribusiness Consortium (SFAC) would ensure continued support and strengthening for the purposes of establishing forward and backward linkages in their preferred value chains. However, continued technical support to these FPOs is required, especially to those 38 FPOs that did not qualify for the CIF support (ICR, p. 21, para 63). 8. Assessment of Bank Performance a. Quality-at-Entry This project was the third in a series of Bank projects that addressed sodic lands in Uttar Pradesh. The first was the Sodic Lands Reclamation Project (1993-2001), which was a pilot, followed by the Uttar Pradesh Sodic Lands Reclamation II Project (1999-2007). The project objectives were in line with the Government of Uttar Pradesh's priorities, India's eleventh five year plan and the Bank's Country Assistance Strategy (CAS FY09-12) for India. The project design benefited from the experience of the Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) first and second phases in land treatment, and added new interventions such as crop diversification and improved marketing to promote sustainability of project outcomes. Notable lessons reflected in the design form the previous phases include: more attention to marketing of agricultural produce and to improving soil fertility/quality, a flexible, needs-based provision of agriculture support services to communities in newly reclaimed sodic lands, and organize campaigns for community mobilization and preparatory activities to ensure successful sodic land reclamation. The design featured mutually reinforcing components through supporting land reclamation and agriculture services to unlock agricultural production on degraded lands, while providing marketing support to facilitate marketing produce that accrue from increased production (ICR, p. 15, para 43). The project design suffered from a number of shortcomings. First, the project implementation time-frame was unrealistic since implementation hinged on the availability of the required amounts of gypsum. However, limited availability and supply of gypsum to meet the technically recommended application rates for reclamation caused implementation delays. Second, the selection criteria for sodic lands to be reclaimed was restrictive and had to be relaxed in 2015 after it was realized that the project would fail to meet its targets. Third, the implementation arrangements for the activities supporting market access were only detailed out during implementation (ICR, p. 16, para 45). Fourth, implementation was also negatively impacted by weak capacity at the implementing agency and high staff turnover. Finally, the design should have better assessed the cost and social acceptance of the press mud treatment as targets were cut due to high costs and environmental concerns raised by farmers (ICR, p. 10, para 23). Nine risks were identified at the appraisal stage. According to the ICR (p. 20, para 60) "implementation risks were adequately identified and appropriate mitigation measures proposed." However, staff hiring issues and turnover was not anticipated, and resulted in implementation delays. M&E suffered from some design shortcomings and implementation weaknesses (see section 9 for more details). Overall, Quality at Entry is rated Moderately Satisfactory due to design shortcomings, overlooking the risk of implementation capacity and local staff hiring issues, and M&E weaknesses. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p. 21, para 61) the project benefitted from regular and timely supervision missions that were adequately staffed. Through most of the project’s implementation, supervision missions provided candid reporting and adequate recommendations. The Bank also worked with government after it requested the extension of the project closing date to "address quality issues, satisfactory completion of activities, and scale-up implementation (p. 21, para 61)." Market access activities and M&E suffered from limited experience and capacity of the Uttar Pradesh Land Development Corporation and line departments. These activities could have benefitted from more attention by the Bank. The ICR (p. 21, para 61) highlighted that "that Bank support in these areas is also recognized as weak across most of India’s agricultural portfolio and might merit a broader response." Overall the Quality of supervision is rated Moderately Satisfactory. Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Based on a Moderately Satisfactory rating for both Quality at Entry and Supervision, Bank performance is rated Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The PAD did include a theory of change as it was not a requirement at the time of appraisal. Nonetheless, the ICR (p. 7) included a theory of change that reflected the relationship between the project inputs, outputs, expected outcomes and long term impacts. The overall responsibility for Monitoring and Evaluation (M&E) of project implementation and progress remained with the Uttar Pradesh Land Development Corporation, which was to establish an M&E Unit. M&E would follow a three pronged approach where internal monitoring would be carried out by each line department and agency involved in project implementation; participatory monitoring by beneficiaries and beneficiary organizations, including WUGs and other groups supported by the project; and external M&E would be conducted by an independent agency hired by UP Land Development Corporation. The PDO was to be assessed through four project outcome indicators: increase in the productivity of wheat and rice (tons per hectare), increase in cropping intensity; percent increase in crop incomes per unit land , and percent increase in land value. The first three indicators were directly linked to the PDO and measurable. However, the fourth indicator (increase in land value-percent) was not the best choice and its relation to the PDO was not clear. The ICR (p. 18, para 52) reported that the fourth indicator was hard to measure as it required monitoring of other factors beyond the control of the project. Also, the ICR (p. 18, para 52) correctly reported the absence of PDO indicators that would assess success or lack of there of market access activities supported by the project. The Results Framework included fifteen intermediate outcome indicators to assess the different activities supported by the project. Most of these were relevant and measurable and had a baseline. Overall M&E design was good with specific assessments planned to measure progress and impact at mid- point and end-of- project. However, a notable weakness was the lack of a PDO indicator to assess market access. Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) b. M&E Implementation According to the ICR (p. 18, para 53) reporting and monitoring on all indicators was systematic, and the mid-term and end-of-project surveys were conducted in a timely manner. However, M&E suffered from a number of weaknesses: M&E capacity weaknesses contributed to a "consistent disconnect between progress data and the actual situation on the ground (ICR, p. 16, para 47)," comparing project outcomes with the counterfactual to capture general trends that are not attributable to the project was not possible due to the lack of appropriate controls, the surveys designed to assess achievement of the outcome indicators lacked accuracy in capturing the project’s attribution and impacts of external factors (ICR, p. 18, para 53), the M&E system failed to capture incremental price realization and marketing transactions, which eventually led to inadequate monitoring of the improvement in market access, finally, there were key concerns relating to the quality and reliability of some of the progress data during implementation, which undermined the integrity of the monitoring system (ICR, p. 18, para 53). In a further communication, the project team explained that "in addition to the project M&E, the third party M&E firm was tasked with the progress, disconnect between progress reports and actual situation in the field was frequent and this was consistently raised by the supervision missions. As noted in various sections in the ICR, the M&E was also unable to capture quality issues in On-Farm Development works as well as in sodic land selection. Moreover, the M&E capacity was weak to adequately measure results and outcome leading to erroneous (inflated) estimates of results which are reflected in the Government’s Project Completion Report. The Annex 6 to the ICR was prepared to provide clarification on differences between the project and ICR estimates. These issues were behind the concerns over the reliability and quality of M&E." c. M&E Utilization According to the ICR (p. 18, para 54) a number of decisions and mid-stream course-corrections relied on the project progress reports, for example: relaxing eligibility and selection criteria for sodic patches to be taken up for reclamation and reducing annual targets for land reclamation in light of reduced supplies of necessary soil amendments. However, in other cases corrective measures were not taken despite that field findings by the Bank supervision missions provided justification for undertaking such remedial measures. For example, corrective actions were not taken despite the overestimation of the proportion of C class lands in some districts, and sustainability assessment findings, which suggested that not all reclaimed sodic lands were being utilized. Overall, M&E Quality is rated Substantial. There were moderate design weaknesses relating to the lack of better indicators to assess progress with respect to improving market access and implementation could have been more effectively handled, but the design was reasonably well informed and the PDO indicators provided a reasonable overview of project performance. M&E Quality Rating Substantial Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) 10. Other Issues a. Safeguards The project was an environmental category B. It triggered the following safeguard policies: Environmental Assessment (OP/BP/GP 4.01), Natural Habitats (OP/BP 4.04), Pest Management (OP 4.09), and Involuntary Resettlement (OP/BP 4.12). According to the PAD (p. 24, para 77) "few adverse environmental impacts are foreseen, since the project’s principal objective relates to improving land-based productivity by reversing water-induced land degradation." Environmental Safeguards. A comprehensive Integrated Environmental and Social Assessment (IESA) was prepared by the government. The IESA identified social and environmental impacts of the project interventions; established the baseline social and environmental information; and developed an Environmental and Social Management Framework (ESMF) for systematic management of safeguards. Also, a specific Wetland Management Plan was implemented to prevent draining of natural wetlands (ICR, p. 19, para 56). Throughout implementation, an Environmental Manager oversaw monitoring and reporting on safeguard compliance According to the ICR (p. 19, para 56) "environmental safeguards compliance was satisfactory overall, and no significant adverse impacts were observed." Social safeguards. The ESMF included a resettlement and rehabilitation policy and an entitlement framework, a gender development strategy and strategies to address issues related to vulnerable groups. Rural Haats infrastructure didn’t involve land acquisition as it was built on land owned by the village coiuncils and field drains were constructed in beneficiaries’ own fields. According to the ICR (p. 19, para 57) "there were no reports of adverse social impacts as a result of project activities." However, for most of the implementation period there was no social development specialist recruited by the project. b. Fiduciary Compliance Financial Management. A Finance Controller (deputed official from the State Financial Services) at the UP Land Development Corporation provided regular financial oversight. Internal and external audit mechanisms were effective and project audits were timely prepared and submitted. The accounting staff benefited from regular training which improved accounting and internal control. However, budget allocation and fund flows were delayed in a number of years which resulted in implementation delays (ICR, p. 19, para 58). Also, there was some auditing issues including: delays in settlement of advances; liabilities appearing in the books pending settlement; and poor accounting, bookkeeping and inventory management at District Project Units, Water User Groups, Women Self-Help Groups and Village Councils. The ICR (p. 19, para 58) reported that: “the Sodic Project Audit Committee meetings reviewed the audit observations to ensure appropriate resolution of issues. NGOs were tasked to support the user groups. The project accounting staff were regularly trained leading to improved accounting and internal control. The statutory audit of UPBSN Limited (Entity Audit) that was supposed to be undertaken for each financial year as agreed to was significantly delayed (due to delay in the appointment of entity auditor) and had not been completed by MTR. Currently, UPBSN is working on closing outstanding issues and submission of Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) final disbursement claims by the grace period deadline on June 29, 2019.21 UPBSN has agreed to submit the last year (FY 2018-19) project audit report to the Bank by December 31, 2019.” Procurement. Procurement of gypsum experienced challenges that stemmed from the limited number of bidders due to limited number of suppliers and high transportation costs (ICR, p. 19, para 59). Also, procurement of fertilizers and seeds was constrained as these are regulated items in India. Towards the end of the project, the Irrigation Department introduced e-procurement service for the procurement of civil works. The Uttar Pradesh Land Development Corporation procurement team reviewed the procurement handled by project affiliated agencies to minimize procedural violations. The extension of the closing date required the extension of many contracts which according to the ICR (p. 20, para 59) was handled well by the project. According to the ICR (p. 20, para 59) "there were no incidents of malfeasance reported or discovered in the course of the project’s implementation." c. Unintended impacts (Positive or Negative) According to the ICR (p.15, para 40): "No unintended outcomes and impacts were documented." d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Design was adequate and the PDO data provided a reasonable Quality of M&E Modest Substantial overview of the project's performance. Quality of ICR --- Substantial 12. Lessons The ICR included three lessons. These are emphasized below with some adaptation of language:  Functional farmer collectives are needed for improving market access. Formation of fully functional FPOs is a long and arduous process that is hard to accomplish within typical project timeframes. Even with the hiring of specialist service providers, to help the Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) project with the formation and strengthening of FPOs, it was only in year 5 of implementation that the initial set of FPOs were deemed ready to undertake their activities and by project closing, many FPOs, (around 38 FPOs were still considered weak and in need of strengthening if they were to serve their roles.  Identification and selection of project targets is critical to ensure timely implementation. The largest share of project investments was meant to reclaim sodic land patches identified and selected based on specific technical selection criteria. During implementation, it became evident that patches in the target districts, meeting the criteria were limited. An assessment of the available patches during preparation would have led to more pragmatic identification and selection criteria.  Sustainability of outcomes needs careful considerations of Operation and Maintenance arrangements. Infrastructure development in many countries has often followed the build-neglect-rebuild paradigm mainly because of poor asset maintenance. Project design anticipated challenges with OM and made provisions for maintenance of the rehabilitated drains, with clear responsibilities for both government (ID) and beneficiaries. These maintenance arrangements were triggered upon completion of rehabilitation of any drains and by project closing they were still functional and should help sustain project outcomes. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain This is the third phase of land reclamation projects in Uttar Pradesh. Further assessment of the outcome and impact on productivity and inequality in the State is warranted given the concerns surrounding the accuracy of the M&E data generated by the project. Also, a field assessment could generate useful lessons that would guide similar future interventions. 14. Comments on Quality of ICR Quality of evidence. In several instances there were minor discrepancies in the reported numbers under the outcomes section and these reported under Annex 1. Quality of Analysis. The ICR provided clear linking between evidence and findings. Extent to which lessons are based on evidence and analysis. Lessons were based on the project experience; however, they were formulated as recommendations. Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: UP Sodic III (P112033) Results Orientation. The ICR included a good discussion on outcomes despite concerns on the accuracy of the M&E data. Internal Consistency. Various parts of the ICR were logically linked and integrated. Consistency with guidelines. The assigned ratings in the ICR were justified. Conciseness. The ICR provided thorough coverage of the implementation experience and candidly reported on shortcomings. There was sufficient clarity in the report’s messaging and the performance story was direct, well informed and tightly presented. Overall, the ICR Quality is rated Substantial, despite some minor shortcomings. a. Quality of ICR Rating Substantial Page 19 of 19