Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) Report Number : ICRR0021491 1. Project Data Project ID Project Name P107617 WUAP Country Practice Area(Lead) Additional Financing Azerbaijan Water P147947 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-80390,IDA-49130 30-Jun-2016 75,864,343.22 Bank Approval Date Closing Date (Actual) 26-Apr-2011 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 80,000,000.00 0.00 Revised Commitment 80,000,000.00 0.00 Actual 75,864,343.22 0.00 Prepared by Reviewed by ICR Review Coordinator Group Cynthia Nunez-Ollero J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components a. Objectives Both the Financing Agreement (IDA FA, p.5) and the Loan Agreement (IBRD LA, p.6), and the Project Appraisal Document (PAD, paragraph 13) defined the Project Development Objective (PDO) for the Water Users Association Development Project (WUAP) as "to improve the effectiveness and financial viability of on- farm irrigation water distribution and management in the project area." ‘On-farm irrigation water distribution system’ included canal networks, which were formerly managed by kolkhoz/sovkhoz, each with their own brigade for canal operations and management (O&M). This review will assess the extent to which the PDO has been achieved by examining the outcomes of its two parts: Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) • Objective 1 - Improve the effectiveness of on-farm irrigation water distribution and management in the project area • Objective 2 - Improve the financial viability of on-farm irrigation water distribution and management in the project area b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components 1: Institutional Strengthening and Capacity Building (US$16.7 million at appraisal, US$15.9 million actual). This component financed capacity building of AIOJSC, training and capacity building of WUAs, provision of MOM equipment and management and offices for WUAs, strengthening the AIOJSC’s capacity for WUA supervision, and institutional capacity-building studies and pilots to improve the effectiveness of administration and on-farm water management by WUAs in about 27 raions. 2: On-farm irrigation and drainage (I&D) Rehabilitation (US$89.9 million at appraisal, US$85.9 million actual). This component financed the rehabilitation of on-farm I&D infrastructure of eligible WUAs in selected districts or raions. 3: Project Management and M&E (US$7.7 million at appraisal, US$5.8 million actual). This component financed project management, including monitoring and evaluation (M&E). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost reached US$114.3 million and disbursed a total of US$107.8 million leaving a balance of US$6.5 million, US$4.1 million of which was from undisbursed IDA resources and US$2.5million from borrower contributions. Financing: The IBRD provided a loan in the amount of US$3.2 million. The International Development Association provided SDR48.90 million or the equivalent of US$76.8 million for a total of US$80 million from the World Bank Group. Of this total, US$75.9 million was disbursed. Borrower contribution: The Borrower committed US$34.3 million to the project and disbursed US$31.9 million. The balance of US$2.5 million was undisbursed because gains from the SDR exchange rate as well as cost savings covered the cost of the expanded scope and corresponding cost (per clarification provided by the Task Team in a March 5, 2019 email). Dates: The project was approved on April 26, 2011 and became effective on December 21, 2011. A Mid Term Review was conducted on July 31, 2014. The original closing date was June 30, 2016. There were two Level 2 Restructurings to extend the closing date: Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) • On June 27, 2016, to extend the closing date to June 30, 2017. There were two reasons for this extension. First, in 2015, after the European Games were held in Baku, there was a delay in the release of the Government’s contribution for civil works that resulted in three schemes to be completed beyond the original closing date. Second, the Azerbaijan manat devaluation resulted in a cost saving of US$7 million that allowed the expansion of the project to rehabilitate additional irrigated areas. • On June 16, 2017 to extend the closing date to June 30, 2018 as requested by the Government to complete the remaining contracted additional rehabilitation works. 3. Relevance of Objectives Rationale This review concurs with the ICR that the PDO was fully aligned with the Bank's Country Partnership Framework (CPF) for FY16–FY20 (ICR, paragraph 9) which focused on ‘supporting enhanced competitiveness of agriculture and rural development companies’. Focus Areas 1 and 2 (Economic competitiveness) promoted investments in infrastructure for better access to public services and infrastructure for growth. The CPF places special focus on rural development (Focus Area 2) through investments in quality infrastructure that would enhance the competitiveness of the rural economy. The CPF also focused on improved human development outcomes and increased prosperity through better access to water (irrigation) and improved quality of the environment. Access to improved irrigation services was critical for increased agricultural productivity and efficiency in water use. This would positively impact social and economic opportunities in rural areas where the majority of the poorest 40 percent of the population reside. The project supported the Government's commitment in this sector. The PDO was also fully consistent with the Government’s third State Program of Social and Economic Development of Regions for 2014–2018, which aimed to reduce regional inequalities and provide households in all raion centers with reliable basic services, such as water and sanitation and other communal services. The project objectives were in line with Government priorities and its state program for poverty reduction and sustainable development. The PDO was deemed highly relevant to contributing to increasing agricultural productivity and boost non-oil growth. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve the effectiveness of on-farm irrigation water distribution and management Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) Rationale The theory of change for achieving improved effectiveness of on-farm irrigation water distribution and management was through (a) rehabilitation of irrigation and drainage systems thereby improving the supply of irrigation water, and (b) institutional strengthening and capacity building to improve management skills (ICR, Figure 1). The effectiveness of on-farm irrigation distribution and management was measured by comparing the actual irrigation services delivery time with the farmers' requests for irrigation water delivery at the beginning of the season and the subsequent expected increase in productivity of 15% in 8 of 10 cases of rehabilitated WUAs vs. non rehabilitated WUAs. These are standard performance expectations for Bank-financed irrigation projects (ICR, paragraph 14). In addition, a satisfaction survey was conducted in October and November 2016 with 25 rehabilitated WUAs who have been through two irrigation seasons after rehabilitating its I&D systems. OUTPUTS: • Rehabilitated 2,487 km of irrigation canal (target achieved) where (i) 2,252 km of open canal were cleared of vegetation and sediments excavated and sections re-formed to restore design flows and (ii) 236 km of canals were lined to reduce seepage • Rehabilitated 915 km of drainage channel (target achieved) • Constructed new hydrotechnical works such as 7,412 structures (e.g., water measurement structure, offtake with check structure, bridge on canal and road crossing, etc) and rehabilitated 1,122 existing structures (e.g., offtake with pipe passage, bridge, etc.) (For details, see ICR annexes 7 and 9). • Completed 39 construction works contracts with a total cost of AZN 72,290,315 (or US$42.5 million) for the above works • 330,873 female water users were provided with improved irrigation and drainage services (baseline 200,000 original target 330,000, exceeded) • 430,016 male water users were provided with improved irrigation and drainage services (baseline 250,000, original target 430,000, exceeded) • 920,274 hectares improved their irrigation and drainage searches (baseline 530,000, original target 900,000, exceeded) • 340 WUAs registered with adequate number of trained and experienced staff 10 from the Regional Support Centers, 82 from the Raion WUA Support Unit, and 3 from the Central WUA Support Unit) implementing good operational and maintenance procedures (baseline 22, target 340, achieved) OUTCOMES: • 23 out of a sample of 25 WUAs achieved a 15% increase in agricultural output associated with corresponding increases in yield of agricultural products covering a total irrigated service area of 71,681 hectares (target in hectares 59,200, baseline 0, exceeded target). Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) • Based on the end of project survey, 92.9% of farmers, representing 23 of 25 rehabilitated WUAs received more than 80% of the irrigation water they requested at the start of the season (target 80%, exceeded). Note that the original target for the number of WUAs was 30 compared with the 23 actually achieved or 77%. The sampling survey, which indicated that farmers received more than 80% of the irrigation water they requested at the start of the season supported the claim that water distribution was within 80% of the rehabilitated systems. This measure matched the crops irrigation water norms for the season after rehabilitation (ICR, Annex 1, Indicator 3) (target almost achieved). • 379 WUAs were operational and strengthened after over 1,500 members of the WUAs received capacity building technical assistance (baseline 208, original target 379, achieved) • 118,510 hectares of on farm Irrigation and drainage infrastructure were returned to operational condition or rehabilitated (target of 85,000 hectares, exceeded) • Overall, 73.4% of WUA members were satisfied with the managerial and operational perforamnce of their WUAs. (baseline 20%, original target 60%, exceeded). The project therefore improved the effectiveness of on-farm irrigation water distribution and management. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve financial viability of on-farm irrigation water distribution and management Rationale The theory of change for achieving improved financial viability of on-farm irrigation water distribution and management was also through (a) rehabilitation of irrigation and drainage systems which improved the supply of irrigation water, and (b) institutional strengthening and capacity building to improve management skills (ICR, Figure 1). Financial viability of on-farm irrigation and water distribution and management was measured by comparing the Irrigation Service Fee collection rates of rehabilitated WUAs with the collection rates on non-rehabilitated WUAs and the recovery of costs that WUAs incurred for the management, operation, and maintenance of canal networks. OUTPUTS: • 27 of the 32 raions with rehabilitated WUAs registered 80% collection rates. This met 80% of the estimated requirements for on-farm system management, operation, and maintenance by WUAs after two irrigations seasons following rehabilitation (original target 20, exceeded) • 281 WUAs (target 379, almost achieved) achieved fee collection rates of at least 65% (baseline 22 WUAs with collection rates above 65%) Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) • 361 WUAs prepared basic asset management plans approved by the WUA Representative or General Assembly and contributed to setting the Irrigation Service Fee (baseline 22, original target 340, exceeded) • 200 WUAs had proper offices (baseline 22, target 222, almost achieved) • 379 WUAs had properly established Representative Zones with water users actively engaged in WUA management through these Zonal Representatives (baseline 22, target 340, exceeded) • 88 WUAs had business plans and were provided equipment under the project (baseline 0, target 90, almost achieved) OUTCOMES: • Rehabilitated WUAs achieved the 80% minimum collection rates, meeting the 80% estimated required level for on-farm management, operation and maintenance needs after two irrigation seasons following the completion of the irrigation and drainage rehabilitation. • Achieved at least 65% WUA fee collection rates, meeting the 55% required level for on-farm system management, operation, and maintenance in the year following the training and technical assistance. The project therefore improved the financial viability of the on-farm irrigation water distribution and management. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale The project directly benefitted around 379 WUAs in 27 raions with approximately 330,873 female and 430,016 male water users and farmers who managed and cultivated 920,274 hectares of irrigated land. There was only a minor shortcoming in achieving the second objective attributable to an overly ambitious target (281 WUAs vs. a target of 379 WUAs with 65% fee collection rates), which was not changed even after two restructurings. Overall the efficacy of the project’s achievements was substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic and Financial Efficiency: The ex-ante economic analysis was replicated at project closing in the ICR. Benefits attributed to building institutional capacity were not quantified but were included in the overall costs of the project. Benefits included the following: Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) • increase in crop yield for almost all major crops in the project WUAs; • increase in cropping area • change in cropping patterns to favor high value crops Other benefits that were not included in the Cost Benefit Analysis were: • survey results attributed institutional strengthening and capacity building of WUAs that received training but not rehabilitated its WUAs showed improvements in better organization and improved managerial skills • estimated incremental income generated by the project (US$400 a month per hectare) was 3-4 times the monthly pension as of January 1, 2018 of about US$122 a month). The elderly and pensioners relied mainly on their land for livelihood. No information, however, was available on how many elderly or pensioners were WUA members. These estimates were obtained from the initial and final surveys conducted in 2012 and at the end of 2016 after the bulk of the rehabilitation works were completed (ICR, Annex 4, paragraph 5). The EIRR at completion was 39% at completion compared to the EIRR of 31% at appraisal. An economic net present value of US$236 million at closing used a discount rate of 12% (not justified in the ICR, only noting that the range of 8-12% was a World Bank recommendation, see ICR, Annex 4, Table 4.9) compared to US$132 million at appraisal (no discount rate noted in the PAD). There was no financial analysis at appraisal. At closing, the FIRR was at 41% and a financial net present value of US$296 million, also using the unexplained 12% discount rate. Operational and Administrative Efficiency: The project was extended twice under two restructurings adding another 24 months to its original completion date. Both extensions resulted in cost savings, which increased the targets achieved and expanded the project area by 39%. The project's management and M&E costs accounted for a modest 5% of actual total project costs. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 78.60 Appraisal  31.00 Not Applicable 79.80 ICR Estimate  39.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) This review has rated the relevance of the project's objectives as high, and the efficacy with which both objectives were achieved was rated substantial. The project's efficiency was also rated substantial. The project's overall outcome was therefore rated satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The following pose moderate risks to development outcome: • Technical: The technical improvements introduced in the rehabilitation of irrigation and drainage may not be carried forward into those water users associations that have not participated in the rehabilitation works under the project. Standards established under the project may also find obstacles in new (as opposed to existing) infrastructure projects in the sector. This was mitigated by the documentation of construction works on which to base future interventions. In addition, the Law on Amelioration and Irrigation, which established water users association transformed these from limited liability companies to voluntary community associations responsible for management of on-farm irrigation systems. The 2004 amendment allowed the associations to have the right to set their own irrigation services fees to cover all costs of managing the water users association. • Financial: There may not be sufficient resources to carry out the operations and maintenance of completed projects. This risk was mitigated by the virtuous cycle established under the project by increasing management capacity and improving water supply delivery. The water users associations that need rehabilitation will be using either the Government budget or external support for their sustainability. There were no firm commitments indicated in the ICR. • Social and stakeholder ownership: The participatory nature of irrigation management may not be sustained post project completion. This risk was mitigated by the improved capacity of the water users associations and commitment to improvements introduced under the project. However, moderate risk to outcome of improved capacity development for non-rehabilitated WUAs remained since the benefits arising from the project were significantly lower compared to those 39 WUAs, which benefitted from infrastructure improvements. • Environmental: The impact from climate change threaten the development outcomes of this project. This risk was mitigated by the government commitment to implementing an ambitious water resources development program that added storage and several large irrigation canals. The project recommended that future design in sector interventions need to address these external factors arising from climate change. • Government commitment: The Amelioration and Irrigation Open Joint Stock Company (AIOJSC), the implementation agency, is aware of rehabilitation needs. The AIOJSC conducted a feasibility study and submitted it to the Cabinet of Ministers; however, its implementation has not been decided yet. The Government and World Bank started discussing Additional Financing (AF) for the project in 2013, because of significant unmet demand for rehabilitation and the capacity building needs of other water users associations Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) not addressed by this project or its predecessor project (IDSMIP). The AF was developed for Board approval before June 30, 2014. After a series of discussions, a feasibility study for the proposed AF was implemented based on the Ministry of Economy's request in 2017. However, the AF did not go ahead because the Government's medium- and long-term public debts management strategy for 2019–2025 did not include the AF for the project. 　 8. Assessment of Bank Performance a. Quality-at-Entry The Government implemented three major World Bank projects in Azerbaijan in support of water user organizations (i) In 1997, the Farm Privatization Project (FPP) (P040544) established water users association (WUA). The FPP piloted six WUAs. (ii) The Rehabilitation and Completion of Irrigation and Drainage Infrastructure Project (RIDIP) (P008284) followed. That project focused on the rehabilitation and completion of major off-farm conveyance canals, structures, and collector drains. (iii) The Irrigation Distribution System and Management Improvement Project (IDSMIP) (P008286) followed and piloted 22 WUAs on 52,000 ha of irrigated land. By the end of the project, the IDSMIP had formed more than 550 WUAs. Measuring structures (hydro-posts) were installed, which allowed WUAs to charge water on the basis of actual quantities supplied. In early 2003, an appropriate legal framework was drafted for the AIOJSC and subsequently approved in June 2004. The amended Law on Amelioration and Irrigation (LAI) included a special chapter consisting of 15 articles regulating the foundation, registration, organization, and supervision of WUAs in Azerbaijan. The law transferred the previous on-farm irrigation systems to WUAs and for designated the WUAs to supply bulk water for irrigation using long-term (20 year) water supply contracts. The Government promoted the WUA concept and restructured existing WUAs according to the amended law. The Water Users Association Development Support Project (WUAP or this project), followed building upon the lessons learned from the three previous operations. Lessons reveal that investing in farm irrigation and drainage rehabilitation could (i) increase yields by as much as 23% (ii) increase irrigated areas by as much as 5,200 hectares in rehabilitated irrigated areas; (iii) lead to a shift to higher value crops; and (iv) foster a willingness to pay more for reliable irrigation service. Another lessons learned centered on the distance of the Central WUA Support Unit in Baku from the WUAs and Raion WUA Support Units. As a result, design created four Regional Support Centers to cluster neighboring RSUs and deliver customized training programs and a system to track support activities and define specific local support needs. This design feature ensured better links between the Project Implementation Unit and the programs of the participant WUAs as well as improved the system for tracking funds and M&E support. Recognizing that the Government could not be the sole provider of the maintenance needs of the rehabilitated WUAs, the design also introduced Participatory Irrigation Management in the WUAs. Finally, project design recognized the medium sized risk from the limited capacity of WUAs to finance the drainage system by excluding this responsibility from the agreement with WUAs undertake management, operations, and maintenance of the transferred infrastructure. The objectives were realistic and clear. The Results Framework included five outcome indicators that were measurable, designed simply, and followed a logical sequence in support of the project’s Theory of Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) Change, although a little bit complicated by the binomial terms in calculating how to achieve outcomes. This meant that each indicator had two criteria to meet. The first defined a desirable outcome as a minimum condition. A second defined another minimum condition that the first condition must fulfill. The indicators also included the World Bank’s sector’s indicators for irrigation projects. There were five outcome indicators and three core sector indicators required by the World Bank, the latter were expressed as outputs. Despite the minor shortcoming at appraisal with regard to the risk in availability of local counterpart funds (evidenced by the shortfall in borrower contributions at project closing), the quality at entry is rated satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision The World Bank team conducted 12 implementation supervisory missions over the 7 year life of the project with frequent technical and field visits in between. The first mission was conducted a year after effectiveness. Candor in the quality of implementation reporting was evident in the Bank’s report on the delay in implementing the first component (institutional strengthening and capacity building). The Raion WUA Support Units considered their support to participating WUAs as an addition to their regular functions and anticipated financial top up to their state salaries. The Bank team recommended a small financial incentive to resolve the misunderstanding. Strong government commitment was evident in the implementing agency’s management initiative in adopting corrective measures such as heightened coaching of the Raion WUA Support Units to better understand their role in WUA service delivery framework. With a focus on development impact, by the time of the third mission, (June - September 2014) the Bank team had proactively identified the opportunities to address the threats to achieving the PDO including the possibility of Additional Financing to address the significant demand for rehabilitation and increased capacity building needs that were supposed to have been covered under the predecessor project. However, the country’s public debt management strategy for 2019-2025 did not envisage additional finance for the project. 　 There were only minor shortcomings in Bank supervision as evident in the three years it took to satisfactorily resolve the initial hiccup in the role of the Raion WUA Support Units in achieving the PDO. This as resolved by rearranging the support provided by consultants and intensified training and retraining activities. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) 9. M&E Design, Implementation, & Utilization a. M&E Design The Implementing Agency was the Amelioration and Irrigation Open Joint Stock Company (AIOJSC), established in 2006 from the State Amelioration and Irrigation Committee (SAIC). The company provided bulk water supplies to irrigation systems and developed and managed I&D systems throughout Azerbaijan. The PDO was clear. The Theory of change was supported by three key components and a robust results framework that consisted of 5 outcome indicators and 6 intermediate outcome indicators. However, the results framework included a couple of intermediate outcome indicators that the ICR acknowledged were not linked to the PDO - the number of WUAs with an adequate number of trained and experienced staff implementing good operational maintenance procedures, and the number of WUAs with established Representative Zones actively engaged in WUA management. The objectives were stated clearly and specific, although the M&E was designed only in 2014 when the project had been in operation for a period of 3 years. b. M&E Implementation The M&E system established for the project was implemented by the AIOJSC. A baseline survey conducted in August -October 2013. A Mid Term Review survey was conducted in September -October the following year. An end of project survey was conducted in October -November 2016. The company which conducted the survey developed the database on which the M&E was established. In 2015, the Implementation Support Mission raised issues on accuracy of the progress of outcome and results indicator and recommended strongly collaboration between the team of engineers, water management specialists and M&E specialist to develop better methods to measure outcome and results as well as design of survey questionnaires. The ICR reported that these concerns were resolved. c. M&E Utilization Survey results were used to improve M&E and project management. Implementation progress were reflected in quarterly project status reports and highlighted both financial and physical progress of the project. There were no change in targets although two intermediate outcome indicators were found to be imprecise but remained unchanged during implementation to avoid project restructuring. M&E Quality Rating Substantial 10. Other Issues Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) a. Safeguards ENVIRONMENTAL SAFEGUARDS: The following safeguards were identified at preparation: OP/BP 4.01 Environmental Assessment, OP/BP 4.09 Pest Management, OP/BP 4.12 Involuntary Resettlement,, and OP/BP 4.37 Safety of Dams and OP/BP 7.50 International Waterways. To avoid triggering OP/BP 4.04, Natural Habitats, and OP/BP 4.36 Forests, the project did not support rehabilitation that would adversely affect natural habitats and forests. The project triggered OP/BP 4.01 Environmental Assessment because of anticipated environmental impacts. The project was assigned a category B because the anticipated impacts were not significant or irreversible and could be addressed by mitigating measures, At project preparation, the exact location of project sites to be rehabilitated were unknown hence the implementing agency developed an Environmental Assessment and an Environmental Management and Monitoring Plan to identify impacts of the infrastructure rehabilitation in project areas with corresponding mitigating measures. Site specific Environmental Management Plans were prepared for each subproject proposal and mitigating measures incorporated into rehabilitation contracts. In the course of subproject implementation, the following issues were noted: (i) inadequate waste management; (ii) inadequate sanitary and hygienic conditions at campsites, (iii) insufficient attention by workers in the use of personal protection equipment, and (iv) unavailability of all required permits and licenses during inspections. The implementing agency closely supervised the adoption of corrective measures by contractors. Since these issues were associated with individual contract performance and not system-wide, compliance with safeguards measures was deemed satisfactory. Good practices adopted to meet the satisfactory environmental management of construction sites included: (i) use of construction materials only from certified/licensed sources; (ii) proper rehabilitation of construction and campsites upon completion of civil works; (iii) safe reuse of dredged materials; and (iv) avoidance of the cutting of trees and vegetation along rehabilitated canals. The project did not procure pesticides but because the PDO stimulated action to improve agricultural productivity, there was an increase in the use of pesticides, triggering OP/BP 4.09 Pest Management. Adverse impacts from the increased used of pesticides were met through customized Integrated Pest Management Training Program under the capacity building component of the project. In addition to pest management practices, the training programs covered good environmental management practices in agriculture. The project drew irrigation water from five dams, triggering OP/BP 4.37 Safety of Dams. Experts deemed these operationally safe. An independent dam safety expert conducted a safety assessment of dams and overall safety of dams were rated satisfactory. Ensuring dams operational safety was assigned to the Ministry of Emergency Situations. The consultant’s assessment of dam safety was conveyed to the Ministry for further actions. The rivers which supplied most of the water to the irrigation systems rehabilitated by the project were international waterways, triggering OP/BP 7.50 International Waterways. At preparation, the project determined that there would not be significant impacts on the quantity or quality of water flowing thorough these rivers because (1) design was limited to rehabilitation of existing schemes and no new constructions were anticipated; and (2) rehabilitation activities would have minimal to no impact on the quantity or quality of water. There were no adverse impacts on the rights of other riparian states on international waterways, A notification exception was issued per paragraph 7(a) of OP/BP 7.50. Monitoring confirmed that there were no significant increases in either withdrawal from or discharge to the international waterways. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) SOCIAL SAFEGUARDS: Specific project sites were unknown at preparation, triggering OP/BP 4.12 Involuntary Resettlement. The project prepared A Resettlement Policy Framework to guide the preparation of Resettlement Action Plans including compensation measures for project-affected people. No Resettlement Action Plans were prepared because all rehabilitations works were conducted within existing rights of way. No Environmental and Social Safeguard Management Plans were prepared. An environmental and social safeguard specialist was employed during the first two years of the project but left afterwards and was not replaced. There was no evidence of any resettlement nor negative social impacts from resettlement and the social safeguard performance was rated satisfactory. Grievances were handled locally and were mostly oral. Some contractors maintained written grievance log but were not retained by the implementing agency. Some grievances reached the implementing agency and were resolved satisfactorily, including an appeal made by local waters associations to draw attention to the low quality of contracted works. This appeal was submitted to the World Bank and the President of Azerbaijan. The ICR reported the appeal to have been resolved satisfactorily. Social safeguards performance was created satisfactory. b. Fiduciary Compliance FINANCIAL MANAGEMENT: The overall financial management system complied with World Bank financial management requirement and was rated satisfactory. Financial statements, audit reports were submitted on time with unmodified opinions. PROCUREMENT: A dedicated procurement specialist ensured compliance with the World Bank procurement procedures. Procurement performance was rated satisfactory and procurement risk rated moderate. Other than the appeal mentioned under the social safeguards above, there were no reported issues with procurement of works. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons The ICR offered five lessons from the project. This review selected three that had broad relevance beyond this project, they are presented below with some editorial changes. 1. Long-term support for Water Users Associations and the sector fosters sustainable service delivery models. This project benefited from a sequence of four World Bank projects beginning in 1997, each building on the outcome of the previous ones. These projects provided the building blocks in the sector which facilitated the outcome of this project - from establishing the legal framework to cumulative institutional support. Government willingness to partner in the reform of the sector is also crucial. The Government is implementing an ambitious water resource development program adding storage and several large irrigation canals. A secondary irrigation and drainage infrastructure will be developed but will depend on the Government priorities for irrigated agriculture in rural areas and response to the question of how economic benefits generated from water use in agriculture compare to those generated by other sectors. The National Water Resource Strategy of Azerbaijan sets a long-term goal, focusing on ‘sustainability and efficiency of water supplies to respective sectors, meeting environmental requirements while maximizing economic benefits’. If the irrigation and drainage programs were implemented as planned, secure water supply could be met by addressing both the secondary system level and the entire water management systems. Adjustments may include optimization of reservoir management, more efficient water conveyance and distribution, a shift toward adoption of advanced irrigation technology. Azerbaijan Water Users Associations are now well-established institutions and could expand their responsibilities to make them a preferred partner for local water resource management. The Azerbaijan model may be one that other countries could consider emulating. 2 . Virtuous improvement cycles could be built on well-defined selection criteria for participation. The project developed a set of well-defined criteria, including performance of the WUAs. The associations participated in the design stage to identify priority needs. They also participated in the rehabilitation works to foster ownership. The associations also better appreciated the importance of maintenance of the on-farm system through training. The project approach combining investments with technical assistance showed the associations that rehabilitating the irrigation and drainage system improved the delivery of water supply, facilitated an increase in agricultural productivity, and enabled association members to pay their service fees. Increased collection rates of service fees led to improved financing of the operations and maintenance needs of the on-farm system. A participatory approach by all key stakeholders reflecting their commitment supported a virtuous circle and provided the incentives for the associations to improve their performance. 3 . Sound construction control facilitates the management of large, decentralized investment programs. The project demonstrated that complex multiple site construction works can be managed without major delays by closely involving end users. Continued involvement by WUA chairmen facilitated timely resolution of issues. This enabled WUAs to improve irrigation service delivery to their members, who in turn acknowledged the improved services by making timely payments of irrigation service fees. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review WUAP (P107617) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a concise and detailed overview of the project. The narrative was consistent with the OPCS guidelines, supported the ratings with available evidence. The story line was clear, particularly in support of the virtuous cycle supported by the approach used in the sector. The ICR was candid, pointing out that it took three and a half years for the project to recover from the early delay by amending the implementation mechanism to address the capacity building needs of the project. In addition, the ICR noted that even extensive preparations for Additional Financing (AF) because of significant demand for rehabilitation works and training needs of WUAs, the AF was dropped because the government’s public debt management strategy for 2019-2025 did not envisage this AF for the project. The ICR highlighted the complementary activities of capacity building and infrastructure supported the impacts of the project to the theory of change. The quality of evidence was adequate, with sources noted although there was no repeat of the 2016 end of project survey following the extension of the project to end in 2018. The annexes provided extensive information, particularly in support of project efficiency (Annex 4). Lessons referenced project operations. The presentation in the report emphasized the various activities that supported the outcomes of the project. The only drawback was the sustainability of project outcomes being subjected to government prioritization. a. Quality of ICR Rating Substantial Page 15 of 15