Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) Report Number: ICRR0021674 1. Project Data Project ID Project Name P113151 VN- Industrial Pollution Management Country Practice Area(Lead) Vietnam Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-51750 30-Sep-2018 22,962,256.99 Bank Approval Date Closing Date (Actual) 25-Oct-2012 30-Sep-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 50,000,000.00 0.00 Actual 23,010,941.10 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p. 4) and the Financing Agreement of December 24, 2012 (p. 5) the objective of the project was “to improve compliance with industrial wastewater treatment regulations in four of the most industrialized provinces in Vietnam.” b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) No c. Will a split evaluation be undertaken? No d. Components Component 1: Environmental Policy, Monitoring and Enforcement (appraisal estimate US$22.68, actual US$14.5 million): This component was to finance: a) policy review and revision: providing support for a comprehensive review of the legal and regulatory framework on pollution management, including reviewing and revising laws, regulations, and other legal documents at central and provincial levels; b) environmental monitoring and enforcement: supporting the development of environmental monitoring infrastructure and the improvement in environmental enforcement activities, including: (i) acquisition of testing laboratory equipment, standard samples, monitoring equipment, protection gears, and vehicles; (ii) acquisition and installation of automatic water quality monitoring stations; (iii) monitoring of surface water quality, industrial zone (IZ) wastewater discharge, and inspection and enforcement; and (iv) development and upgrade of the environmental monitoring information management systems of Ministry of Natural Resources and Environment (MONRE) and the project provinces; and c) supporting the development of an industrial pollution information disclosure system, including piloting a public rating program of IZs environmental performance with full disclosure of results and scaling the program to a national level. Component 2: Performance-Based CETP (Centralized Effluent Treatment Plant) Financing (appraisal estimate US$22.59 million, actual US$5.14 million): This component was to provide performance-based loans to finance the construction of new Centralized Effluent Treatment Plants (CETPs), the expansion of existing CETPs, and the improvement of other relevant infrastructure of industrial wastewater management in the industrial zones of the project provinces. To qualify for such financing, CETP investors were required to meet robust design criteria and to adopt improved technical specifications. Performance-based disbursements will be subject to satisfactory quality of construction as assessed by Vietnam Environment Protection Fund (VEPF) (90% of loan) and a commitment letter by the CETP investors to meet compliance standards (final 10% of loan). Recognizing that traditional CETP financing (either concessional or commercial) had focused solely on CETP construction but paid little attention to actual performance of CETPs, this component was to use performance-based disbursement to incentivize CETP investors to comply with environmental regulations during the operation phase. Failure to do so was to result in reductions in the loan repayment term if discharge violations occur: immediate full repayment was required if three violations occur. Together with improved enforcement pressure and improved technical capacity at CETP investors, this performance-based concessional financing was expected to improve compliance of project financed IZs with wastewater effluent standards. Component 3: Implementation Support (appraisal estimate US$10.58 million, actual US$3.37 million): This component was to finance; i) capacity building with facilitating training on various topics such as environmental management, surface water quality monitoring, CETP operation and maintenance, and environmental information disclosure, facilitating stakeholder consultation workshops, and participating in overseas training courses, international study tours, domestic study tours on industrial pollution management; ii) technical assistance (TA) with supporting the improvement of knowledge on sound industrial pollution management in the project's river basin. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) Project Cost: The total project cost was estimated to US$58.85 million. Actual total cost was only US$29.87 million due to several reasons: procurement delays, the MoF’s decision that recurring costs such as support for monitoring and capacity building should not be covered by the project, the existing requirement in the Law on Public Debt Management for Industrial Zones to report profits for three consecutive years, and improved terms for commercial credit. Financing: The project was financed by a US$50 million IDA credit of which US$23.01 million was disbursed, i.e. less than 59% of the credit was disbursed Borrower Contribution: The Borrower was to contribute US$8.85 million. Actual contribution was US$6.86 million. Dates: The project closed on its original closing date and no restructuring took place. 3. Relevance of Objectives Rationale Over the last two decades Vietnam’s economy transformed from a primarily agrarian to a modern, diversified and market-oriented economy in addition to continuous periods of gross domestic product (GDP) growth, a high per capita gross national income, and a significant decline in the poverty head count ratio. Critical for Vietnam’s economic growth was the development and expansion of industrial zones (IZs). In 2010, the IZs contributed to about 30 percent of national industrial output and attracted 35 percent of national foreign direct investment (FDI), creating 1.6 million jobs directly and 1.8 million jobs indirectly. However, the country’s industrial development was accompanied by serious pollution and strong pressure on the natural resource base. In 2009, a law was implemented which required the IZs to provide sewer networks and centralized effluent treatment plants (CETPs) to collect and treat wastewater discharge by their tenants. At project appraisal it was estimated that IZs discharged approximately one cubic meters of untreated wastewater per day directly into receiving water bodies, mostly the Nhue-Day River and Don Nai River, the most heavily polluted rivers in Vietnam. Also, only approximately 60 percent of operating IZs had CETPs installed. The government started to address this issue in several ways such as promoting concessional financing for the construction of CETP construction through the Vietnam Development Plan and concessional loans for wastewater management in IZs through the state-managed Vietnam Environment Protection Fund (VEPF). According to the Bank team (May 9, 2019), the government developed the Socio-Economic Development Strategy (SEDS) (2011-2020) which aims to foster sustainable and effective growth along with social progress and equality, national resources and environment protection, socio-political stability, firm protection of independence-sovereignty-unification and territorial integrity of the country. The PDO was in line with national priorities such as the Socio-Economic Development Plan (SEDP) (2011-2015) which aimed to achieve rapid and sustainable economic development associated with transforming the growth model and restructuring the economy towards higher quality, effectiveness and competitiveness. The project’s objective was in line with the Bank’s Country Partnership Strategy (CPS) for FY2012 to FY2016 which focused under its second pillar on strengthening the environmental management of water Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) resources and pollution control. The project’s objective also supports the Bank’s most recent CPS for FY2018 to FY2022 which focuses in focus area three on ensuring environmental sustainability and resilience by reducing water pollution through improved compliance with industrial wastewater treatment standards. However, while there is clear alignment between the project’s development objectives and the country- and World Bank strategies, the relevance of the objectives is pitched at a level that does not adequately reflect a potential solution to a development problem. While acknowledging the difficulty of the operational environment, a shortcoming here was that the objective was not defined such that its achievements would be plausibly traceable to improvements envisioned to arise from improved compliance with industrial wastewater treatment regulations, whether those improvements were related to a cleaner environment and improved health for the affected population. These may be longer term targets but tracking them and identifying them is an important aspect of a successful development operation. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve compliance with industrial wastewater treatment regulations in four of the most industrialized provinces in Vietnam Rationale The project’s theory of change linked the provision of performance-based loans to financing construction or expansion of CETPs with repayment schedules tied to operational compliance to improving compliance with industrial wastewater treatment regulations in four of the most industrialized provinces in Vietnam. The project’s theory of change linked the review and revision of environmental policy, the provision of financial support to regular monitoring program at provincial DONREs, the acquisition of monitoring and laboratory equipment for DONRE inspection as well as the installation of automatic monitoring stations to monitor river surface water quality with improved compliance with industrial wastewater regulations in four of the most industrialized provinces in the country. Outputs: • Nine key legal documents were reviewed and revised. However, the adoption of four documents was still pending at project closure. Therefore, the target of seven documents was not achieved. These documents included Decree 82 on industrial zones and economic zones management (still pending), draft law on special economic and administrative zones (still pending), a circular on regulations for environmental monitoring activities, circular on environmental protection for used ship breaking (still pending), a circular on environment protection for chemicals, plants, protection chemicals, and veterinary drugs (still pending), a circular on Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) economic and technical norms for calibration of measurement devices of automatic and continuous monitoring stations, a circular on economic and technical norms for development of national state of environment reports, and a circular on economic and technical norms for environmental monitoring of ambient air, surface water, soil, underground water and rain water. • The percentage of operating IZs in the project provinces under proper wastewater effluent monitoring increased from 10 percent in 2012 to 97 percent in 2018, almost achieving the target of 100 percent. • 24 percent of IZs in the project provinces were rated for their environmental performance, not achieving the target of 100 percent. • 17 AMSs for measuring river surface water quality were established and are functioning, achieving the target of 17 AMSs. • 13 CETP financing proposals were received, surpassing the target of 8 proposals. • Five CETPs were constructed and are operational, not achieving the target of eight CETPs. • Eight technical assistance packages to improve knowledge on sound industrial pollution management in the project’s river basin context were completed, surpassing the target of seven technical assistance packages. • 3,257 participants were trained in over 75 project financed capacity building activities, surpassing the target of 2,500 participants. The training included three training modules: environmental management, environmental monitoring, and environmental information. • 90 percent of participating trainees stated in feed-back survey forms that the training improved their capacity, achieving the target of 90 percent. Outcomes: • Five project-financed IZs in the project provinces were in compliance with effluent discharge standards, not achieving the target of eight project financed IZs. • 69 percent of the 34 non-project financed operating IZs in the four project provinces complied with the effluent discharge standards, surpassing the target of 50 percent. However, according to the ICR (p. 14) the data used to assess IZ’s compliance with industrial wastewater regulation is derived from the provincial DONRE’s on-site monitoring activities. The frequency of these monitoring activities ranged between bi-annual and quarterly which makes the reliability of using such few data points to ensure that industrial wastewater discharge were in full compliance with effluent standards low. In 2015, the country introduced the legal requirement of having to install AMS at the CETP discharge which was supported by the project through technical assistance and direct CETP financing. However, provincial DONRE offices still need to adapt their technical capability to receive this data and report on it. Rating Substantial Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) OVERALL EFF TBL OLD Rationale The achievement of the objective is rated Substantial, but given the marginal achievement against a number of the indicators and the shortcomings on setting up rigorous systems for the monitoring and oversight of wastewater discharge, the overall Efficacy rating here does not sufficiently justify an overall Satisfactory rating for the project outcome. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: Both the PAD and the ICR conducted a cost-benefit analysis. The PAD identified the benefit as avoided cost of foregone medical treatment, the value of avoided lost earnings for individuals made ill as a result of pollution, the gains from reduced mortality, and the avoided cost of reduction in agricultural in aquaculture productivity. The PAD applied a discount rate of 12 percent and estimated a Net Present Value (NPV) of US$169 million over 17 years. The Internal Rate of Return (IRR) was estimated at 28 percent. The ICR (p. 17) used the same model with updated figures and assumption where possible. The benefits were identified as averted health care costs and productivity gains (the same as in the PAD but reduced by 30 percent to take into account that wastewater pollution is only one source of pollution in the two river basins). The averted health care costs resulting from reduced pollution was estimated at US$10 per beneficiary per year and the increased productivity resulting from three fewer sick days for two beneficiaries in a household were estimated at approximately US$7 per beneficiary per year. The ICR (p. 18) stated that in 2010 the Industrial Water World Magazine conducted a benchmarking survey among +11,000 subscribers which found that surcharges by municipalities for wastewater treatment ranged between US$0.33 to US$0.90 per cubic meter which is similar to US$0.41 which was used in the project’s ex- ante and ex-post cost-benefit analysis. The ICR estimated a NPV of US$17.6 million at a discount rate of 12 percent over 17 years and an IRR of 26 percent. According to the Bank team (May 9, 2019) the reason for the difference in NPV at appraisal and at closing was due to some differences in the investments and benefits amount and timing of the cash-flows. However, the Bank team noted that the IRR remained the same even under much more conservative assumptions than applied at the time of preparation. Operational Efficiency: At project closing, the project had disbursed less than 50 percent of the credit. According to the ICR (p. 18) the low disbursement of Component 1 (66 percent) resulted from procurement delays due to low capacity. Also, during the last two years of project implementation the MoF decided that recurring costs such as support for monitoring and capacity building should not be covered by the project. The low disbursement of Component 2 Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) (25 percent, i.e. ¼ of the allocated credit for that component) was a result of the existing requirement in the Law on Public Debt Management for Industrial Zones to report profits for three consecutive years. Also, the terms for commercial credit improved substantially during project implementation. In sum, the project’s Efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the objective is rated Substantial given that the PDO was relevant, but was pitched at a level that did not adequately reflect a potential solution to a development problem. Efficacy and Efficiency are both rated Substantial, but the rating for Efficacy is balanced against moderate shortcomings in the project's performance and thus the overall outcome rating is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome According to the ICR (p. 28) the government continues to be strongly committed to foster economic growth while ensuring sustainable environmental management. National policies on pollution control and environmental compliance have been implemented supporting the government’s responsibility to ensure human and environmental health. The Bank will continue to support the government in this area. According to the Bank team (May 9, 2019) while there is no dedicated follow-on investment project, pollution control continues to be a strategic development objective for Vietnam. The Bank is addressing this issue through Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) strategic water resource and water pollution analytical work which will further raise the profile and assist to prioritize the top pollution sources and recommended actions. The ICR (p. 28) stated that critical institutions such as the MONRE, DONRE, CEM, and VEPF have adequate capacity to monitor IZs compliance with industrial effluent discharge standards. The state budget has designated a budget for MONRE’s continuous operation of the 17 AMS after project closing. However, MONRE has decreased the frequency of its monitoring missions in the four provinces from a monthly to a quarterly or biannual basis. According to the Bank team (May 9, 2019) this indicates a transition towards self- monitoring, which will help improve compliance among IZs. Also, through self-monitoring the industrial zones can immediately take action upon an alert on an exceedance. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (p. 20) the project’s design was based on pre-feasibility studies, extensive stakeholder consultations, and lessons learned from previous Bank projects implemented in the country. Furthermore, the ICR stated that the project’s institutional arrangements for implementation were aligned with the project’s three key stakeholders, the Ministry of Planning and Investment (MPI), the Ministry of Natural Resources and Environment (MONRE), and the Vietnam Environment Protection Fund (VEPF) resulting in the entities being able to embed project activities into their daily work program and therefore increasing project ownership. However, with three PMUs, the implementation arrangements were complex, requiring intense coordination. The Bank team identified relevant risks such as low capacity for procurement at MPO and MONRE due to no previous experience in implementing Bank projects. The Bank addressed this risk by allocating project funds for intense training (especially in procurement and financial management) and capacity building at all implementation levels. Another risk that was rated as Substantial during project preparation was the risk that narrower and short-term economic interests would undermine the government’s efforts to decrease industrial pollution. The project mitigated this risk by focusing on a limited but high-profile pollution issue – industrial wastewater pollution from large IZs. Also, the project built monitoring and enforcement capacity within the environmental agencies in the four provinces to enable them to adequately monitor wastewater discharges from their respective IZs and take necessary enforcement actions and involving the public in industrial pollution monitoring by making relevant information public in a timely manner and processing public complaints properly. While most mitigation measures were adequate, low capacity in procurement and financial management resulted in implementation delays which was not adequately taken into account at project appraisal. A few other risks not identified during project appraisal materialized during implementation, such as the falling commercial interest rate and restriction of use of ODA funds for certain expenditure categories. Also, according to the ICR (p. 22), lengthy approval process within MONRE to obtain land clearances for the installation of basic and extended automatic monitoring stations at the specified coordinates along the river bed on public lands resulted in lack of time for monitoring river water quality for consecutive years. According to the ICR (p. 20) a shortcoming of the project’s design was that it did not incorporate into its Operations Manual the 2010 decree on on-lending of the Government’s Foreign Loans. The decree required beneficiaries of on-lend overseas development assistance (ODA) funds to provide proof of three Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) consecutive years of profit. The Bank addressed this issue in May 2015 by revising the project’s Operations Manual. Furthermore, the M&E design had some significant shortcomings such as most indicators lacking a baseline, inadequate selection of indicators to measure project activities, and insufficient frequency of measuring compliance with regulations (see section 9a for more details). Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision According to the ICR (p. 27) the Bank conducted regular supervision missions throughout project implementation. Also, the project benefited from a co-Task Team Leader being located in the country. Furthermore, the ICR stated that the Bank team monitored financial management, procurement, and safeguard compliance closely. Also, the Bank team provided training to strengthen the PMU’s capacity in financial management and procurement. According to the ICR (p. 22) the Bank team was proactive in identifying new opportunities for additional financing and project restructuring. When it became obvious after the mid-term review that only one of the PDO indicators could be achieved within the original implementation period the Bank tried to restructure the project. While the restructuring was approved by the Prime Minister in February 2017, the Ministry of Finance blocked the revision of the Financing Agreement due to the proposed direct support to monitoring and enforcement activities at provincial DONRE offices which was already covered by the budget support to line ministries. The Bank team did not collect data on IZs compliance in non-project provinces as originally planned. This made attributing project activities to results challenging, which was a significant shortcoming. The ICR (p. 28) stated that the Bank team ensured partial transition arrangements. The CETPs, which were financed under the project, will continue their cooperation with VEPF while MONRE will maintain the 17 AMS, which were installed to monitor the river water quality. Also, the remaining financing for CETP will be repaid, and the regular monitoring at the four provincial DONRE returned to the schedules which had been in place before project implementation. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) a. M&E Design The project’s theory of change and how key activities would lead to intended outcomes was sound and reflected in the Results Framework. The project’s objective was plausibly traceable to improvements envisioned to arise from improved compliance with industrial wastewater treatment regulations, whether those improvements were related to a cleaner environment and improved health for the affected population, challenging. Also, while the two PDO outcome indicators measured “compliance performance for IZs with project financed CETPs” and for “IZs with non-project financed CETPs”, a single indicator to measure” IZs compliance as a percentage of all IZs in the four project provinces” would have better measured the achievement of the project’s objective. While in the PAD the definition of “compliance” as stated in the PDO statement was tied to the industrial wastewater treatment regulation QCVN40:2011, it was found during project implementation that this definition was too strict. The Bank team discussed a scale and based on international best practice “compliance” was defined as “no exceedances above 1.2 times the allowed threshold in any of the 30 parameters.” Also, the wording “improved compliance” was not sufficiently specific and instead could have been phrased as “reduced pollution load” which is a quantitative Bank core indicator and is often used in other pollution control and abatement projects. Also, during project preparation it was planned that compliance performance would be assessed twice a year which was found to be too infrequent given for establishing industrial zones’ compliance with wastewater treatment regulations. Furthermore, both PDO indicators and most intermediate outcome indicators lacked a baseline. b. M&E Implementation According to the ICR (p. 25) the three PMUs collected and analyzed data in a systematic way. The MPI PMU integrated all reports into a combined report on a quarterly basis. However, a framework, which was outlined in the PAD (p. 33) for benchmarking compliance of the four project provinces with compliance performance in four non-project provinces within the same two river basins was not implemented and data for these four provinces was never collected. This did not allow for establishing the attribution between project activities and outcomes. Also, as stated in section 4, the data used to assess IZ’s compliance with industrial wastewater regulation was derived from the provincial DONRE’s on-site monitoring activities which only took place on a bi-annual or quarterly basis, making reliability of such few data points low. In 2015, the country introduced the legal requirement of having to install AMS at the CETP discharge which was supported by the project through technical assistance and direct CETP financing. However, provincial DONRE offices still need to adapt their technical capability to receive this data and report on it. Therefore, since a continuous monitoring and reporting on this data to the national level has not been possible yet, it cannot be used for compliance monitoring. c. M&E Utilization According to the ICR (p. 25) the PMUs at MONRE and MPO reviewed collected monitoring, inspection, enforcement, and implementation progress data to assess the project’s performance and achievement of the PDO and to make modifications to project implementation if necessary. For example, the project’s Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) mid-term data indicated that the project would benefit from a restructuring and a closing date extension. In addition, surveys of workshop participants were used to refocus the training materials to match the participants’ demands. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as category A and triggered the Bank’s safeguard policy OP/BP 4.01 (Environmental Assessment) and OP/BP 4.12 (Involuntary Resettlement). According to the ICR (p. 26) the project developed an Environmental and Social Management Framework (ESMF) which was adopted by the Ministry of Planning and Investment (MPI), the Ministry of Natural Resources and Environment (MONRE), and the Vietnam Environment and Protection Fund (VEPF) and applied to all Centralized Effluent Treatment Plants (CETPs) and Automatic Monitoring Stations (AMSs). The ICR (p. 26) stated that CETPs and AMSs were screened for financing and compliance with the ESMF throughout project implementation. Furthermore, the Environmental Management Plans (EMPs) were prepared in accordance to the government’s and the Bank’s safeguard policies and publicly disclosed. According to the ICR (p. 26) the project addressed all environmental issues such as the construction of the CETPs and AMSs satisfactorily. Also, due diligence was conducted on all industrial zones in regard to land acquisition and compensation and the functionality of the grievances redress mechanism was found to be accessible and functional. b. Fiduciary Compliance Financial Management: According to the ICR (p. 26) regular financial management reviews were conducted and the project’s financial management system provided accurate and timely information that the loan was being used for the intended purposes. However, from late 2013 onwards the project’s financial management was consistently rated Moderately Satisfactory resulting from: i) significant delays in project budget approval and insufficient budget allocation; ii) lack of coordination between implementation agencies and disagreements with the Ministry of Finance regarding eligible expenditures; iii) slow payments of contractors; iv) shortcomings in the contract management system; and v) need to implement audit recommendations in a timely manner. According to the Bank team (May 9, 2019) the Bank addressed these issues by: i) contracting a consultant to review reports to facilitate timely payments to consultants for work delivered, but also to ensure that the contractually required deliverable were delivered with appropriate quality (ii) conducting frequent meetings with implementing agencies to discuss budget approval and budget allocation requirements; and (iii) holding discussions with the Ministry of Finance and implementing ministries to clarify the eligibility of expenditures of the project. Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) The project’s end disbursement date was further extended to allow for the MOF to approve the final applications for eligible expenditures which had been submitted before. According to the Bank team (May 16, 2019) the project had a qualified audit in 2017 due to incomplete recognition of project expenditure for the VEPF component in that year. Financial Management has subsequently been followed up closely and the issue was resolved, and the project did not receive any more qualified audited opinions for the FY2018 audit. Procurement: According to the ICR (p. 26) procurement was rated Moderately Satisfactory throughout project implementation. The project developed a procurement plan which was cleared by the Bank and approved by the MONRE/MPI and was modified throughout implementation to adjust cost estimates, time schedules, and new procurement packages. The project experienced procurement delays in all stages of the procurement cycle due to weak capacity, as there was a lengthy process to prepare procurement documents such as the technical specifications for the procurement of goods and Terms of References for consulting services. The ICR (p. 21) stated that the harmonization of the provisions of the Law on Tendering and the Bank’s procurement guidelines resulted in an early implementation delay. Also, lengthy bid/proposal evaluation period, and weak contract management resulted in delays. The ICR (p. 21) stated that the procurement of equipment was challenging due to MONRE’s internal procedures and low capacity. In addition, also only a few global suppliers were able to deliver to the requested standards. The procurement process suffered from mis-procurement, incomplete order delivery, and user training without translated material delivered. According to the Bank team (May 9, 2019) the Bank addressed these issues by providing frequent implementation support to assist the client in the implement of the procurement plan. The Bank also reviewed drafted terms of reference for contracts and consultancy reports. c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) 12. Lessons The ICR (p. 29-30) provided useful lessons learned which were adapted by IEG: • If indicators to ensure monitoring of Industrial Zone’s compliance with industrial wastewater treatment regulations are omitted in the Results Framework, there is a risk that reporting on compliance will be jeopardized. In this project’s Results Framework no such indicator was included but in a pollution abatement project in Lebanon an indicator measuring “regularly published monitoring reports covering environmental compliance of IZs by authorities” was selected. • Selecting indicators that can be monitored in sufficient detail and on a regular basis is critical for ensuring more reliable monitoring of outcomes. In this project the data used to assess IZ’s compliance with industrial wastewater regulation was derived from the provincial DONRE’s on-site monitoring activities which only took place on a bi-annual or quarterly basis, making reliability of such few data points low. Average representative sampling with more samples being averaged out or continuous sampling and including minor flexibilities for exceedances would have been beneficial for measuring improvement in compliance. • Leaving some degree of flexibility in the project design is useful for adjusting activities during implementation when needed. This project defined the exact rules and regulations to be reviewed and revised, the technical specifications for the technical equipment to be procured, and the locations for AMSs to be installed, not allowing for sufficient adjustment to accommodate circumstances on the ground. This resulted in significant implementation bottlenecks and delays. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a good overview of project preparation and implementation. Also, the ICR was outcome driven, internally consistent and sufficiently candid. It included an adequate Economic analysis and useful lessons learned. However, it did not provide sufficient information on how financial management and procurement issues were addressed by the Bank and whether the external auditor’s opinion was qualified. a. Quality of ICR Rating Substantial Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review VN- Industrial Pollution Management (P113151) Page 14 of 14