Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) Report Number: ICRR0021745 1. Project Data Project ID Project Name P112615 Kiribati Adaptation Program - Phase III Country Practice Area(Lead) Kiribati Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-10875,TF-11351,TF-11448,TF- 31-Dec-2018 11,561,597.92 11818,TF-A5560 Bank Approval Date Closing Date (Actual) 15-Sep-2011 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 11,567,162.00 11,567,162.00 Revised Commitment 11,561,597.92 11,561,597.92 Actual 11,561,597.92 11,561,597.92 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Fernando Manibog Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (p. iv) and the Global Environment Trust Fund Grant Agreement of October 14, 2011 (p. 6) the objective of the project was “to improve the resilience of Kiribati to the impacts of climate change on freshwater supply and coastal infrastructure.” Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) The project was the third phase of a multi-year program supported by the World Bank. The first phase of the Kiribati Adaptation Program (KAP) was implemented from 2002 to 2005 and the second phase was implemented from 2006 to 2011. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project included four components: Component C1: Improve water resource use and management (appraisal estimate US$4.4 million, actual US$5.4 million): This component was to finance the following activities: i) expand the installation of groundwater abstraction systems in North Tarawa based on investigations and community consultation work completed under KAP II; ii) detect leakage and repair real losses; iii) expand the program of installing rainwater harvesting systems on public buildings for community use that was started under KAP II; and iv) improve the legislative and regulatory framework and governance model for water resources management with a focus on improved management and protection of the water reserves in Bonriki and Buota. Component C2: Increase coastal resilience (appraisal estimate US$2.7 million, actual US$2.6 million): This component was to finance the following activities: i) implement further works at priority sites identified and started under KAP II; ii) continue to further build the capability of the Ministry of Public Works and Utilities (MPWU) capability in coastal assessment, options analysis, design and construction through the appointment of a Senior Civil Engineer seconded to MPWU; iii) develop skills in coastal infrastructure asset management; and iv) continue/expand the mangrove planting program commenced under KAP III. Component C3: Strengthen the Capacity to Manage the Effects of Climate Change and Natural Hazards (appraisal estimate US$2.1 million, actual US$2.1 million): This component was to finance the following activities: i) providing additional support to the Strategic Risk Management Unit (SRMU) to better undertake its role with respect to Climate Change Adaptation (CCA) and Disaster Risk Reduction (DRR) coordination, integration and policy harmonization functions; ii) preparing coastal management policy and locally managed adaptation planning; iii) implementing communications and media activities relating to CCA and DRR over the duration of the project; iv) maintaining and continuation of populating the GoK’s website with KAP III outputs, stories and general information; and v) operating a small grants scheme for CCA and DRR activities at community level with an agreed proportion of activities increasingly identified through the locally managed adaptation plans. Component C4: Project Management, Monitoring and Evaluation (appraisal estimate US$1.5 million, actual US$1.4 million): This component was to finance the establishment and operation of an appropriately staffed Project Management Unit (PMU), audit of the project’s financial systems and results, independent review of the project progress and achievements at the time the project has achieved roughly 50% disbursement of funds, and monitoring of project implementation progress and other key performance Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) indicators, end of project review of the project achievements and final verification of the indicators from the Results Framework. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$10.8 million. Actual cost was US$11.62 million. Financing: The project was to be financed by a Global Environment Facility Least Developed Country Fund (GEF LDCF) in the amount of US$3 million (actual disbursement was US$2.9 million), Japan Policy and Human Resources Development Fund in the amount of US$1.8 million (which was fully disbursed), Global Facility for Disaster Reduction and Recovery (GFDRR) in the amount of US$900,000 (which was fully disbursed), and the Australian government in the amount of US$4.85 million (which was increased by US$0.85 million during the 2017 restructuring and actually disbursed US$5.4 million). Borrower Contribution: The Borrower was to contribute US$250,000. Actual contribution was US$53,000. According to the Bank team (June 25, 2019) significant savings were realized in project administrative costs, and the full planned amount of Borrower’s in-kind contribution was not needed. Dates: The project was restructured twice:  On January 11, 2016 the project was restructured to: i) adapt the Results Framework through revising or introducing new indicators to better measure desired outcomes, deletion of some indicators, and modification of yearly targets to better reflect realistic implementation conditions and progress; ii) addition of an extension to the President’s office building as part of Component 3; iii) extension of the project’s closing date by 18 months from August 31, 2016 to February 28, 2018 to utilize project savings resulting from favorable exchange rate movements.  On August 3, 2017 the project was restructured to: i) scale up existing activities on water network improvement, resilience fund grants, and capacity building for the lead implementing agency; (ii) revise the Results Framework to effectively capture the outcomes associated with the scale-up; (iii) allow inclusion of a new activity, the development of a Long-Term Coastal Security Strategy; and (iv) extend the closing date from February 28, 2018 to December 31,2018 to align the closing date of all funding sources. 3. Relevance of Objectives Rationale Kiribati is one of the most vulnerable countries in the world to the effects of climate change. The country’s capacity to respond to climate change related risks is impacted by low atolls, isolated location, small land area separated by vast oceans, high population concentration, and the costs of providing basic services. The rise of the sea-level and natural disasters such as drought and weather fluctuations pose significant and direct additional threats to sectors and resources central to human and national development. Kiribati’s remote location and many small islands impose a significant economic challenge. At the time of project appraisal in 2010, Kiribati had the lowest gross national per capita income in the region of US$2,090 and Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) one of the highest rates of poverty with 22 percent of the population in extreme poverty and 66 percent at risk of falling into extreme poverty. Exports of marine products and income from fishing license account for approximately 10 to 20 percent and 20 to 25 percent, respectively, of the Gross National Product (GNP). The Kiribati Sustainable Development Plan (2008-2011) estimated that without adaptation to climate change, Kiribati could face approximately US$8 to US$16 million a year (approximately six percent of annual Gross Domestic Product (GDP) at appraisal) in climate change related damages by 2050. The objective of the project is in line with the National Adaptation Program of Action which was adopted by the government in 2007 and identified water resources, coastal zone management, and community resilience as key priorities. The project’s objective also supported the Kiribati Development Plan (2016- 2019), which identified climate change as the long-term issue threatening the sustainability of the country’s economic development. The project contributed to the Plan’s key performance areas on environment (coastal resilience) and infrastructure (protection of water resources and improvement of fresh water supply). The project’s objective also supported Kiribati’s Climate Change Policy (2018), which identified coastal protection and infrastructure, water security, environmental sustainability and resilience, and disaster risk management as priorities. Furthermore, the project’s objective is also in line with the Bank’s Pacific Islands Regional Partnership Framework (FY2017-FY2021), which emphasized the importance of protecting incomes and livelihoods, through strengthened resilience to natural disasters and climate change. The project’s objective was overly ambitious in terms of scope and geographic coverage. According to the ICR (p. 23) the project benefited from the Bank’s decade long involvement with the government to address climate change impacts. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve the resilience of Kiribati to the impacts of climate change on freshwater supply: Rationale The project’s theory of change established a valid causal chain between the project’s provision of additional groundwater abstraction and water harvesting systems and the project outcome of improving fresh water supply because those project activities would increase the number of people who would gain access to more sources of water supply. The project outcome on improved water supply resilience can be partly attributed to this Phase III project, since earlier interventions under KAP I and KAP II already provided inputs and outputs related to groundwater abstraction and water harvesting systems, which were expanded under Phase III. Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) The project output of water transmission rehabilitation was causally and directly linked to the project outcome of improved resilience in fresh water supply because the rehabilitation activity would decrease leakage and thus increase the number of people with permanent access to pressurized water. The project’s outputs on construction of seawalls completed on South Tarawa and coastline protected by mangroves was causally and directly linked to the project outcome of increased amount of people and assets protected from coastal erosion, flooding, and storm damage. This Theory of Change applies to the two PDOs. The PAD did not include a Theory of Change. Outputs (the ICRR only mentions targets when they were included in the ICR):  Three groundwater abstraction galleries at the Immaculate Heart College, the Notoue village, and the Tabonibara village, were completed as planned.  Health and hygiene training programs for beneficiaries were implemented by the Ministry of Health and Medical Services (MoHMS) and Operation & Maintenance trainings of village-based committees were implemented by the Ministry of Infrastructure and Sustainable Energy (MISE).  Six rainwater harvesting systems were built at faith-based and public buildings and community halls in North Tarawa.  A large-scale rainwater harvesting system, with a storage capacity of 300,000 liters, was constructed in Buota, North Tarawa.  Collection roofs over the tanks, in-line ultraviolet treatment and on-demand, solar-powered pumping system and associated reticulation and tap-stands to serve the adjacent five villages were constructed.  The Resilience Fund for community-led small adaptation grants financed the construction of (a) 328 rainwater harvesting tanks (ranging from 5,000 to 10,500 liter volume), and (b) 36 solar and Tamana pump systems (simple hand-powered system made using plastic tubes that can greatly reduce water contamination by allowing pumping from closed wells), thus providing 14.2 m³ / day to approximately 13,400 people in 70 community groups.  The secondary distribution networks were replaced: laying of 4,602 meters of new pipes, surpassing the target of 3,000 meters; installing 438 tap-stands and water meters, surpassing the target of 180 meters, and replacing a 20 m³ storage tank, fittings, and valves. Outcomes:  The number of people with access to improved water sources rose from 5,000 people in 2017 to 12,780 people in 2018, surpassing the target of 11,000 people.  The number of women provided with access to improved water resources increased from 2,500 women in 2017 to 6,495 women in 2018, surpassing the target of 6,000 women.  The number of people provided with access to improved water resources in rural areas increased from 3,600 people in 2017 to 8,435 people in 2018.  The number of people provided with access to improved water sources in urban areas increased from 1,400 people in 2017 to 4,345 people in 2018, surpassing the target of 2,600 people. Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615)  The volume of potable water provided from new groundwater sources increased from six kiloliters per day in 2016 to 22 kiloliters per day in 2018, surpassing the target of 21 kiloliters per day.  The volume of potable water saved through reduced leakage increased from >5 kiloliters per day in 2011 to 645 kiloliters per day in 2016, surpassing the target of 190 kiloliters per day.  The volume of potable water provided from new rainwater harvesting system increased from one kiloliter per day in 2016 to 6.5 kiloliters per day in 2018, surpassing the target of 6.1 kiloliters per day.  645 m³ of water were saved from rehabilitation of the transmission main and storage reservoirs.  438 households had 24/7 pressurized water supply in the pilot zones providing an additional of 140 m³ of water, surpassing the target of 180 of households and the service level of providing four hours per day. Rating Substantial OBJECTIVE 2 Objective To improve the resilience of Kiribati to the impacts of climate change on coastal infrastructure: Rationale Outputs:  Detailed coastal risk assessments and engineering studies for nine vulnerable coastal erosion sites on South Tarawa were conducted.  A coastline asset condition assessment was completed and documented for all major non-government assets along the South Tarawa coastline, achieving the target.  Three seawalls measuring 370 linear meters were completed in South Tarawa.  Community-based mangrove managements plans were prepared and adopted under the village development committees.  Over 35,000 seedlings for mangrove plantings along 46 segments of coastline measuring 1.005 linear kilometers were completed by communities in nine outer islands, under the training and monitoring of the Ministry of Environment, Lands, and Agricultural Development (MELAD).  The number of groundwater abstraction systems installed and operating in North Tarawa increased from one system in 2011 to three systems in 2018, achieving the target of three systems.  A Long-Term Coastal Security Strategy (LTCSS) was developed. The strategy established a coherent framework for managing the use of the coast by responsible agencies and communities. An updated coastal risk assessment for highly vulnerable areas in South Tarawa and outer islands was conducted. Also, a consistent set of practical tools aiming at breaking the reactionary cycle of the government communities funding low-cost, poorly constructed, under-designed seawalls requiring frequent maintenance.  The government adopted a structured approach of identifying hazards, development risks, and potential interventions called the Whole of Island Approach (WoIA). In line with the WoIA the project supported the development of nine island vulnerability assessments and four island strategic plans covering a population of 33,761 and 13,745 people, respectively. The strategic plans identified the Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) risks, issues, and development priorities for each island for expenditure planning and directing donor investment.  The number of community-led resilience sub-projects completed increased from 35 sub-projects in 2017 to 100 sub-projects in 2018, surpassing the target of 80 sub-projects. Outcomes:  The amount of coastline protected increased from 0.50 kilometers in 2011 to 1.87 kilometers in 2017, surpassing the original target of 1.60 kilometers and the revised target of 1.20 kilometers. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The achievement of both objectives is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: The PAD did not conduct an Economic analysis stating (p. 12) that it is challenging to quantify the damage associated with future climate events and the project benefits associated with climate risk reduction. Also, all project activities had not been fully defined during appraisal, therefore, costs and benefits could not be estimated. The ICR (p. 20) conducted a cost-benefit analysis for key investments under Components 1 and 2, which comprised 84 percent of the of the two components and 51 percent of the entire project cost. The analysis did not include any costs related to capacity building and policy advisory support since the benefits of these activities were difficult to quantify and fully attribute to the project. The analysis applied a discount rate of six percent. Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) The Net Present Value (NPV) for water resilience investments ranged from US$1.32 million to US$3.53 million. The Economic Internal Rate of Return (EIRR) for these activities ranged from 10 percent to 38 percent. The NPV for the coastal resilience investment was US$1.06 million and the EIRR was 10 percent. The ICR (p. 21) stated that several benefits were not quantified, such as: reduction in the incidence of water-borne diseases, opportunity costs of school absenteeism among school age population, loss-of-life avoided, savings resulting from switching from alternative water supply to piped water supply, avoided damage costs on power and water mains, avoided business disruptions due to power and water outages and traffic delays, and avoided disruptions in lifeline and other public services. Operational Efficiency: During appraisal, the project’s administrative costs were estimated to be 13.6 percent of the revised budget, which ended up being lower at 12 percent. The 28-month extension of the project’s closing date did not have a negative impact on the administrative costs, indicating that the project’s operation was efficient. Also, the ICR (p. 21) stated that the share of management costs in the total budget decreased from 21 percent under KAPII to 12 percent under KAPIII, indicating that the project benefited from efficiency gains from systems, structures and capacity building activities implemented during the previous phases. However, the project design included a large amount of Implementing Agencies (IAs). The ICR (p. 24) stated that this resulted in a lack of clarity in terms of responsibility among IAs and competing demands on the technical advisors who supported the IAs. Third, the Bank team did not take into account the amount of infrastructure projects the IAs were responsible for. The ICR (p. 24) stated that in October 2014, three IAs were responsible for 24 different infrastructure projects, resulting in KAP III being given less priority due to their smaller size. And finally, the ICR (p. 28) stated that the project design assumed that the water supply network and coastal infrastructure could be constructed by local contractors, which turned out to be inaccurate and resulted in implementation delays. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome During the 2016 project restructuring and the Additional Financing in 2017, the Results Framework was revised to modify or introduce new indicators to better measure desired outcomes, delete some indicators, and modify Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) yearly targets to better reflect realistic implementation conditions and progress. Since the end targets were not revised, a split rating was not conducted in this Review. The relevance of the objective was High given its alignment with Kiribati’s Climate Change Policy and the Bank’s Pacific Islands Regional Partnership Framework (FY2017-FY2021). Efficacy and Efficiency were Substantial. On the basis of these sub-ratings, the project’s overall outcome rating is Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (p. 17) stated that in order to ensure the sustainability of O&M of the groundwater abstraction galleries, the Ministry of Women, Youth, and Social Affairs registered village committees that were empowered with constitutions and authority to collect small fees from residents. The registration was authorized under the Incorporated Societies Act. In addition, to support sustainability and appreciate the voluntary land agreements by future generations, a traditional community agreement "Te Berita", which binds all members to abide by the pledge, entailing a cultural sanction for enforcement, was implemented. According to the ICR (p. 17) MISE is currently taking steps to draft legislation for similar voluntary water protection zones for future use. The ICR (p. 29) stated that greater social fragmentation in Buota poses a moderate risk on the sustainability of village water supply systems which is being mitigated by the management of the Public Utilities Board (PUB). PUB has adequate capacity for long-term O&M but requires government funding since it only generates limited income. In order to ensure the sustainability of PUB's operations it will be critical to establish a more efficient and accessible customer billing and collection system. The sustainability of coastal protection works is threatened by the limited public financing and capacity of the MISE. Also, the Office of the President, which has a clear work program and well identified priorities on climate change policies, is heavily dependent on donor funds. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (p. 24) the design of the project was built on lessons learned from KAP I and II. Lessons learned included the importance of day-to-day community engagement across all components and the need to address complex and sensitive policy issues such as land and asset ownership, to ensure the sustainability of the infrastructure investments. In order to apply these lessons, long-term resources were budgeted. The ICR (p. 28) stated that the Bank team included the relevant expertise including a mix of coastal and water resource engineering and urban and community infrastructure development specialists. The PAD (p. 11) stated that the Bank identified relevant risks such as weak implementation capacity, competing demands from other infrastructure projects, and project rejections by communities. Mitigation measures included on-the-job training and capacity building, partnering with donors and programs to Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) increase complementarity and effectiveness of interventions, and consultations with and engagement of communities. According to the ICR (p. 24) mitigation measures were adequately identified. However, there were also some shortcomings. First, the project scope and geographic coverage was ambitious. Second, the project design included a large amount of Implementing Agencies (IAs). The ICR (p. 24) stated that this resulted in a lack of clarity in terms of responsibility among IAs and competing demands on the technical advisors who supported the IAs. Third, the Bank team did not take into account the large number of infrastructure projects the IAs were responsible for. The ICR (p. 24) stated that in October 2014, three IAs were responsible for 24 different infrastructure projects, resulting in KAP III being given less priority due to their smaller size. And finally, the ICR (p. 28) stated that the project design assumed that the water supply network and coastal infrastructure could be constructed by local contractors, which turned out to be inaccurate and resulted in implementation delays. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p. 28) the Bank team conducted regular supervision missions on a semi-annual basis. In addition, the Bank team conducted technical missions, procurement support missions, and financial management implementation review missions. The ICR stated that the Bank team’s reporting was of high quality and appropriately candid. Furthermore, the ICR (p. 29) stated that the Bank team provided support to strengthen the fiduciary and technical capacity within the IAs and the PMU, reviewed tender documents, and ensured the project’s compliance with the Bank’s safeguards. Even though there were changes in Task Team Leaders, who were all based in DC, specialists based in the Sydney office were closely involved throughout implementation which allowed for continuity and long-term relationships with the counterpart. When it became clear that international contractors were not interested in providing services due to insufficient profit gains due to the remote location of Kiribati and domestic contractors did not have sufficient capacity, the Bank team used Force Account and specialist technical consulting services. The ICR (p. 29) stated that this resulted capacity building within the IAs and job creation for local communities. The Bank team restructured the project twice to modify the Results Framework to allow for better measurement of outcomes and to obtain Additional Financing for strengthening the resilience at the community level and build on well-functioning implementation mechanisms. However, according to the ICR (p. 29) an earlier restructuring might have had a positive impact on the project’s performance. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s theory of change and how key activities and outputs would lead to the intended outcomes was sound and reflected in the Results Framework. The objectives were clearly specified. The indicators included a mix of qualitative and quantitative targets and were specific, measurable, relevant and the majority had a baseline. Data sampling methods included beneficiary surveys and impact assessment for selected components. However, a shortcoming of the original M&E design was that it mainly focused on outputs. The ICR (p. 26) stated that the M&E design would have benefitted from additional indicators focusing on outcomes such as the behavior change of water users, governance outcomes for sustainability, and adoption of relevant policies. It seems that some indicator’s targets could have been more ambitious given that for example the indicator for measuring the volume of potable water saved through reduced leakage already surpassed the target in 2016, almost two years before project closure. b. M&E Implementation During the 2016 project restructuring and the Additional Financing in 2017, the Results Framework was revised to modify or introduce new indicators to better measure desired outcomes, delete some indicators, and modify yearly targets to better reflect realistic implementation conditions and progress. However, end targets were not revised. According to the ICR (p. 26) baseline data for the project were collected and analyzed during the completion of KAP II. The project conducted random-sample household surveys, expenditure diaries and an impact assessment for the sub-projects of the Resilience Fund. The ICR (p. 26) stated that the Implementation Status Reports consistently reported on all indicators. According to the Bank team (June 25, 2019) the data were found to be reliable and of good quality and the M&E functions and processes are likely to be sustained after project closure. For example, ongoing mangrove awareness-raising, initiated by MELAD under the project, is being conducted during Island Council meetings, at village meetings in maneabas, and during school visits. Students and teachers tend to the plantings and revegetation as part of their environmental studies. MELAD officers continuously monitor the seedlings, as two years (on average) are needed to reach full viability. According to the Bank team (June 25, 2019) the Office of the President (OB), with the support of the PMU, worked with Island Councils and the Ministry of Internal Affairs to monitor the completion of the Resilience Fund sub-projects, and MELAD actively monitored and regularly reported on implementation progress of the outer island mangrove planting activities. c. M&E Utilization According to the ICR (p. 26) M&E results were used to inform decision making and supported resource allocation and modification of project implementation. Also, donors, especially the Australian High Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) Commission was briefed on a regular basis on progress towards the achievement of the outcomes resulting in Additional Financing. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project triggered the Bank’s safeguard policies on Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12) and was classified as category B. The ICR (p. 27) stated that an Environmental Management Plan (EMP) was developed and disclosed and all civil work contractors and implementing agencies carrying out Force Account works followed the EMP. Also, all major works included an environmental impact assessment. The PMU collaborated with MELAD to monitor safeguard compliance on a regular basis and addressed environmental issues that were identified. According to the ICR (p. 27) the project complied with the environmental safeguard policy. According to the ICR (p. 27) the project used an existing Land Acquisition and Resettlement Policy Framework since the impacts of the project were not known before implementation. The framework was modified during implementation before being disclosed. The project did not require any physical displacement or relocation. The ICR stated that for the construction of rainwater harvesting systems in North Tarawa included full consultation and community agreements. Also, for the groundwater abstraction systems implemented in North Tarawa, landowners provided land voluntarily for use as a water reserve and were fully compensated for any livelihood loss. According to the ICR, the safeguard policy was adequately complied with. b. Fiduciary Compliance Financial Management: According to the ICR (p. 28) the project complied with legal covenants throughout implementation. Interim financial reports were submitted in a timely manner, transactions were accounted for and documented adequately, accounting information was up-to-date and reliable and the external auditor’s opinions were unqualified. Also, financial management for Force Account works was maintained and effective. However, the Commitments Register was not updated on a regular basis and budget management was weak and required substantial support by the Bank. Procurement: According to the ICR (p. 27) the project complied with the procurement procedures as outlined in the Grant Agreements and Procurement Plan. The project experienced implementation delays due to poor bidder and consultant response given the country’s remote location, limited understanding of the supply market, and lack of capacity among local suppliers and contractors. These challenges resulted in unsuccessful Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) bidding processes and the need to use the Force Account mechanism for the replacement of the secondary water distribution network and the Buota rainwater harvesting system. The ICR (p. 28) stated that after the 2016 project restructuring, procurement improved due to extensive implementation support and capacity building by the Bank team, the synchronization of procurement plans with design/consultant engineers’ targets for civil works procurement, and the establishment of a longer-term contractual agreement for centralized procurement of Resilience Fund materials through an Indefinite Delivery Contract. Also, the PMU hired a Procurement Specialist for the large number of goods, works and services contracts (in contrast to large infrastructure projects with a limited number of sites and contracts). c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (p. 30 -31) provided lessons learned that were adapted by IEG: 1. Engaging local communities in project activities throughout the project cycle is essential for ensuring the sustainability of project outcomes. In this project, community engagement was critical for a successful implementation and subsequent use and maintenance of public infrastructure such as coastal protection measures and water supplies. Community engagement evolved from information seeking, to semi-commercial behavior change campaigns, to empowering residents and allowing for public voice in decision-making and proactively recognizing the needs and priorities of different groups. Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Adaptation Program - Phase III (P112615) 2. Understanding local administrative systems, local contracting and suppliers’ markets and taking advantage of local expertise are critical for successful implementation. In this project, an appropriate contract size and pre-bid training for local contractors allowed for the eventual completion of 13 of the 16 works contracts by local contractors using Force Account in two cases. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a good overview of project preparation and implementation. The ICR was internally consistent and mainly outcome driven. The ICR included an adequate Economic analysis and useful lessons learned. However, the ICR did not mention the risks and mitigation measures that were identified during appraisal. Overall, the Quality of the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 14 of 14