ICRR 14283 Report Number : ICRR14283 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/10/2014 Country : South Eastern Europe and Balkans Project ID : P084605 Appraisal Actual Project Name : Albania/montenegro US$M ): Project Costs (US$M): 19.76 ? Lake Skhoder Integrated Ecosystem Manage L/C Number : Loan/ Loan /Credit (US$M US$M): ): 4.55 4.00 Sector Board : Environment US$M ): Cofinancing (US$M): 5.9 0.00 Cofinanciers : Board Approval Date : 05/27/2008 Closing Date : 09/30/2012 12/31/2012 Sector (s): Central government administration (50%); General agriculture fishing and forestry sector (20%); Other industry (20%); Sewerage (10%) Theme (s): Water resource management (33% - P); Environmental policies and institutions (33% - P); Pollution management and environmental health (17% - S); Biodiversity (17% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : John Redwood Lauren Kelly Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: Project Development Objective: (from both PAD and Grant Agreement): to help establish and strengthen institutional mechanisms for transboundary cooperation through joint efforts to improve sustainable management of Lake Skadar-Shkoder. Global Environment Objective: (from PAD): to maintain and enhance the long -term economic value and environmental services of Lake Skadar -Shkoder and its natural resources . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: 1. Capacity Building for Improved Understanding and Joint Management of the Lake (appraisal cost: US$ 3.43 million; actual cost: US$ 1.92 million, or 56 percent of the appraisal estimate ). This component was intended to build capacity to establish and strengthen institutional cooperation to operationalize the Skadar -Shkodra Lake Management Commission (SLC) and its Secretariat, including through the establishment of four technical working groups to support priority activities for : (i) planning and legal management; (ii) monitoring and research; (iii) communications/ outreach and sustainable tourism; and (iv) water management ; and specific tasks including : (a) creation of a predictive hydrological model of the lake basin; (b) research and monitoring to better understand the impacts of changes in inflowing water quantity and quality; and (c) harmonizing monitoring on both sides of the lake through a publicly accessible joint database . 2. Promoting Sustainable Use of the Lake (appraisal cost: US$ 4.79 million; actual cost: US$ 14.64 million, including additional parallel financing, or 306 percent of the appraisal estimate ). This component was intended to promote adoption of sustainable approaches to economic development of the lake and its natural resources by focusing on tourism and fisheries where there is a high potential for economically significant sustainable use . It was expected to support environmentally and socially sustainable tourism by : (i) improving nature- and culture-based facilities and attractions; (ii) raising public awareness; and (iii) providing technical assistance to local residents considering tourism-based businesses. It was expected to support sustainable fisheries management by : (i) helping to develop lake-wide stock assessment and fisheries management plans; and (ii) integrating plan results and recommendations into national plans, regulations, and programs . 3. Catalyze Pollution Reduction Investments (appraisal cost: US$ 11.51 million; actual cost: US$ 36.46 million, including additional parallel financing, or 317 percent of the appraisal estimate ). This component was intended to : (i) support selected investments to stimulate pollution reduction activities; (ii) educate and encourage people to replicate demonstration projects of village -level wastewater treatment and buffer vegetation restoration; and (iii) provide Technical Assistance (TA) to catalyze remediation of the lake ’s largest scale industrial pollution “hotspot� (KAP, or the Kombinat Aluminijuma Podgotica, along the Moraca River in Montenegro . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project costs were estimated at US$ 19.76m. This included a US$4.55mGEF grant, 0.74m in borrower financing, US$ 5.9m labeled donor "co-financing"in the PAD, and US$8.57m from Local Sources of the Borrowing Countries (National Parks, privatization proceeds, local municipalities ). Actual costs were reported as US$ 53.02m, mainly owing to an increase in a cost category labeled "parallel finance" in the ICR. As opposed to the US$5.9m of co-finance anticipated from other donors, the project reports benefitting from US$ 28.68 of parallel finance from other donors active in the project area, including GIZ, REC, UNDP,USAID, NVA, KFW, and the Swiss . The originally anticipated US$8.57m from Local Sources of the Borrowing Countries was revised upwards to 19.9m, again, in "parallel finance." Meanwhile, the GEF grant of US$ 4.55m was only 88 percent disbursed, with an actual figure of US$ 4.0 million, when the project closed. The reason the GEF grant was not fully disbursed was that an investment to reduce point source pollution into Lake Skadar -Shkoder failed, and the associated resources were cancelled . The Borrower contribution was also reduced from an anticipated US$ 0.74m to US$0.44m. The ICR bases its achievements on the total actual reported costs of US$ 53.02m. Without more information as to what parts of the parallel finance contributed directly to this project's outcomes, it is not possible to estimate an accurate actual cost figure . For example, the TTL interview indicated that most of the "parallel financing" was for water supply and other non-project infrastructure investments in the project area . It is likely that Annex 1 of the ICR overstates actual project costs; this should be explored further if the project is selected for a field -based performance assessment. Dates : The project closing date was extended by three months, from September 30, 2012 to December 31, 2012 in order to permit completion of project activities and additional disbursement of GEF grant resources, as just US$ 3.43 million had been disbursed at the time the closing date was extended on September 27, 2012. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High . The project objective was and continue to be relevant for both The Relevance of the Objective is rated High. countries. The current CPS (2011-2014) for Montenegro calls for improving environmental management and reducing the cost of environmental problems, including pollution, while that for Albania seeks to help the country improve conservation and the management and efficient use of its water resources . The project was likewise designed in full consistency with the GEF's strategy for the management of international waters, and particularly its strategic objectives 2 -- to play a catalytic role in addressing transboundary water concerns by assisting countries to utilize the full range of necessary technical assistance and economic, financial, regulatory, and institutional reforms -- and 3 - balancing overuse and conflicting uses of water resources in transboundary surface and groundwater basins . The project also had a Global Environmental Objective (GEO), to which achievement of the PDO was expected to contribute, as successful and sustainable joint management of the lake and its natural resources would be an important precondition for maintaining their economic value and environmental services over the long term . Cross-country cooperation is particularly essential in this regard . b. Relevance of Design: Project design is rated Modest owing to the modest investment, lack of monitoring and reporting on the main aim of the project, which was to build capacity to establish and strengthen transboundary cooperation of a regional environmental good between two countries . Funding for capacity building activities was mainly concentrated in the development of the hydrological model and a joint database - on the development of a technical tool that by project end, still required sensitization, training, and uptake by skilled professionals in both countries . Project design recognizes that sustainable lake management will require attention to modeling and to agreeing on parameters of both water quality and quantity . But the project only monitors and reports on water quality . Water quantity is a far tougher transboundary issue to tackle but necessary for long -term sustainability. Data sharing, in addition to its production, and the facilitated dialogue around roles and responsibilities is also a necessary component - alongside the modeling. Project design was also challenged by a lack of clarity around project financing, the role of the GEF vis a vis other donor agencies, and the different allocations and expectations for the respective countries . 4. Achievement of Objectives (Efficacy): Efficacy is rated Substantial . The project substantially helped to establish and strengthen institutional mechanisms for transboundary cooperation through joint efforts to improve sustainable management of Lake Skadar -Shkoder. The project built and/or strengthened a number of key institutional and technical mechanisms for stakeholder cooperation. It supported the functioning of the Skadar -Shkoder Lake Management Commission and its Secretariat, as well as its four thematic Working Groups for Planning and Legal Affairs, Monitoring and Research . Just prior to project closing, Albania and Montenegro signed an agreement for continued joint funding of the SLC in 2013, but, as of the time the ICR was drafted, such arrangements had yet to be made for future years, which needs to be done on an annual basis. The project further strengthened the institutional mechanisms for transboundary cooperation by providing a number of public agencies and research institutions in both countries with hardware, software and training for the creation and use of a joint monitoring program and database for the lake . Bilateral agreements were signed several agencies in December 2012 for the maintenance, updating, and future use of these instruments for lake management. These agencies participated in the development of a lake -wide zoning and management plan. This plan fell short of being formally adopted during implementation however it is expected to be incorporated in spatial plan updates once a bilateral agreement is signed on the establishment of Lake Skadar -Shkoder Transboundary Park. Through study tours, the project supported knowledge transfer from other International Water Resource Commissions (Lake Constance, the Danube River, and the Sava River ). The project also financed a number of natural and cultural resources in the lake area which reportedly "produced valuable information for the joint management of the lake's protected areas and fisheries, and the promotion of cross -boundary tourism in the lake area. Specific lake functions were also improved that can be attributable to the project interventions . Fisheries management was improved. Targets for the improvement of fisheries management were largely achieved as the number of unlicensed fishermen and the frequency of illegal fishing methods in both countries decreased . An environmental impact assessment for the remediation of hazardous waste at the KAP aluminum plant site were also completed and a request for an IBRD loan to help finance the recommended interventions has now been approved . However, demonstration of a pilot wastewater treatment plant based on constructed wetlands failed due to the Government’s disapproval of the associated costs associated with plans for using buffer vegetation against lake siltation and runoff. Instead, an additional number of “storm water streams� were rehabilitated, but the ICR argues that “reforestation of the mountain slope and initiation of a community -driven program for cultivation of the lake shore buffer area would have more significant and longer lasting positive impact on existing stream canals .� Thus, there appears to have been a missed opportunity in this regard . In substantially achieving its PDO, the project also demonstrated progress toward achievement of the GEO . However, the objective of reducing immediate and longer -term threats to lake water quality and ecological services was only partially achieved and the intended establishment of the aforementioned Transboundary Park and associated adoption of a lake -wide zoning and management plan did not occur during project implementation, even though the documents were drafted and awaiting signature by the two Governments . 5. Efficiency: Efficiency is rated Modest owing to a lack of information on the costs and benefits associated with this project, or its value for money. While an ERR may not have been able to be calculated for the whole project, economic aspects of parts of the project such as fisheries management, wastewater management or pollution reduction could have been estimated. Instead, the GEF project uses an incremental cost assessment at appraisal, a methodology that does not measure efficiency but rather justifies the incremental costs of the provision of the global financing for the regional public good and at close, does not attempt to measure any of the economic aspects of the project . The ICR states that "economic benefits of the project would include enhanced sustainability in using natural resources and increased tourism and increased tourist visitation to the lake and its surroundings ." However, it did not attempt to measure these benefits. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Overall outcome is rated Moderately Satisfactory owing to a highly relevant objective and substantial achievements, noting that design was weakened by an under investment in capacity building and focus on the shared use and decision-making associated with the lake management modeling and data acquired though the project. Efficiency is rated modest due to a lack of calculation of the economic return of several of the investments, the costs and benefits of the project, or an analysis of the value for money . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: There are a number of serious potential risks to the development outcome, including the Government of Montenegro's decision to proceed with construction of hydropower dams on the Moraca River which could negatively affect the lake's hydrology and ecosystem . In addition, a couple of key institutional elements were not yet firmly in place at the time the project closed . On the one hand, while the two governments had made what the ICR describes as "modest funding" to keep the SLC and its Working Groups operational in 2013, the longer-term financial arrangements in this regard had not yet been formalized . Similarly, the bilateral agreement to establish the Lake Skadar-Shkoder Park, which the ICR (para. 66, pg. 20) characterizes as being "critically important for maintaining the long-term value of the lake ecosystem and enhancing it environmental services " -- i.e., achieving the GEO -- remained under review by the two governments, observing further that "until the agreement is signed by parties, the future of the Park remains uncertain ." Given the recent changes in administration in both Albania and Montenegro, but particularly in the former country, prospects for resolving these pending issues are even more at risk than at the time the ICR was drafted according to the project team in its subsequent meeting with IEG . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: Quality at Entry is rated Moderately Satisfactory . Quality at Entry was positively affected by a number of relevant projects and initiatives for joint management and sustainable use of transboundary water bodies, including the Baltic Sea, Caspian Sea, Lake Geneva, Lake Constance, and Danube River, as well as other projects in which the Bank and GEF were involved in both Albania and Montenegro that were supporting development and environmental management activities in the project region . A study tour was also undertaken during project preparation to the transboundary Lake Neusiedl -Ferto, which helped to shape the project's design and to understand how cooperation can work between countries with substantial differences . However, there were weaknesses in the results framework and monitoring arrangements (See M&E section) and despite the study tours, the project inadequately assessed the feasibility of promoting constructed wetlands, which was included as part of component 3, but which later had to be dropped as being too costly . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Quality of Supervision is rated Moderately Unsatisfactory . Bank supervision provided adequate support for the implementation of the Bank's fiduciary and environmental and social standards . There were good relations established with the two recipients, but the Bank team could have been more proactive in managing issues concerning the construction of the wastepaper treatment plant . Inadequate attention was also paid to the implementation of the planned socio -economic survey which failed to yield useful data for project monitoring and learning. In addition, the supervision team could have more strongly influenced creation and maintenance of the institutional capacity in the project implementing entities - as well as the quality of monitoring and evaluation . Finally, the supervision team appears to have provided insufficient instructions regarding the importance of tracking the provision of parallel financing by other donors . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Government Performance is rated Moderately Unsatisfactory . This regional transboundary GEF project was supported by the two governments of Albania and Montenegro . Borrower performance was positive during the project period, marked by the signing of four bilateral agreements on the joint financing of the Commission and Secretariat, as well as on the shared use of the lake information systems . The GoA and the GoM created an enabling environment for participatory conduct of background studies for the project and for transboundary technical cooperation throughout its implementation - both unprecedented in the past . However, the governments did not provide adequate resources or staffing for the project implementation unit or the Technical Service Unit of Montenegro and project co-financing was less than anticipated . The former factor led to shortcomings with regard to procurement and financial management . By project close, the governments also had not adopted the lake -wide zoning and management plan, and financing for the commission, secretariat and working groups was uncertain . Since project close, government ownership of the process has declined, mainly owing to a change in administration in Albania. Interviews with the Bank team revealed that Montenegro will need to take stock of its neighbor's new positions in order to continue the process . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: Implementing Agency Performance is rated Moderately Satisfactory . Implementing agencies in both Albania and Montenegro were cooperative and reportedly worked well together, which was highly important for technical support and management of multiple joint activities on both sides of the border . The project teams reported that the placement of the Albania Project Coordinator in the town of Shkodra, and the Director in Tirana, was an excellent managerial decision, given that the Commission works from Shkodra and this town is within the immediate lake area project. While delays caused by the lack of adequate financial management and procurement capacity within the project implementation units are attributed to the IAs in the ICR, this review finds that the ICR also expressed concern with the lack of staffing and resources made available by the borrowers, which seems to have been at the heart of these delays . Bank supervision could have also provided more technical assistance to support the IAs in this regard . Meanwhile, much of the data used to report results in this project (fisheries management, water quality etc .) was collected by the IA. caused slow start-up of the Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: M&E design is rated Substantial, since efforts were made at appraisal to set a baseline to assess change in the local understanding of sustainable tourism and natural resource management around the lake; to survey the number of unlicensed fishermen operating in the lake and the frequency of illegal fishing methods; and a plan was put in place to set a baseline for the chemical and physical indicators of lake water quality during the first year of project implementation. However, M&E design should have included metrics to assess transboundary cooperation (institutional capacity, cooperative behaviors, data sharing and joint decision -making, recurrent financing for the institutions and adequate technical capacity and political support etc .). The efficacy rating reflects substantial achievement over the project period against many of these suggested parameters i .e. the Lake Commission and its Secretariat were supported with finding, equipment, and training, even though future funding is uncertain . A lake-wide zoning and management plan was developed, although not yet adopted, signaling cooperation between agencies within the two countries . etc. Some minor differences in two of the intermediate outcome indicators in Annex 3 of the PAD caused confusion at the project completion stage because the Bank's ISR and the present ICR report used indicators as worded in the Results Framework (PAD, Annex 3, first table), while the Recipients picked different formulation of the same indicators from the Arrangements for Results Monitoring (PAD, Annex 3, second table). b. M&E Implementation: M&E implementation is rated Modest . It was rated moderately satisfactory in the ISRs because the above mentioned socio-economic survey was carried out later than scheduled and did not produce specific indicators of public attitudes to be used for measuring progress as recommended by the mid -term review, which occurred more or less as originally anticipated in November 2010. Nor was the follow-up survey scheduled for May 2012 undertaken. While two public surveys were carried out in Shkodra in 2011 and 2012 they were not adequate for measuring the project's contribution to local understanding of, and engagement in, sustainable tourism and natural resource management because [they] predominantly describe public awareness of the project rather than commitment to sustainable resource use . The project, did on the other hand, track changes in illegal fishing activity (the studies indicated a decrease in the numbers of unlicensed fishermen in both Albania and Montenegro, as well as a decline in the use of illegal fishing methods ) Lake water quality was also monitored, and slight improvement was detected but attribution of these monitoring outcomes to the project interventions is almost impossible . Unfortunately, moreover, monitoring of water quality near the specific project intervention sites "did not work out" because construction of the planned Vranjina wastewater treatment plant, which would have had the most positive impact on local water quality, did not occur, while the other investment that was expected to impact water quality, the rehabilitation of streams in Shiroka village, was only completed in the project's last year and, thus, their effect could not be monitored before project closing. c. M&E Utilization: M&E utilization is rated Modest . While there evidence of improved lake monitoring capacity in both Albania and Montenegro through the provision of laboratory equipment, training, and establishment of a joint database and a lake monitoring program, there is no evidence that this information is begin used to make environmental policy decisions, in a transboundary context, with regard to the sustainable management of the lake . The two public surveys conducted were not used in project decision -making since they failed to achieve their aim of measuring the project's contribution to the local understanding of, and engagement in, sustainable tourism and natural resource management. It should be noted however that all information on the state of the lake environment is now publicly available for use in planning and decision making by stakeholders . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project triggered the Environmental Assessment, Natural Habitats, Physical Cultural Resources, Involuntary Resettlement, and Projects on International Waterways safeguard policies and was classified in Category B . The Involuntary Resettlement policy was triggered to address possible social implications of reducing illegal fishing and imposing stronger regulations for sustaining fisheries of the lake . According to the ICR (para. 37), "there were no significant deviations or waivers from the Bank safeguard policies during the project life " and that safeguard compliance was monitored on a regular basis during supervision and "was always rated Satisfactory ." Bank environmental and social specialists participated in both the preparation /appraisal and supervision teams . b. Fiduciary Compliance: Other than initial delays in setting up project implementation units -- including the hiring of procurement specialists, which took two years in the case of Montenegro -- some quality issues regarding the quality of evaluation reports and contract negotiations, and the ultimate inability to procure works for the construction of a wastewater collection and treatment system and the administrative building for the National Park in Montenegro as originally planned, the project's performance with respect to procurement was generally satisfactory . ISR financial management ratings also varied between Satisfactory and Moderately Satisfactory, as there were some issues with respect to flow of funds, transactions documentation, eligible expenses (although these were reportedly "minor"), delays in processing payments, and financial reporting in Albania . According to the ICR (para. 39, pg. 12), by the last year of project implementation, however, all of these problems had been addressed and financial management was rated Satisfactory. c. Unintended Impacts (positive or negative): The project provided technical assistance, including for and EIA and preliminary designs, for remediation of a hazardous waste dump site at KAP, an aluminum plant located on the Moraca River which flows into the Lake, as part of component 3. These studies were expected to lead to preparation of an investment project for this purpose . By the time the present project was completed, the Government of Montenegro had formally approached the Bank with a request for a loan for an Industrial Waste Management and Clean -up Project, which would cover the KAP site among others. According to the ICR (para. 61, pg. 19), the project concept had been approved by the Bank and was expected to go to the Board later in FY 2014. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Given that the project achieved its Unsatisfactory Satisfactory stated PDO, as stated in the Grant Agreements, in terms of the establishment and strengthening of institutional mechanisms for transboundary cooperation -- even though its achievements with respect to its GEO regarding maintenance and enhancement of the long-term economic value an environmental services of the lake are far less certain -- and considering that the relevance of project objectives and design were both Substantial, while efficiency is rated Modest due mainly to insufficient information provided in the ICR, the overall outcome rating is rated Moderately Satisfactory. Risk to Development Significant High Risks to development effectiveness, Outcome : especially in relation to the GEO, have increased as the result of the changes in government in both countries over the past year, but particularly in Albania. Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR (contains a number of useful lessons, the most important of which for other operations going forward are: 1. When projects rely or are supported by various forms of finance - including direct co-finance and/or parallel finance from other donors supporting the project goal - it is important that projects clearly delineate the purpose and use of each of the funds, and properly track the use of these funds throughout the life of the project cycle . At project close, there should be a clear description of results achieved that are attributable to the different sources of funds. 2. In many of the Bank's client countries the governments and local communities have deficient information on the advantages and disadvantages of using constructed wetlands for waste water treatment, and lack trust in this technology . The conclusion is that including establishment of a constructed wetland into a project design needs to be approached very carefully . More specifically, "confidence in a true buy-in by the beneficiaries and stakeholders is critical, and so is the quality assurance of the feasibility studies and designs ." To this should be added, that the economic costs and benefits of such an approach also need to be carefully assessed together with its public acceptance. 14. Assessment Recommended? Yes No Why? In order to determine; (i) whether the GEO will be achieved or not, which will require revisiting the project and its results some years further down the road; (ii) to assess whether the significant risks to development effectiveness identified in the ICR have materialized or not and with what impacts on project outcomes; (iii) to clarify the role of parallel financing by other donors and local sources in relation to project activities and results; and (iv) to obtain a better picture than that provided by the ICR in relation to the efficiency of the use of the GEF grant and other project resources. 15. Comments on Quality of ICR: The ICR is well-written and provides a balanced account of project achievements and shortfalls . It could have been stronger in two ways, however: (i) its treatment of project efficiency which did not discuss the cost - effectiveness of use of project resources; and (ii) its failure to differentiate between co -financing, if any, of actual project activities and parallel financing of related, but non -project, investments in the project area from other donors and local sources, which together are estimated by the ICR itself to have been close to twelve times larger (i.e., nearly US$ 48 million) than the disbursed portion of the Bank -administered GEF grant, due in part to the inadequate tracking of such resources by the respective national implementing agencies . Nonetheless, this begs the question as to for what purposes this parallel financing was used, and with what results, and how does their use and effectiveness of these non-project resources compare with that of the GEF grant . a.Quality of ICR Rating : Satisfactory