Document of The World Bank Report No: ICR3687 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-46240) ON A LOAN IN THE AMOUNT OF XDR 23.6 MILLION (US$35 MILLION EQUIVALENT) TO THE FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA FOR A ETHIOPIA SUSTAINABLE TOURISM DEVELOPMENT PROJECT June 30, 2016 Trade and Competitiveness Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 10, 2016) Currency Unit = Ethiopian Birr (ETB) XDR 1.00 = US$1.411390 US$1.00 = ETB 21.729999 FISCAL YEAR 2016 ABBREVIATIONS AND ACRONYMS CHP Cultural Heritage Project CIP Community Intervention Project CPS Country Partnership Strategy ERR Economic Rate of Return ESMF Environmental and Social Management Framework ESTDP Ethiopia Sustainable Tourism Development Project ETO Ethiopian Tourism Organization ETSMS Ethiopian Tourism Statistics Management System GOE Government of Ethiopia GTP Growth and Transformation Plan ICA Implementation Completion Audit ICR Implementation Completion and Results Report ISR Implementation Status and Results Report M&E Monitoring and Evaluation MoCT Ministry of Culture and Tourism MTR Midterm Review NPV Net Present Value NTIS National Tourism Information System PAD Project Appraisal Document PAP Project-affected Person PCU Project Coordination Unit PDO Project Development Objective PSC Project Steering Committee RAP Resettlement Action Plan RPF Resettlement Policy Framework SME Small and Medium Enterprise SMU Site Management Unit TA Technical Assistance UNESCO United Nations Educational, Scientific, and Cultural Organization WTTC World Travel and Tourism Council ii Senior Global Practice Director: Anabel Gonzalez Practice Manager: Catherine Masinde Project Team Leaders: Hannah R. Messerli, Senidu Fanuel ICR Team Leader: Jade Ndiaye iii ETHIOPIA Ethiopia Sustainable Tourism Development Project CONTENTS Data Sheet A. Basic Information ................................................................................................................... v  B. Key Dates................................................................................................................................ v  C. Ratings Summary .................................................................................................................... v  D. Sector and Theme Codes ....................................................................................................... vi  E. Bank Staff .............................................................................................................................. vi  F. Results Framework Analysis ................................................................................................. vii  G. Ratings of Project Performance in ISRs ............................................................................... xii  H. Restructuring ........................................................................................................................ xii  I. Disbursement Profile ............................................................................................................. xiii 1.  Project Context, Development Objectives, and Design .......................................................... 1  2.  Key Factors Affecting Implementation and Outcomes .......................................................... 4  3.  Assessment of Outcomes ...................................................................................................... 11  4.  Assessment of Risk to Development Outcome ..................................................................... 21  5.  Assessment of Bank and Borrower Performance ................................................................. 21  6.  Lessons Learned.................................................................................................................... 24  7.  Comments on Issues Raised by Borrower/Implementing Agencies/Partners....................... 26 Annex 1. Project Costs and Financing .......................................................................................... 27  Annex 2. Outputs by Component.................................................................................................. 29  Annex 3. Economic and Financial Analysis ................................................................................. 40  Annex 4. Bank Lending and Implementation Support/Supervision Processes............................. 43  Annex 5. Stakeholder Workshop Report and Results ................................................................... 45  Annex 6. Summary of Borrower's Comments and ICR................................................................ 50  Annex 7. List of Supporting Documents ...................................................................................... 62  Annex 8. Overview of Lalibela Resettlement ............................................................................... 63  Annex 9. Detailed Assessment of Implementation Factors .......................................................... 67  MAP .............................................................................................................................................. 70  iv A. Basic Information Country: Ethiopia Project Name: Tourism Development Project ID: P098132 L/C/TF Number(s): IDA-46240 ICR Date: 06/30/2016 ICR Type: Core ICR FEDERAL DEMOCRATIC Lending Instrument: SIL Borrower: REPUBLIC OF ETHIOPIA Original Total XDR 23.60 million Disbursed Amount: XDR 23.5 million Commitment: Revised Amount: XDR 23.60 million Environmental Category: B Implementing Agencies: Ministry of Culture and Tourism Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 06/21/2007 Effectiveness: 12/31/2009 02/25/2010 03/15/2013, 03/15/2013, Appraisal: 05/06/2009 Restructuring(s): 06/27/2014 06/27/2014 Approval: 06/30/2009 Midterm Review: 10/22–11/02/2012 10/22–11/02/2012 Closing: 12/30/2014 12/30/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: v C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project at Quality at Entry No None any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision Yes None (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) General education sector 5 5 Other domestic and international trade 41 41 Other industry 45 45 Public administration - Industry and trade 9 9 Theme Code (as % of total Bank financing) Cultural Heritage 15 15 Education for the knowledge economy 6 6 Infrastructure services for private sector development 4 4 Other Private Sector Development 31 31 Rural non-farm income generation 44 44 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli K. Ezekwesili Country Director: Carolyn Turk Kenichi Ohashi Practice Manager/Manager: Catherine Masinde Gerardo M. Corrochano Hannah R. Messerli, Project Team Leaders: Francois Nankobogo Senidu Fanuel ICR Team Leader: Jade Ndiaye ICR Primary Author: Jade Ndiaye vi F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The proposed project development objective (PDO) was to contribute to the enhancement of the quality and variety of tourism products and services in targeted destinations so as to increase tourist visitation, foreign exchange earnings, and jobs. The implementation of the project was expected to yield the following key outcomes, particularly in key destinations: (a) an increase of annual international visitors; (b) increased average spending by visitors translating into higher foreign exchange earnings; and (c) an increased number of direct and indirect tourism-related jobs. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicators Original Target Values Actual Value Formally Revised Indicator Baseline Value (from approval Achieved at Target Values documents) Completion Original: Average percentage increase in the number of international tourist arrivals in Indicator 1: targeted destinations Revised: Average increase of the number of international visitors in the targeted destinations of Axum and Lalibela1 Value Axum: 19% Axum: 23% Axum: 61% (quantitative or 15% Lalibela: 27% Lalibela: 33% Lalibela: 24% qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Comments Partially achieved according to data on hotel occupancy. This indicator was revised during the (including % first level II restructuring in March 2013 to enhance attribution and to capture incremental Achievement) increases in the number of international tourist arrivals. Addis Ababa and its environs were not included in the revised targets because separate data were not available by purpose of visit. Original: Average expenditure by tourists in targeted destinations Indicator 2: Revised: Increase of average foreign currency spending per international tourist per day in all the three targeted destinations of Addis/Environs, Axum and Lalibela Value (quantitative or US$310 US$9632 US$350 US$362 qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 1 While the indicator was revised as: “average annual increase in the number of international visitors in the targeted destinations,” during the first restructuring it was meant to capture the increase in growth in tourism arrivals averaged out over the project period rather than annual (year-on-year) growth. For further discussion, see section 2.3 on monitoring and evaluation. 2 When the baseline was initially established it included flight expenditures. Flight expenditures were taken out of the calculation since some tourists travel by car and it is difficult to calculate for people who travel as part of packaged tours. vii Comments Target exceeded; 103 percent of the target has been achieved. Average expenditure per person (including % per night is calculated using the figures from each destination (Addis Ababa, Lalibela, and achievement) Axum). This indicator was revised during the first level II restructuring in March 2013. Original: Number of new tourism-related jobs created in targeted destinations Indicator 3: Revised: Number of jobs created in tourism-related industries in the targeted destinations of Axum and Lalibela Value Axum: 440 Axum: 580 Axum: 2,324 (quantitative or Lalibela: 650 Total: 400 Lalibela: 860 Lalibela: 3,229 qualitative) Total: 1,090 Total: 1,440 Total: 5,553 Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Comments Target exceeded; 385% of target has been met. This indicator reflects the number of new jobs (including % created in the four tourism-related industries of accommodation, catering, tourist transportation, achievement) and tourist guides, in line with the baseline categories. This indicator was revised during the first level II restructuring in March 2013 by defining tourism-related industries to improve attribution. (b) Intermediate Outcome Indicators Original Target Values Actual Value Formally Revised Indicator Baseline Value (from approval Achieved at Target Values documents) Completion Original: At least ten historic monuments rehabilitated (including church conservation) in key destinations (Lalibela, Axum, Addis Ababa) by the end of the project Indicator 1: Revised: Number of historic sites/monuments rehabilitated in all the three targeted destinations (Addis Ababa/environs, Axum and Lalibela) Value (quantitative 0 10 6 6 or qualitative) Date achieved 10/31/2012 07/01/2009 10/31/2012 12/30/2015 Target 100 percent met. The project rehabilitated six sites in total including the Addis Ababa Comments museum, Stelae field in Axum, the Archeological museum of Axum, Tsion Church in Axum, (including % museum in Melka Kunture, and archeological museum and ethnographic museum in Tiya. This achievement) indicator was revised during the first level II restructuring in March 2013. Original: A minimum of 12 km of secondary roads rehabilitated in Lalibela by the end of the project Indicator 2: Revised: Kilometers of access/town roads rehabilitated Value (quantitative 0 12 4 21.1 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Comments This target has been exceeded; 527 percent of the target has been achieved. This indicator was (including % revised during the first level II restructuring in March 2013. achievement) Original: Enhanced visitor centers operational in a minimum of three destinations Indicator 3: (Lalibela, Axum, and Addis Ababa) by the end of the project viii Revised: Number of visitor centers established and staffed in all three targeted destinations of Addis Ababa/environs, Axum, and Lalibela Value (quantitative 0 3 3 3 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Comments Target met. Visitor centers in Addis Ababa, Lalibela, and Axum are fully staffed. This indicator (including % was revised during the first level II restructuring in March 2013. achievement) Indicator 4 Number of trials with signage in targeted destinations (dropped): Value (quantitative 0 10 n.a. n.a. or qualitative) Date achieved 07/01/2009 07/01/2009 n.a. n.a. Comments This indicator was dropped during the midterm review in light of financial and project timeline (including % constraints. achievement) Original: At least three emerging destinations (three regions) have developed tourism development plans Indicator 5: Revised: Number of tourism product development plans prepared in secondary destinations Value (quantitative 0 3 3 3 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Target met. Product development plans for the southern, western, and eastern routes have been Comments prepared with extensive consultation. The preparation of these has enabled identification and (including % planning for the sector to continue to expand beyond the targeted sites in this project. This indicator achievement) was revised during the first level II restructuring in March 2013. Original: International standard promotional materials developed by the end of the project Indicator 6: Revised: Number of marketing and promotion strategies adopted with at least three international tourism market segments Value (quantitative 0 Developed 3 3 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 10/30/2015 Comments Target met. The marketing development strategy and the branding strategy have been finalized. (including % This indicator was revised during the first level II restructuring in March 2013. achievement) Indicator 7 Number of independent small and medium enterprise travel service providers using the e- (dropped): travel platform Value (quantitative 0 100 n.a. n.a. or qualitative) ix Date achieved 07/01/2009 07/01/2009 n.a. n.a. Comments This indicator was dropped during the first level II restructuring in March 2013, as it was assessed (including % that the sector was not yet developed enough to manage or sustain such a technical system. achievement) Original: Number of businesses in the tourism value chain that have successfully implemented service/product upgrading plans through the matching grant scheme Indicator 8: Revised: Number of beneficiary firms that utilized the matching grant financing Value (quantitative 0 60 50 79 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/30/2015 Target exceeded; 158 percent of the target has been achieved. The matching grant scheme Comments implementation has been well received with strong implementation in Lalibela, Axum, and Addis (including % Ababa and benefitting entities across the tourism value chain. This indicator was revised during achievement) the first level II restructuring in March 2013. Original: Number (percent) of artisans selling craft products to tourists in selected destinations (Lalibela, Axum and Addis Ababa) Indicator 9: Revised: Number of community-based and tourism-related business ventures established Value (quantitative 0 10 45 54 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 10/30/2015 Target exceeded. The Community Intervention Project (CIP) has met 120 percent of its target. Comments The CIP established 54 business ventures (20 from Lalibela, 26 from Axum, and 8 from Addis (including % Ababa and environs) as of end of October 2015. This intervention has enabled the communities achievement) in the three destinations to directly benefit from tourism industry activity. This indicator was revised during the first level II restructuring in March 2013. Original: Number of new norms and standards supporting tourism development adopted Indicator 10: Original: A strategy with a detailed 3–5 year action plan to support the implementation of a national tourism strategy developed Value Adopted and (quantitative 0 1 1 implemented or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Comments Target met. The national strategy was developed through extensive consultations and has guided (including % the Government (national, state, and local level) in tourism development and decision making. achievement) This indicator was revised during the first level II restructuring in March 2013. Original: Hospitality industry operational standards established and implemented Indicator 11: Revised: A lodging grading regime compatible to international standards developed and pilot tested Value Established and (quantitative 0 1 1 implemented or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 x Comments Target met. The regime has been developed, 52 grading experts have been trained, and 337 hotels (including % in 7 regions have been assessed/graded. This indicator was revised during the first level II achievement) restructuring in March 2013. Original: National Tourism Council and Tourism Board established Indicator 12: Revised: An upgraded national tourism statistics system that is capable of capturing basic tourism indicators Value (quantitative 0 Established 1 1 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Target met. The system is operational: it has been designed, users are trained, and software has Comments been migrated to the Government of Ethiopia’s systems. This is a crucial intervention benefitting (including % all areas in the country to better track and monitor tourism sector performance. This indicator was achievement) revised during the first level II restructuring in March 2013. Original: Tourism training and education upgraded at technical, vocational, and university levels Indicator 13: Revised: Number of instructions/trainers trained on training curricula developed under the project at vocational schools Value (quantitative 0 Improved and updated 40 44 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Target exceeded; 110 percent of target achieved. The project supported beneficiaries to partake in training in overseas programs that demonstrate good practices and international standards. Also, those trained were required to share the learned content with colleagues upon return to their training institutions. Number of people in the tourism industry trained in Addis Ababa/environs, Axum, and Indicator 14: Lalibela Value (quantitative 0 Improved and updated 200 1661 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Comments Target exceeded; 830 percent of target achieved. Training enabled improved understanding of the (including % sector and standards necessary for achieving a competitive product. Of those trained 983 are achievement) women. This indicator was added during the first level II restructuring in March 2013. Indicator 15: Direct project beneficiaries (added as core indicator) Value (quantitative 0 n.a. 500 3,837 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Target exceeded; 767 percent of the target has been achieved. An additional 26,738 individuals Comments have benefitted from improved access to water and sanitation in Lalibela that previously were (including % affected by water shortages. Additionally, 8,316 workers have been employed through temporary achievement) jobs (mainly in construction). This indicator was added during the first level II restructuring in March 2013. Indicator 16: Female beneficiaries (added as core indicator) (%) xi Value (quantitative 0 n.a. 100 1,704 or qualitative) Date achieved 10/31/2012 07/01/2009 03/15/2013 12/31/2015 Target exceeded; 1,704 percent of the target has been met. The Ethiopia Sustainable Tourism Development Project interventions have had positive impacts for women, who have directly Comments benefitted from grants, training, and in-kind support provided by the project. In addition to the (including % 1,704 direct beneficiaries listed above, women have benefitted from temporary jobs (especially in achievement) construction) provided by the project. This indicator was added during the first level II restructuring in March 2013. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (US$, millions) 1 12/22/2009 Satisfactory Satisfactory 0.00 2 06/30/2010 Satisfactory Satisfactory 4.08 3 03/26/2011 Moderately Unsatisfactory Moderately Unsatisfactory 4.12 4 12/29/2011 Moderately Unsatisfactory Moderately Unsatisfactory 4.48 5 07/07/2012 Moderately Unsatisfactory Moderately Unsatisfactory 5.53 6 10/01/2012 Moderately Unsatisfactory Moderately Satisfactory 6.06 7 05/29/2013 Moderately Unsatisfactory Moderately Satisfactory 9.65 8 09/11/2013 Moderately Unsatisfactory Moderately Satisfactory 10.68 9 12/07/2013 Moderately Unsatisfactory Moderately Satisfactory 13.41 10 02/03/2014 Moderately Satisfactory Moderately Satisfactory 15.03 11 09/02/2014 Moderately Satisfactory Moderately Satisfactory 20.76 12 01/28/2015 Moderately Satisfactory Moderately Unsatisfactory 22.53 13 07/30/2015 Moderately Satisfactory Moderately Unsatisfactory 29.76 14 12/21/2015 Moderately Satisfactory Moderately Satisfactory 32.32 H. Restructuring Amount Board Disbursed Reason for Approved at ISR Ratings at Restructuring Restructuring and PDO Restructur Restructuring Key Changes Made Change ing in US$, Date(s) millions DO IP Reallocation of resources in addition to adjustment of Moderately Moderately 03/15/2013 No 8.3 the PDO-level targets and Unsatisfactory Satisfactory intermediate results indicators xii Extension of the closing date from December 30, Moderately Moderately 2014, to December 30, 06/27/2014 No 20.9 Satisfactory Satisfactory 2015, and also reallocation of resources to under- budgeted activities. I. Disbursement Profile xiii 1. Project Context, Development Objectives, and Design 1.1 Context at Appraisal 1. Country and sector background. At appraisal, Ethiopia had (and still has) one of the fastest population growth rates in Africa, but its struggle with poverty and per capita income (at US$210 per capita) highlighted the urgent need for interventions to accelerate sustainable growth and create jobs. To lower unemployment and reduce poverty, economic diversification outside of agriculture was identified as crucial. Opportunities in the services arena (in particular through strategic and focused interventions in the tourism sector) were regarded as an unrecognized export activity with the ability to generate foreign exchange earnings and create jobs. Ethiopia’s wealth in cultural and natural tourism assets gave it strong potential as a tourism destination offering a rich array of sites that set it apart from its neighbors. At the time of appraisal, the United Nations Educational, Scientific, and Cultural Organization (UNESCO) recognized eight World Heritage Sites in Ethiopia. 2. Ethiopia faced a number of challenges in attempting to capture a larger share of the tourism market including low public investments, lack of high-quality services, a negative image, and well- preserved and well-presented sites. The key obstacles to tourism development in Ethiopia were identified as follows:  Infrastructure constraints. Poor infrastructure development, in particular, of roads, water and sanitation, telecommunications, and air transport limited the possibility of visiting cultural and natural assets at various destinations and also constrained private investments in hotels and lodges. Poor telecommunications and lack of modern information and communication technology hindered professionals’ and tourists’ ability to leverage technology in accessing information.  Land allocation. The cost and time of access were severe constraints with lack of clear city zoning policies, resulting in lodging developers with limited success in securing prime locations.  Term financing. Limited bank financing as a result of collateral requirements hampered the development of new and innovative products as well as reinvestment in existing assets. Moreover, lack of or limitation of modern payment systems constrained sector growth.  Tourism institutions. Limited capacity in both the public and private sectors contributed to lack of sector-specific skills and a mismatch between the wealth of cultural assets and the substandard quality of services. 3. In 2005, under the leadership of the prime minister, the Government articulated a new paradigm for tourism development, which set in motion the focus on tourism development for poverty reduction. In response, a new Ministry of Culture and Tourism (MoCT) and a state MoCT were created in 2006. 1 4. Rationale for Bank assistance. The World Bank was well placed to provide an integrated set of support services to address deficiencies in infrastructure, finance, land, and institutions drawing on international practice and experience in areas such as policy, cultural heritage protection and infrastructure improvement, linkages for private sector and community involvement, and skills enhancement. This was especially justified by the fact that other donors with whom the Bank coordinated throughout project preparation had limited involvement in the tourism sector. Moreover, related Bank involvement included (a) the Cultural Heritage Project (CHP), a US$5 million equivalent learning and innovation loan that focused on the conservation of World Heritage Sites in Gondar, Axum, and Harrar (closed in October 2008); (b) the Private Sector Capacity Building Project (FY06); and (c) the Urban Local Government Development Project (FY08). Each of these projects contributed additional momentum to the improvement of the tourism sector directly and indirectly. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 5. The proposed PDO was to contribute to the enhancement of the quality and variety of tourism products and services in targeted destinations so as to increase tourist visitation, foreign exchange earnings, and jobs. 6. The implementation of the project was expected to yield the following outcomes in key destinations: (a) an increase of annual international visitors; (b) increased average spending by visitors translating into higher foreign exchange earnings; and (c) an increased number of direct and indirect tourism-related jobs. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators and reasons/justification 7. The overall PDO was not revised during the course of project implementation; however, the formulation of some of the PDO indicators and targets for all PDO-level indicators were revised. Some intermediate indicators were revised, dropped, or added with the March 2013 restructuring (for details, see Results Framework Analysis, section F in the data sheet). 1.4 Main Beneficiaries 8. The Project Appraisal Document (PAD) did not specifically mention project beneficiaries; however, the primary target group of beneficiaries included a range of individuals/institutions in the public and private sectors such as (a) private sector establishments in the tourism industry in targeted destinations; (b) local and national tourism institutions (notably the MoCT and tourism bureaus); (c) local communities benefitting from improved infrastructure, access to services, and employment, in addition to in-kind and technical assistance (TA); and (d) domestic and international tourists. The primary target group remained consistent throughout project implementation. 1.5 Original Components (as approved) 9. The project comprised four components. 2 10. Component 1: Destination Development (US$15.43 million), which financed the following activities:  Subcomponent 1.1: Rehabilitation and enhancement of basic infrastructure (US$10.33 million) in historic sites, namely, Lalibela, Axum, and Addis Ababa  Subcomponent 1.2: Visitor services enhancement in selected destinations (US$4.0 million) focusing on upgrading of tourist/visitor facilities in Lalibela, Axum, and Addis Ababa and environs  Subcomponent 1.3: Tourism products development (US$1.1 million) seeking to diversify offerings in targeted destinations 11. Component 2: Market Development (US$7.84 million), which financed the following:  Subcomponent 2.1: Branding, positioning, and marketing (US$2.69 million) to enable Ethiopia to develop and implement strategies to attract specific target segments from key markets through collaterals, communication, and promotion  Subcomponent 2.2: Demand-driven linkages program (US$5.15 million) to enhance the competitiveness of the Ethiopian tourism private sector through a US$3 million matching grant scheme and to foster community involvement in the tourism sector to optimize benefits for the poor through a US$2.15 million Community Involvement Project (CIP) 12. Component 3: Institutional Development and Capacity Building (US$5.55 million), which financed the following:  Subcomponent 3.1: Institutional development (US$3.92 million) including policy implementation frameworks; a sector strategic plan; strengthening of the MoCT and regional tourism bureaus; establishment of standards and rating schemes for tourism enterprises; tourism data collection and statistics strengthening; institutionalization and support to the Tourism Council; and establishment of a Tourism Board to foster public-private partnerships in the area of tourism promotion  Subcomponent 3.2: Capacity building (US$1.63 million) including assessment of tourism education and training strategic needs, improvement of tourism institutions’ programs, skills enhancement at the MoCT, upgrading of guides, and the hospitality industry workforce to standard levels of professionalization 13. Component 4: Implementation Support (US$2.7 million), which financed the setup of a Project Coordination Unit (PCU) and Site Management Units (SMUs). 1.6 Revised Components 14. Not applicable. 3 1.7 Other Significant Changes 15. Partial suspension of Component 1. A partial suspension of project activities and disbursement of Bank funds for Component 1 became effective on November 16, 2010, due to the involuntary resettlement of the displaced households in Lalibela and resulting noncompliance with the Financial Agreement signed between the Government of Ethiopia (GoE) and the Bank. The partial suspension was lifted on June 29, 2012, based on compliance of the GoE with the relevant safeguard requirements. Further detail regarding the resettlement and safeguards issues are covered in section 2.4 and annex 8. 16. The project underwent two level II restructurings.  The first restructuring on March 15, 2013, following the midterm review (MTR) in October 2012, focused on improving attribution of PDO-level outcome indicators, better aligning project results with design of the project components, streamlining the indicators, and, accordingly, adjusting targets and resources. A summary of the budget realignment is provided by category in annex 1 part c.  The second restructuring on June 27, 2014, extended the closing date from December 30, 2014, to December 30, 2015, and also reallocated resources to under- budgeted activities given that the cost for some of the activities, especially in the works category, had significantly increased from project design. The restructuring also intended to streamline and focus the number of activities that would be initiated in the remaining period that would have a major impact on the project outcome. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry 17. Project preparation was informed by background analysis carried out in a 2006 diagnostic on tourism3 and extensive destination-level studies that were completed during the project preparation phase, in addition to lessons learned internationally and domestically from tourism development projects, including the CHP.4 Lessons from the CHP and previous tourism projects were appropriately integrated during preparation, most importantly the need to finance an integrated and comprehensive set of activities to lay the foundation for growth in the sector rather than financing isolated activities. Other lessons integrated during project preparation included the need for capacity strengthening for civil service, linkages, and community involvement for pro-poor growth; geographic focus on historically and culturally important cities; 3 The study “In Makeda’s Footsteps: Towards a Strategy for Pro-poor Tourism Development” estimated that the tourism sector generated approximately US$132 million of in-country expenditure (making it the third largest exchange earner behind coffee and oil seeds). However, as a tourism destination, Ethiopia was at an early stage of development, characterized by relatively short and expensive stays due to poor tourism infrastructure and a weak supply side. 4 The CHP was a US$5 million learning and innovation loan with the objective of testing out and developing, on a pilot basis, the means for more fully integrating the conservation and management of cultural heritage into local and national economic development. The general goal of the project was to contribute to poverty reduction through increased income generated by cultural assets and handicrafts development. 4 and the need for legal/regulatory strengthening. Furthermore, the links between the recommendations made by the 2006 tourism diagnostic and the proposed set of interventions highlighted the appropriate and adequate consideration of the constraints facing the tourism sector.5 Additional destination-level studies were carried out during the project preparation phase including: (a) Lalibela: Cultural Heritage and Social Safeguards Plan/Tourism and Heritage Management Plan; (b) Lalibela Water Supply and Sanitation Plan; (c) Axum: Integrated Destination Development Plan; (d) Destination Development Plan for Addis Ababa and its Environs; (e) Destination Development Plan for an Eastern Route in Ethiopia; and (f) Destination Development Plan for a Southern Route in Ethiopia. 18. The overall design of the project was comprehensive and highly relevant to the Country Assistance Strategy. All three of the project’s objectives—(a) increase in international visitation; (b) increase in average spending by visitors; and (c) increase in number of direct and indirect tourism jobs—were relevant to the Country Assistance Strategy goals at the time, when the focus was on employment, poverty reduction, and strengthening of the supply and responsiveness of Ethiopia’s services sectors. 19. While the overall design was informed by thorough background analysis and clear linkages to the Bank’s strategy, it was ambitious for the operating context in three aspects: (a) with a tight implementation timeline for complex civil works activities that were later on dropped or minimized in scope;6 (b) with a focus on at least one PDO-level result (tourist arrivals) that was arguably beyond the control of the operation; and (c) with the requirement for a high level of institutional and community coordination. Capacity risk was considered in the design (and rated Moderate), but coordination, timeline, and attribution risks were not specifically highlighted. One important factor that is critical to note is that the Ministry of Culture and Ministry of Tourism had newly merged into one entity a few years before project preparation, and, while the ministries benefitted from joint institutional leadership of the previous CHP, the Ethiopia Sustainable Tourism Development Project (ESTDP) was the ministry’s first major, large-scale lending operation. Even for an experienced ministry, the project design (particularly the destination development) would have posed implementation challenges, as it necessitated the facilitation of cooperation among a large number of stakeholders at the community, municipal, regional, and national levels. More robust mitigation measures to address the risks of weak coordination capacity could have been addressed through carefully considered design interventions.7 20. The project was classified as environmental/social safeguards Category B at the time of appraisal with the appropriate safeguards policies triggered, particularly in relation to the involuntary resettlement in Lalibela. The policies triggered included OP 4.01 - Environmental Assessment; OP 4.11 - Physical Cultural Resources; and OP 4.12 - Involuntary Resettlement. The project design appropriately identified the need for an Environmental and Social 5 For example, the recommendations made by the 2006 diagnostic to improve tourism market development in Ethiopia, provide better quality tourism products to improve the poverty impact of tourism, improve infrastructure, and the need for a tourism policy and strategy were directly addressed through the ESTDP design interventions. 6 For example, the designs for the project works were more complicated and more expensive than planned and some activities (the e-commerce platform, improvement of information centers at the airport, and expanded documentation and information displays) had to be entirely dropped from the project. 7 The PCU and SMU were included as part of project design to help mitigate capacity-related risks, but this was arguably not sufficient to address capacity constraints. 5 Management Framework (ESMF) and Resettlement Policy Framework (RPF), which were completed at appraisal. The resettlement in Lalibela was identified as a critical risk with several mitigation measures proposed including: (a) recruiting a qualified resettlement specialist to work out of the mayor’s office in Lalibela; (b) hiring a qualified environmental consultant (to be located in Addis Ababa) to coordinate the requisite environmental management activities; and (c) training in safeguards issues to relevant stakeholders. The RFP included a brief socioeconomic study that was undertaken as part of the RFP, providing information on the community in Lalibela and highlighting the main issues for resettlement. 2.2 Implementation 21. Due to the partial suspension and capacity constraints within the PCU and SMU, implementation of the ESTDP was initially characterized by delayed progress, followed by rapid acceleration of project activities as a result of the lifting of the project’s suspension and two (level II) restructurings that put the project back on track. Figure 1 provides an overview of the project’s timeline and key dates. Figure 1. ESTDP Project Timeline Revised Start, Lifting of the Restructuring Closing Date. February suspension, Paper, March December 2010 June 2012 2013 2015 Partial MTR, October Extension Suspension 2012 Closing Date Component I, Approval, November June 2014 2010 Source: ESTDP Terminal Evaluation. 22. Key implementation factors contributing to the achievement of the project’s outcomes included proactive identification and resolution of problems (particularly during the MTR and subsequent restructurings), close monitoring/supervision support, recruitment of new and competent staff during the project’s implementation, and governance arrangements at the destination level. Implementation challenges included the partial suspension from November 2010 to June 2012, high turnover of PCU and SMU staff, delays resulting from contract management and procurement, and escalating real prices contributing to a budget shortfall. The partial suspension, which was put in place for 19 months, was the project’s major challenge as it resulted in the shortening of the delivery time frame for a complex set of civil works activities at all sites. The partial suspension also diverted PCU/GoE attention away from the implementation of the other project components and overstretched the capacity of the PCU to focus on implementation. Annex 8 covers the resettlement in Lalibela and the events leading up to the suspension and annex 9 provides a more detailed assessment of the key factors affecting implementation. 2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization 23. The original M&E design for the project was based on an ambitious and extensive (16 in total) set of indicators. Regular data collection for project activities did not occur until the last two years of project implementation (after the MTR), but this did not significantly affect the 6 project’s M&E system given that many activities were implemented later in the project’s timeline. The late data collection was due to the partial suspension of the project and delays in the completion of the baseline study,8 which was concluded in October 2012. 24. Following the MTR and restructuring process, the Results Framework was significantly revised to incorporate baseline data and more realistic end-of-project targets. While the revised M&E design significantly improved the project’s overall M&E system, there were two shortcomings with respect to the formulation of the PDO and corresponding PDO-level indicators. First, the PDO included a reference to foreign exchange earnings, but there was no explicit indicator to capture this (although arrivals of mostly foreign tourists and their expenditures could have been considered a reasonable proxy). Second, there was an error in the revision of the annual increase target for arrivals in relation to the first PDO-level indicator. The PDO-level indicator on visitation was changed to “average annual increase of the number of international visitors in the targeted destinations of Axum and Lalibela,” with targets set at 23 and 33 percent growth for Axum and Lalibela, respectively. The indicator was meant to capture growth over the project duration rather than annual growth. This finding was confirmed by the task team and by the MTR Aide Memoire, which explicitly mentioned the intention to set modest end-of-project targets due to the long delays in project implementation. Average annual growth rates of 23 percent and 33 percent in Axum and Lalibela, respectively, would have resulted in doubling and tripling of the number of international visitors, which would have been an unrealistic target and inconsistent with the restructuring’s overall intent. 25. After the restructuring, collection of data and reporting generally improved although challenges with the methods used for data collection and communication of data persisted. To strengthen supervision of the M&E function and support the PCU to ensure appropriate data collection methods, the Bank engaged an M&E consultant in 2013. With the support of the Bank’s M&E consultant and supervision team, the PCU carried out several evaluations to assess the impact of the project’s activities,9 which, in turn, directly influenced the reliability and quality of the data provided for the project’s Implementation Status and Results Reports (ISRs) and Implementation Completion and Results Report (ICR). 26. Overall, M&E data were appropriately used to inform decision making and resource allocation during implementation, as evidenced through the MTR and restructuring processes. Of particular importance was the use of baseline data, which were updated from the baseline data collected during project preparation to develop more realistic targets. Furthermore, M&E utilization was strengthened through the project’s interventions, specifically the assessment and improvement of the National Tourism Information System (NTIS) (see section 3.5 on institutional change/strengthening), which the PCU used to report on targets. The NTIS 8 The October 2012 ISR reported that the process to secure a qualified firm to conduct the baseline survey was protracted. 9 Studies completed included (a) the Matching Grant and Community Involvement Evaluation to assess the impact of the CIP on the business performance of business ventures established through the intervention; (b) the Project Development Objective Results Status Survey to provide data used to track the progress of the PDO indicators in the three destinations (Addis Ababa and its environs, Lalibela, and Axum) for six consecutive years (2010–2015) in line with the project’s Results Framework; and (c) the terminal evaluation to measure the impact of the program and its activities. 7 significantly improved the sustainability of the M&E arrangements for the tourism sector and also consolidated tourism-related data in one single repository. 2.4 Safeguard and Fiduciary Compliance Compliance with Environmental and Social Safeguards 27. Social safeguards compliance specifically with respect to OP 4.12 - Involuntary Resettlement - initially proved to be a major challenge for project implementation, though these obstacles were overcome and the Resettlement Action Plan (RAP) was effectively carried out. Annex 8 provides a detailed assessment and timeline of the Lalibela resettlement, including the events leading up to the suspension and an overview of the resettlement process. The issues began shortly after the start of project implementation when the Bank team learned that 138 households had been displaced by the local municipality without an appropriate RAP, contrary to OP 4.12 and the provisions set out by the project’s legal agreements. This resulted in a suspension for over a year and a half. Following the implementation of the RAP, a total of approximately US$2.2 million (ETB 44,727,809.75) was paid to 307 project-affected persons (PAPs) entitled for compensation. More than 547 households were moved from the Lalibela core zone to the relocation site of Kurakur. During the implementation period, a total of 670 PAPs were addressed under the livelihood and income restoration scheme. Extensive infrastructure was developed in the resettlement area, which created a new neighborhood that did not exist before the resettlement, in line with the city’s plan for its expansion. An independent Implementation Completion Audit (ICA) for the RAP concluded that the resettlement was carried out in an effective and participatory manner in line with the RAP and Bank policies. The Lalibela resettlement was transformative both in terms of the preserving the Lalibela tourist attraction core zone (which would not have been protected and preserved without the resettlement initiatives) and providing livelihood and job opportunities to the communities impacted by the resettlement. Table 1 summarizes the social as well as environmental assessments carried out by the project. Table 1. Key Environmental and Social Management Assessments  RPF completed at appraisal  RAP report completed in September 2011 Social safeguards  An addendum to the Lalibela RAP concerning the Land Acquisition Social assessments Impact Screening and Livelihood Restoration Plan for Development Works in Tiya and Melka Kunture completed in December 2014  ICA for RAP completed in 2015  ESMF completed at appraisal  Environmental Impact Assessment for improvement of visitors walk routes, Environmental plazas, and stelae field in Axum completed in December 2011 safeguards assessments  Environmental Impact Assessment for the design and supervision of Lalibela and Environs Water Supply and Sanitation Project completed in 2012 28. Environmental safeguards compliance for activities carried out under the project (mostly in relation to the civil works and construction activities linked to Component 1) was problematic. The project used an ESMF to address environmental and social risks. Based on the ESMF, environmental and social impact assessments, including environmental and social management plans, were prepared for the subprojects. However, the Bank’s environmental assessment at project closure noted that both the preparation and implementation of the site- specific safeguard instruments (in particular the issuance of environmental clearance letters and 8 environment M&E reports) did not follow the procedures set for preparing and implementing safeguard instruments in the ESMF10. The main conclusions of the report included that: (a) all site- specific instruments were not approved by the competent environmental authorities of the four regional states (Tigray; Amhara; Oromia; and Southern Nations, Nationalities, and Peoples’ Region); (b) the safeguards training provided at the federal level was not cascaded to the local stakeholders concerned; (c) there was lack of follow-up on supervising the general implementation of the ESMF and preparation of the monitoring report on environmental performance of the project; (d) apart from the Axum site interventions, the environmental sustainability of some of the sites is questionable due to environmental (and/or social) issues that remain to be addressed.11 Furthermore, although the project was rated Category B, some of the subprojects (such as the sanitary landfill site constructed in the Lalibela project site) are expected to have significant adverse impacts (if they start to operate) and could likely be categorized as Category A. The project could have rejected the financing of the sanitary landfill subproject according to the guidelines and procedures provided in the ESMF. The project's performance in environmental safeguards is attributed to: (a) lack of recruitment of an environmental specialist (as stipulated in the ESMF and project implementation manual) to follow up on the implementation of the ESMF and; (b) lack of regular follow-up on the implementation of the ESMF by the Bank. Financial Management and Procurement 29. Financial management was effectively carried out throughout the project duration with no major issues affecting implementation. The budgeting, accounting, internal controls, and audit arrangements were generally acceptable, with only minor issues raised during implementation, which were later rectified in a timely matter. A marginal proportion of the project funding (US$101,63812) was undisbursed, and subsequently, canceled at project closure. 30. Compliance with Bank policy and procedural requirements for procurement and contract management proved to be a major challenge, though many of the procurement issues were resolved during project implementation. Delays and inefficiencies in procurement were the result of several factors, including (a) limited institutional recognition of the importance and significance of effective procurement management; (b) inadequate procurement staffing within the PCU; (c) inordinate delay of decision making during procurement processing for major contracts; (d) lack of institutional business delivery standards and limited capacity and knowledge of procurement practices and procedures; and (e) generic (institutional) weakness with regard to accountability for delayed actions. To address these concerns, several efforts were undertaken by the supervision team during the course of implementation, including providing a high level of technical support and guidance from Bank experts in procurement and specialized trainings. Furthermore, hands-on support was provided through a dedicated Bank-recruited procurement specialist who was embedded within the PCU to help expedite procurements and ensure timelines of deliverables and support contract management of works and consultancy assignments. While procurement issues were later resolved, contract management continued to be a challenge up until 10 Back to office report, Environmental Performance of Ethiopia’s Sustainable Tourism Development Project 11 The sites include the Tiya and the Melka Kunture sites and some of the infrastructures of the Lalibela site (such as the sanitary landfill site and the walkway from the Sandiago/Senbet Washa to the Church Ticket Office). 12 XDR 71,713 based on the exchange rate at project closure 9 the end of implementation due to understaffing of the role, insufficient supervision of contracts by the PCU, and limited proactive measures, 13 resulting in delayed completion of contracts and compromised quality of some of the works (particularly in Tiya and Melka Kunture). Dedication and extra effort by the Bank team helped the project achieve major planned contracts delivery and contributed to meeting the intermediate outcomes. 2.5 Postcompletion Operation/Next Phase 31. The appropriate financial, technical, and institutional provisions have been secured to enable the future operation of the project’s achievements with respect to the capacity- building interventions. The project has positively promoted institutional change/strengthening of the federal, regional, and local tourism bureaus, and these changes are likely to be sustained through the implementation of effective actions and policies to better manage Ethiopia’s tourism sector (discussed further in section 3.5). The financial sustainability of the various bodies established under the project (including the Ethiopian Tourism Organization (ETO), Tourism Transformation Council, and Tourism Board—also discussed in section 3.5) was aptly considered during the implementation of the project, and several options, including privatization, are under review to support their continued financial viability. 32. The project’s development works were provisionally handed over to the relevant local administrations at all regions and a management plan developed for each intervention (including required resources and an operating model). The local administrations will request a budget from the treasury for their operation and maintenance needs as part of their annual operation budget. Almost all of the facilities were in full operation at the time of the ICR mission, although staffing and management arrangements for the museums in Tiya, Melka Kunture, and Addis Ababa were at different stages of development, and it was unclear if the Oromia Cultural and Tourism Bureau had the available resources to equip and operationalize the facilities. Based on information provided by the PCU, the ETO is supporting the operationalization of the Melka Kunture site. The construction of the liquid and solid waste management site in Lalibela along with delivery of a vacuum skip and loading truck was completed, but the sites have not been in use due to the poor state of the road connecting the main town to the solid waste site.14 At the time of the latest environmental safeguards mission in May 2016, some of the toilets built at the project sites were not fully functional. 33. There are no follow-up or next phases of the project currently in Ethiopia’s lending pipeline with the Bank. However, the Government has expressed interest in continued engagement with the Bank in the tourism sector. The Growth and Transformation Plan (GTP) II targets US$6 billion in revenue from the tourism sector from an expected tripling of foreign visitors to 2.5 million by the year 2020.15 The MoCT plans to meet these ambitious targets through 13 The supervision team highlighted concerns related to procurement and contract management to the MoCT in a memo, which prompted the development of an action plan. While some administrative measures had been put in place, there was not much progress made in actual improvement of the contract administration of project interventions. 14 This was to be completed by the Lalibela Municipality but has not yet been done despite repeated requests from the supervision team to get the road to be passable. At the time of the ICR mission, the liquid and solid waste management site was not in use. 15 An estimated 790,000 international tourists visited Ethiopia in 2014, up from 383,000 visitors in 2008. 10 conservation and development, marketing of culture and tourism products, and enhanced cooperation and collaboration with development partners. 3. Assessment of Outcomes 34. The assessment of outcomes are performed on the basis of a split evaluation, since the key change in outcome and results-level indicators at the 2013 level II restructuring resulted in a material difference to the achievement against the PDO. The split evaluation assesses both the original and revised project objectives (and associated outcomes) then weights the disbursement as of the restructuring date (further detailed in Table 3). 3.1 Relevance of Objectives, Design, and Implementation Rating for Relevance of Objectives (against original PDO and associated indicators): High Rating for Relevance of Objectives (against revised PDO and associated indicators): High 35. Objectives: The PDO of the ESTDP—to increase the volume of tourism, foreign exchange earnings, and jobs—continues to remain relevant at the time of project closure as reflected in the GoE’s current priorities as well as the Bank’s Country Partnership Strategy (CPS, FY13–16). At the time of project design, the project objectives aligned with the GoE’s Plan for Accelerated and Sustained Development to End Poverty, 2005–2010, which included development of the tourism sector as a source of jobs, incomes, and economic growth for Ethiopians. Tourism continues to remain a highly relevant sector under the GTP I and II (covering 2010–2015 and 2016–2021, respectively) as well as the CPS FY13–16, which recognizes tourism as an important source of job growth, particularly for women and micro and small enterprises. More specifically, the first pillar of the CPS FY13–16 focuses on fostering competitiveness and employment through increased productivity in manufacturing and services as well as access of small and medium enterprises (SMEs) to the financial sector. The ESTDP made a direct contribution to this overall goal by providing both new employment and financing opportunities to SMEs in the tourism sector. Furthermore, GTP I highlights the importance of tourism as a cross- cutting sector that contributes to economic growth and progress toward achieving middle-income status. It includes ambitious targets for growth, including doubling of tourist arrivals and a twelve- fold increase in tourism expenditures. The targets set under GTP I—including the development and promotion of cultural heritage, contribution of earnings from the tourism sector to middle- income status, accessibility of tourism attractions, and contribution of tourism to foreign exchange earnings—link directly to the ESTDP PDO-level and results indicators focused on jobs, tourist spending, destination development, and variety and quality of tourism products. The targets set by the GTP II – including tripling of foreign visitors and increase in tourism revenue to over $3 billion by July 2016 – further reinforce the relevance of the tourism sector to Ethiopia’s overall economic development. Rating for Relevance of Design (against original PDO and associated indicators): Modest Rating for Relevance of Design (against revised PDO and associated indicators): Substantial 36. The overall design of the ESTDP was broadly aligned to the project’s objectives. The original and revised project design adequately considered the need for an integrated set of interventions to address the binding constraints in the tourism sector (including low investment, 11 lack of access to finance, and weak capacity of institutions), and this was reflected in a suitable mix of activities to directly meet the sector’s needs. As shown in the project’s causal chain (annex 2, table 2.1), the list of outputs made a direct contribution to enhancing the quality and variety of tourism products and to increasing foreign exchange earnings and jobs. One shortfall is noted with respect to initial consideration on establishing attribution of expenditure impacts. Tourism projects are generally problematic in that it can be challenging to identify and attribute economic effects directly to tourism development activities, particularly for outcomes that may be influenced by external factors. The causal relationship between activities/outputs and outcomes (in particular for international visitation)—though plausible—was difficult to unambiguously establish due to a number of external factors that could have reasonably influenced project outcomes. The MTR addressed many of these issues by revising the PDO-level targets to make them more attributable; however, the outcome on visitation remained part of the overall PDO. Rating of Relevance of Implementation (against original PDO and associated outcome targets): Substantial Rating of Relevance of Implementation (against revised PDO and associated outcome targets): Substantial 37. The Bank’s implementation assistance was responsive to changing needs to achieve the PDOs and remained flexible throughout project execution. The MTR effectively identified different challenges relating to the performance of core components under the project, including delays in start-up, lack of technical expertise, a low level of involvement by key stakeholders, high price quotations by contractors over the budgeted amount, and turnover of key project staff, among other issues. Despite the delayed implementation process that hampered the first three years of the project, these difficulties were overcome in 2013 and 2014 and were followed by major efforts to realign project resources to achieve stated objectives. 3.2 Achievement of Project Development Objectives Rating of Efficacy (against original PDO and associated outcome targets): Modest Rating of Efficacy (against revised PDO and associated outcome targets): Substantial 38. The PDO was to enhance the quality and variety of tourism products and services in targeted destinations so as to increase the volume of tourism, foreign exchange earnings, and jobs. Accordingly, the implementation of the project focused on the following key elements: (i) an increase of annual international visitors; (ii) increased average spending by visitors (translating into higher foreign exchange earnings); (iii) an increased number of direct and indirect tourism- related jobs. The project substantially achieved the PDO-level outcomes and met or exceeded all of the revised intermediate indicators. Annex 2 provides a breakdown of the project outputs by component as well as a detailed analysis of the key outcome indicators, including an assessment of attribution and ratings. Building on the analysis of the project’s level of achievement of the outcome indicators and outputs, the following section assesses the project’s contribution to the elements of the PDO. The achievement of the first part of the PDO “enhancing the quality and variety of tourism products and services” is measured through the fulfillment of the project’s intermediate outputs (discussed at length in annex 2). Enhancing the Quality and Variety of Tourism Products and Services 12 39. Quality of tourism products and services. The Destination Development component focused on improvement in the quality of tourism products and services through the rehabilitation of six historic sites/monuments in the targeted destinations of Lalibela, Axum, and Addis Ababa and its environs; completion of three visitor centers; and improved access to infrastructure (roads, water, and sanitation). These interventions significantly improved and transformed the quality of the visitor experience in the targeted destinations by facilitating access to the sites, enhancing their presentation and providing valuable information to tourists. Sites that were previously poorly presented and lacked pathways/landscaping and signage now have well-delimited tourist spaces that are accessible and informative and contribute to the overall positive image and preservation of the attractions. Furthermore, facilities and services including visitor centers, exhibition spaces, cafes, and shops have contributed to an enhanced visitor experience in line with internationally recognized destinations. As discussed further in annex 5 (table 5.1), overall tourist satisfaction is high for the improvements in the Axum stelae field and landscaping/improvement of the heritage sites in Lalibela. The number of improvements made in Axum and Lalibela (in particular in Lalibela where an entire community was resettled) have resulted in a major transformation of both towns. Without the project’s investment in infrastructure, facilities, products, and services, it is unlikely that this transformation would have occurred, in particular for Lalibela where overcrowding and lack of sanitation posed a risk to the physical structure of the sites and the tourism industry as a whole (see annex 8 on the Lalibela Resettlement for further discussion). 40. Variety of tourism products and services. The product and market development components of the ESTDP contributed to the diversification of tourism products and services offered in Ethiopia through TA and direct financing. The matching grant and CIP interventions (totaling approximately US$3.0 million in disbursement) specifically targeted the strengthening of tourism products/services and resulted in better quality and expanded offerings. Examples of matching grant projects that directly contributed to improved variety of tourism products/services included new tour packages offered by tour operators, improvement of variety in the menus at restaurants, improved entertainment offerings for tourist establishments, food products for hotels/restaurants, and diversified craft goods for the tourism market. In a survey administered to the matching grant beneficiaries, around half of the 54 firms that participated in the survey confirmed that their investment resulted in improvement in their product/service diversification and a similar proportion indicated their investment resulted in an enhancement of the quality of their products/services. 16 The CIP also specifically contributed to product diversification, especially in the handicraft, agribusiness, and small catering/coffee subsectors, where in-kind assistance and training resulted in the introduction of new products and services. 41. Building from a base of improved quality and variety of tourism products and services, the ESTDP set out to achieve outcomes on arrivals, exchange earnings, and jobs, most of which were achieved and/or exceeded. Table 2 provides a summary of the PDO-level outcomes and ratings, as discussed further in annex 2. 16 This is taken from the Evaluation Report on Business Performance of Matching Grant Scheme in which a survey of beneficiary firms was undertaken reporting on the level of product/service diversification and quality of products/services. ‘Good’ improvement was defined as 51 percent–70 percent of the agreed business plan. 13 Table 2. Summary of PDO-level Outcomes, Indicators and Ratings PDO-level Indicator PDO Outcome Level of Achievement Rating (Revised) In Axum, the average growth rate of international visitation (from 2012–2015) of Average increase in the 61.4 percent is well above the targeted number of international Increased volume of average growth rate of 23 percent. In visitors in the targeted Substantial tourism Lalibela, the average annual growth rate of destinations of Axum almost 24 percent is below the expectations and Lalibela of 33 percent average growth, but arrivals overall have increased. Increase in average of foreign currency spending per international tourist per day in all the three The total average expenditure of US$361.86 Increased foreign targeted destinations of is above the target figure of US$350 from a Substantial exchange earnings Addis/Environs, Axum baseline of US$310. and Lalibela (translating into higher foreign exchange earnings) Number of jobs created The total number of jobs (permanent and in tourism-related temporary) in the tourism sector in the target Increased tourism- industries in the destinations reached 5,553, well above the High related jobs targeted destinations of target figure of 1,440, starting from a Axum and Lalibela baseline of 1,090. 42. Increased volume of tourism. Overall growth in tourism in the targeted destinations increased over the project period, although growth in Lalibela was lower than expected. Average international tourism growth from 2012–2015 in Axum of 61 percent exceeded the targeted average growth rate of 23 percent, whereas average growth of almost 24 percent in Lalibela was lower than the projected growth rate of 33 percent (although international arrivals increased overall by over 20,000 tourists). 17 Arrivals growth in Lalibela is expected to be achieved once international tourists learn of new and improved sites and accommodations and suppliers offer new products and services to meet demand. It is important to note that there was significant growth in the number of hotels in both destinations: in 2010 there were 9 small hotels in Axum and 8 in Lalibela; by 2015 these had increased to 43 hotels in Axum and 55 in Lalibela. Moreover, hotel occupancy rates increased over the project period for both destinations as detailed further in annex 2. Attribution of arrivals to both destinations is established through the efforts made to transform both towns into modern tourism destinations which, in turn, provided the foundation to attract tourism. The accelerated growth in arrivals and hotels in 2014 (see figure 2.1 and table 2.3) which coincides with disbursement for the project at 86 percent illustrates the positive impact of the project on arrivals growth. It is unlikely that the increase in the volume of tourism would have been achieved without the project interventions, given that the lack of tourism infrastructure and service availability had the effect of deterring both independent travelers as well as groups led by tour operators. 17 The calculation for this indicator captures average growth over the project duration rather than annual (year-on- year) growth (see section 2.3 on M&E for further discussion). 14 43. Increased foreign exchange earnings. The project’s activities contributed to an overall increase in average spending by visitors to US$361 per day, though the impact on foreign exchange earnings would need to consider the total number of arrivals to determine the total consumption expenditure by foreign visitors in the economy. Given that tourism overall has increased and total average spending has increased, it is reasonable to conclude that the project has had an overall positive impact on foreign exchange. Increased expenditures by tourists can be directly attributed to project activities that have enabled the growth of tourism-related businesses, products, and services in each of the destinations and provided the capital, skills, and TA to capture opportunities in tourism, as discussed further in annex 2. 44. Increased tourism-related jobs. The total number of tourism-related jobs created (permanent and temporary) reached 5,553, well above the target of 1,440. An estimated 1,608 permanent jobs are attributable directly to the matching grant and CIP interventions alone, which helped generate new investment and formal sector employment and another 632 jobs are directly attributable to the livelihood restoration scheme, implemented as part of the project’s social safeguards requirements. The 457 permanent jobs linked to the matching grant intervention were the result of new jobs added by firms in the year(s) following implementation of their projects (these were tracked on a bimonthly basis). The remaining jobs generated through the CIP are the result of 54 new formally registered business ventures. Further analysis of attribution is provided in annex 2. 45. The rating for overall efficacy is Substantial, in light of the above contributions of the project to the PDO, supported by the High rating for the attainment of the results of increased tourism related jobs and the Substantial ratings for the attainment of increased tourist arrivals and foreign exchange earnings. The Modest rating of efficacy against the original PDO and associated outcome targets is supported by Modest achievement toward the outcome targets, in particular for international tourist arrivals and average expenditures, which were likely not to be achieved by project closure. Given the excellent achievements of the project on the level of the outputs and intermediate results for the revised outcomes, it can be argued that the project has put in place a significant amount of groundwork for the tourism sector to flourish. The accomplishments of the project are not only linked to the direct effects of arrivals, expenditures and jobs, but also to longer- term impacts that contribute to the capacity of the sector to drive linkages, foster indirect employment and entrepreneurship and catalyze investment. The dramatic growth in hotels in both destinations from 2010-2015 is a testament to the project’s ability to catalyze investment and the overall impact of this growth is seen as providing an enhanced role for both the private sector and communities in the tourism sector. According to the World Travel and Tourism Council (WTTC), overall tourism investment in Ethiopia is expected to have attracted capital investment of ETB14,054 million in 2015 and is expected to rise by 3.7 percent in 2016 and rise by an additional 5.4 percent over the next ten years18. 3.3 Efficiency Rating of Efficiency (against original PDO and associated outcome targets): Modest Rating of Efficiency (against revised PDO and associated outcome targets): Modest 18 World Travel and Tourism Council, Travel and Tourism Economic Impact 2016 15 46. The efficiency of the project is assessed as “Modest”, based on the rate of return calculated for the project and resource utilization comparing the planned expenditures with actual disbursements. The economic rate of return (ERR) calculated for the project at 37 percent falls somewhat below the expected return calculated at appraisal of 43 percent as detailed further in annex 3. The lowered net benefits resulting from the project activities are largely the result of the revised projections for tourist arrivals19 during the project period (and also estimated beyond the project period), though this has been offset somewhat by the increase in average spending from tourist arrivals. At the time of appraisal, sensitivity analysis was undertaken to account for risks associated with weak implementation capacity, specifically in a ‘slow disbursement’ schedule that considered an additional year for implementation and a ‘conservative’ scenario that considered a lower growth rate of tourist arrivals. Both the ‘slow disbursement’ and ‘conservative’ scenarios, were more realistic projections in line with the ERR calculated at project closure. With regard to resource utilization, the historical disbursed funds for the project totaled US$34.7 million compared to US$35 million calculated at appraisal, resulting in savings, although some activities initially envisaged as part of the project were dropped (activities related to visitor services enhancement, the e-travel platform, signage in selected destinations, and some of the civil works activities). The operating costs totaling US$3.3 million (see section (c) of annex 1) are slightly higher than expectations following the restructuring, although the costs are more than double of what was expected at appraisal. With regard to the timeline, project implementation lasted a year longer than initially envisioned due to the suspension; however, the civil works activities were completed in four and a half years as opposed to the five years allotted during the design phase. These activities, which accounted for more than half of the budget, were implemented more efficiently than initially envisaged, although the cost on the works exceeded both the initially envisaged budget at appraisal and the budgets calculated during the project’s two restructurings (by over US$ 3 million, despite having dropped some civil works activities from the project). The rating for efficiency for the original PDO and associated outcome targets is Modest based on limited progress toward the achievement of project objectives relative to the cost incurred during the initial phase of the project’s operation. Furthermore, many of the estimated costs were underestimated for the expected level of achievement of the outcome targets. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 47. The overall outcome rating of the project is assessed to be Moderately Satisfactory. Table 3 presents the results of the split evaluation, including the weighted rating. The Moderately Unsatisfactory rating against the original PDO is justified by a Substantial rating for relevance, Modest efficacy and Modest efficiency. The Moderately Satisfactory rating against the revised PDO and associated outcome targets is justified by a High rating for relevance, Substantial efficacy and Modest efficiency. The restructuring of the project and revision of targets early-on resulted in a more relevant design and improved performance of the project against the revised PDO and associated outcomes. 19 Initial projections on arrivals at appraisal included over 300,000 international arrivals starting from 2010. 16 Table 3. Split Evaluation of Project Outcomes Against Original PDO Against Revised PDO Overall Indicators Indicators Moderately Rating Moderately Satisfactory – Unsatisfactory Rating value 3 4 – Weight (% disbursed 26 74 – before/after PDO change) Weighted value 0.78 2.96 3.74 Final rating (rounded) – Moderately Satisfactory – 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 48. The ESTDP has had positive impacts on poverty reduction and the population mainly through the jobs, business ventures, skills, and access to basic services provided. With regard to the poverty impact, the project has helped create enterprises and new jobs in the tourism sector that have had a direct impact on the lives of the poor through wages and skills development. The CIP, in particular, targeted poor and vulnerable members of the communities where the ESTDP was implemented and enabled the establishment of business ventures, which helped support the sustainability of the jobs created. Poverty Impact through Creation of New Businesses, Jobs, and Skills Development  During the period from 2009 to 2015, 5,553 new tourism-related jobs were created in the targeted destinations of Addis Ababa, Lalibela, and Axum in tourism-related industries including accommodation, catering, tourist transportation, and tourist guides.  The CIP provided training and in-kind support to 1,153 beneficiaries. In total 54 new business ventures were created in the handicrafts, agriculture, small catering/coffee, and local tour guides/ecotourism businesses in the project target destinations.  A total of 8,316 people were employed during the construction and rehabilitation of access roads and project sites. These temporary jobs were especially important for poor people and also helped develop transferable skills.  During the resettlement period, a total of 670 PAPs were included under the livelihood and income restoration scheme. A total of 510 people were organized either in cooperatives or as individual owners. Among this group, 160 PAPs identified as vulnerable persons received special support through the contracted nongovernmental organization, Save Your Holy Land Association.  Young people have also been clear beneficiaries of the different programs with regard to training and jobs. Of new jobs created, youths accounted for 42 percent in Axum and 27 percent in Lalibela. 17 Impact on Women 49. The ESTDP interventions also had positive impacts for women, who benefitted from the grants, training, and in-kind support.  The project beneficiaries included a total of 1,704 women (44 percent of the total number of direct project beneficiaries).  232 new jobs were created for women  983 women were trained through the matching grant and other training initiatives  401 women were included under the livelihood and income restoration scheme  313 women were beneficiaries of the community involvement intervention Social Development 50. The project has had a positive impact on improving access to basic services, with a total of 26,738 people having benefited from improved access to water and sanitation in Lalibela and surrounding communities:  Potable water. In Lalibela, the data provided by the SMU office reflects that the quantity of potable water has been increased by 22 l/s as a result of four main activities: pipeline installation, fixing submersible pumps, connecting the system to the main water tank for the town, and upgrading the pumping capacity by installing additional pumps.  Sanitation. Sanitation actions have improved the potential for existing sanitation in the town, including construction of 20 public toilets in selected attraction sites and construction of a water reservoir to make these toilets sustainable, along with installation of additional pumps, which are situated so that they may be used by locals and tourists. The construction of liquid and solid waste management sites and purchase of a vacuum and skip loading truck to transport liquid and solid waste are also expected to contribute to improved sanitation once the main road connecting the sanitation site to the Lalibela town is completed. (b) Institutional Change/Strengthening 51. Several capacity-building interventions were implemented to promote institutional change/strengthening of the federal, regional, and local tourism administrations. Moreover, the project also supported the formation of several new strategic policy-oriented bodies, including the Tourism Transformation Council, the Tourism Board, and the ETO. The following are identified as interventions that have had a positive impact on the longer-term development of the country’s capacity and institutions:  Assessment and improvement of the NTIS. The goal of this activity was to improve Ethiopian tourism statistics management through an information system designed to 18 gather, process, and disseminate tourism data. A system containing data on international and domestic tourists was implemented in 2014. This was the first comprehensive attempt to gather reliable tourism data, resulting in the first statistical tourism data of its kind. Training was also provided, in Addis Ababa, on how to use and run the system for the persons in charge of the Ethiopian Tourism Statistics Management System (ETSMS) Platform. The overall impact of the information system is substantial, given that there is now information on sector performance that can be used by the GoE for improving the sector, informing policy, and attracting further private sector investment.  Preparation of the National Tourism Policy Implementation Framework. A tourism policy was in place at the time of the start of the project. However, it was never effectively applied due to the lack of a practical plan (including specific actions and steps), together with stakeholder capacity-building measures to foster coordination. The plan that was developed by the project outlined the institutional arrangements and enforcement mechanisms, budgets, funding, and other requirements for the achievement of the different policy goals that helped operationalize the tourism policy.  The creation of the Tourism Transformation Council, the Tourism Board, and the ETO. The ETO is expected to play a pivotal role in building the capacity of tour operators, hotels, travel agencies, and stakeholders to fully promote and exploit the country’s tourism resources and thereby generate worthwhile income.  Implementation of lodging standards scheme. This intervention involved defining a grading/standard system for hotels in Ethiopia in line with the United Nations World Tourism Organization’s globally accepted framework, training assessors to perform the grading in their respective region, and engaging private sector operators on how to participate/benefit from the standards scheme. The scheme contributed to the formation of internationally compatible standards and ratings in Ethiopia, which, in turn, provided greater awareness of needed improvements for operators. The implementation of the scheme also contributed to improved capacity to design and implement a hotel grading system, thereby laying the foundation for Ethiopia to compete with other tourist attractions over the long term. (c) Other Unintended Outcomes and Impacts (positive or negative) 52. During the course of implementation, there was growth in positive publicity, which helped shift the image of Ethiopia from poor and destitute to a destination with natural beauty and an ancient culture. For example, in 2015, Ethiopia was declared the World’s Best Tourist Destination in 2015 by the European Council on Tourism and Trade, which is the highest honor to be awarded for a country for achievements in tourism. The award is generally offered to countries that comply with the United Nations Tourism Division, UNESCO, and European Council on Tourism and Trade principles on fair tourism, ethical tourism and safety standards, and historic preservation of cultural sites. Furthermore, Ethiopia was named among the Lonely Planet’s 10 best-value travel destinations for 2014, due to the great-value and time-saving flights along the 19 country’s historic route. The ‘buzz’ generated by the positive publicity helped garner more interest and curiosity overall in Ethiopia as a travel destination. 53. Negative unintended consequences of the project came to light during the discussions with beneficiaries, in particular regarding the speculative inflation during the resettlement and increased competition among tourism establishments. Based on discussions with some members of the resettlement community, there was an alleged rise in the cost of building materials as a result of the compensation payments made to some of the community members in Lalibela. Some beneficiaries complained that, as a result, they could not afford the cost of the necessary building materials with the compensation received, although this was not a finding of the ICA for the RAP and the property valuation fees were adjusted according to the price index at the time of compensation payment. Moreover, increased competition among tourism establishments resulted in some disgruntled beneficiaries in Axum. In particular, shop owners/shop keepers complained that training and capacity-building interventions built into the CIP increased competition and hurt their businesses (see annex 5 for further details). 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 54. Focus group and beneficiary discussions were held in Ethiopia in November 2015 and March 2016 during the ESTDP Terminal Evaluation and ICR missions to gain an understanding of the perceptions, views, and opinions of ESTDP-related interventions. The groups consulted included souvenir shop owners/shopkeepers (in Axum and Lalibela), resettled households, CIP beneficiaries, local and national tourism sector institutions in addition to tour guides and transport service providers. A detailed discussion of the key issues raised is included in annex 5. Some of the general feedback summarized from the various beneficiaries is as follows:  The project has been successful overall in meeting its objectives, notably on employment where visible opportunities for both women and men in Lalibela and Axum were brought about through the construction of roads, hotels, cafes, and the manufacturing of souvenir products.  Beneficiaries observed visible improvements in Lalibela and Axum as a result of improved sanitation, drainages, and cobblestone roads, which were areas of concern before the project. Beneficiaries further commented on the project’s overall contribution to creating a more modern international tourist destination.  Overall beneficiaries expressed their support for tourism-related investment due to the tourism sector’s contribution to livelihoods and lack of alternative opportunities in some of the designated sites.  Feedback from the resettled households was mixed. While interviewees from the resettlement group confirmed that the benefits from resettlement left them in an overall better position, the compensation granted from the resettlement was considered by some interviewees to be too small.  Among the CIP beneficiaries in Axum, participants reiterated the need for continued support and also noted that they were asked by the municipality to hand over materials 20 and equipment provided by the ESTDP at project closure (though this feedback has been contested)20. 4. Assessment of Risk to Development Outcome Rating: Moderate 55. The overall risk that the development outcomes on visitation, foreign exchange earnings, and jobs will not be maintained (or realized) is assessed to be Moderate. Given the general character of the tourism sector, including unpredictability, fluctuations in demand, seasonal nature of visitation, as well as the strong link between tourism and external economic and political factors, some part of this risk is attributable to factors unrelated to the operation’s design. The 2015 forecast for Ethiopia from the WTTC Council presents a generally positive outlook for growth in the tourism sector in Ethiopia over the next ten years. The direct contribution of travel and tourism to gross domestic product is expected to increase by 5 percent and arrivals are expected to increase by 35 percent. Travel and tourism-related jobs are forecasted to fluctuate, with a 4.1 percent drop in the short term, but with an overall modest gain of 0.2 percent by 2026.21 Some of the potential exogenous and endogenous factors that could affect the sustainability of the development outcomes include: (a) economic downturns domestically and internationally (in particular from sending countries, including the United States and countries in Europe), which could, in turn, affect the performance of SMEs, the supply of international tourists, and tourism spending; (b) internal political events in Ethiopia or neighboring countries that could have consequences for the perception of safety and security; (c) change in priorities/commitment of the Government with regard to tourism, resulting in decreased spending or institutional changes; and (d) lack of follow-through in the operation and maintenance of development works, leading to deterioration of the investments/project sites. Based on forecasted growth, the current macroeconomic environment, and government commitment to expanding the tourism sector under GTP II, the likelihood of the abovementioned risks is assessed to be moderate. The project interventions themselves, especially the capacity-building interventions, are likely to improve the resilience of Ethiopia’s tourism sector to be better prepared to respond to future risks. Furthermore, the management plans developed for the project works and strong likelihood of each of the local administrations receiving adequate budget as part of their annual operation budget minimize the risk of deterioration of the investments/project sites. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 56. The quality at entry of the operation was grounded on sound technical analysis conducted during preparation, integration of lessons learned from previous interventions, and adequate consideration of the poverty, gender, and social development aspects of expanding tourism growth opportunities, though there were major shortcomings related to 20 The GOE has affirmed that there was not any arrangement which indicated handing over of materials to the Axum Tourism Office in the funding agreement with CIP beneficiaries. 21 World Travel and Tourism Council, Travel and Tourism Economic Impact 2016 21 the PDO and results framework. The strengths in design included a unique strategy of linking community involvement with the tourism sector to enhance the project’s impact on poverty reduction, in addition to incorporation of lessons learned from previous projects, to prepare an operation that addressed the tourism sector’s binding constraints through an integrated set of interventions. Some major shortcomings, however, were noted in the project’s appraisal. Most notably, the design of the operation was optimistic in its consideration of the number and complexity of activities to be completed for the level of capacity and coordination required (though it is understood that the final set of project activities was the result of significant compromise between the Bank and GoE 22 ). The optimistic design was partially the result of inadequate identification of risks – especially for client capacity, ownership and fiduciary aspects of the operation –that proved to be problematic during implementation. Furthermore, a complex PDO, lack of adequate consideration of the attribution linkages for the selected PDO-level indicator on arrivals, omission of an explicitly defined PDO-level indicator for foreign exchange earnings and overly ambitious targets/intermediate indicators established for project impacted the project’s level of achievement of planned development outcomes. Better consideration at the appraisal stage of the baselines and appropriate indicators that could be tracked through the project and at completion would have helped to eliminate indicators that would be difficult to measure and attribute. (b) Quality of Supervision Rating: Moderately Satisfactory (c) The Bank’s supervision team was focused on the development impact of the project, as demonstrated through the intensive efforts to lift the project’s partial suspension and restructure the project, though there were some moderate shortcomings in the quality of supervision. Overall, supervision for the project was adequate with an appropriate mix of skills, adequate frequency of supervision missions (12 in total), and over US$1.6 million spent on supervision. The notable efforts of the Bank team to recognize and respond to the involuntary resettlement in Lalibela through a quick and decisive partial suspension of the project, provide intensive and consistent input to facilitate the lifting of the suspension and then ensure the full completion of the RAP enabled the completion of the project outcomes despite initial pressure to cancel the operation. Furthermore, the MTR and resulting restructurings resulted in a complete turnaround of the operation, including faster and expanded implementation of the matching grant and CIP, improved technical capacity to deliver expected results, and reformulation of targets to more achievable goals. The challenges resulting from capacity constraints were especially demanding, yet these were met with hands-on support, including embedded expertise in contract management/procurement in addition to recruitment of Bank-funded experts to support supervision (M&E and the matching grant, for example). The main shortcoming in the quality of supervision is with respect to ensuring environmental safeguards compliance through regular follow-up on ESMF implementation and related actions of the PCU, as outlined in section 2.4. Furthermore, the Bank team missed the opportunity to reconsider the PDO-level indicator related 22 The GOE initially requested the Bank to support 11 destinations in total under the project 22 to tourist arrivals during implementation, in favor of an indicator that more accurately captured the outcomes (in particular mobilization of private investment). Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 57. Overall Bank performance is assessed to be Moderately Satisfactory based on the ratings for quality at entry, quality of supervision and the overall outcome rating23. Notable efforts were made by the Bank team to improve and streamline the operation during the course of implementation, including appropriately restructuring the project and providing hands-on supervision support. Shortcomings were encountered during the design and implementation of the project, where a more realistic assessment of the risks and capacity constraints, in addition to closer supervision of the fiduciary aspects of the project (especially environmental safeguards) could have supported smoother implementation and appropriate supervision. Despite these shortcomings, however, it is notable that the most of the overall outcomes for the project are substantially achieved or exceeded, resulting in an overall Moderately Satisfactory rating. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory 58. Overall Government performance is assessed as Moderately Unsatisfactory. With respect to the performance of the MoCT, a key challenge for the operation was a wavering level of commitment and ownership, which resulted in significant shortcomings in the proactive identification of opportunities and resolution of threats. While effective commitment and ownership were in order at the time of appraisal, turnover in leadership during the course of implementation resulted in a shift of priorities and an overall reduction of engagement by the ministry. Additionally, the extremely lengthy ministry processes for securing contracts that were not intended during project preparation, including the approval requirement at the state ministerial level for major contracts during the initial years of the project’s operation, caused inordinate delays in major contracts delivery. Staffing of the implementing agency was also another major concern as the Ministry prevented the PCU from hiring the specialists required to support project implementation. Some of these issues were later resolved during the course of project implementation, including development of an action plan to address shortcomings in contract management, although many persisted through the end of the project. 59. With respect to the performance of the regional and local administrations, participation and ownership varied depending on the destination. In Lalibela, the lack of compliance with social safeguards policies hindered the effective and timely implementation of the project. While the resettlement was eventually carried out with the support of the MoCT, the delayed process for lifting of the partial suspension affected project momentum and shortened the time frame for the delivery of civil works activities planned under Component 1. Furthermore, 23 When the rating for one dimension is in the satisfactory range and the rating for the other dimension is in the unsatisfactory range, the rating for overall borrower performance normally depends on the outcome rating. 23 each destination was supported by a steering committee, some of which were active (including in Lalibela and Axum) and some of which lacked consistent participation (in Addis Ababa, for example). (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 60. The PCU and SMUs demonstrated a commitment to the completion of the project’s development objectives, although moderate shortcomings in relation to the timely resolution of implementation issues, specifically with respect to fiduciary compliance, affected performance. During the initial years of the project, the PCU and SMUs included a small team with limited expertise and capacity challenges (including in critical areas such as M&E, tourism, and grants management), but following the restructuring, the project underwent significant changes and relevant expertise was recruited. At project closure, almost all of the project’s planned activities were implemented (despite the shortened time frame for implementation and numerous delays and setbacks). Furthermore, the PCU and Lalibela SMU were instrumental in coordinating with the relevant stakeholders to ensure that the partial suspension of the project was lifted, with efforts including day-to-day follow-up on the resettlement (for example, the SMU in Lalibela helped support over 30 rounds of consultations for different groups of PAPs). Some shortcomings with respect to performance included gaps in effective and proactive contract management, such as adequacy of supervision of contract works, understaffing of certain critical roles (resulting in the Bank team hiring embedded experts), and unclear transition arrangements for operation of some of the facilities (notably Tiya and Melka Kunture). Furthermore, the PCU did not recruit an Environmental Specialist, resulting in lack of compliance with the implementation of the ESMF and lack of adequate training for local project stakeholders. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 61. Overall borrower performance is assessed as Moderately Satisfactory, based on the combination of ratings for Government performance, implementing agency performance, and the overall outcome rating. Despite the varying levels of commitment and ownership of the project by the MoCT and regional/local governments, the major outputs and outcomes were substantially achieved through notable efforts to lift the project suspension, recruit competent staff in certain critical areas, and remain responsive to challenges. 6. Lessons Learned 62. Integrating the community in the tourism industry to foster pro-poor growth. The unique approach of the CIP to provide awareness-raising and participation among the community helped ensure that the poor benefitted from the project’s investments in tourism. Specifically, the CIP provided training and skills enhancement to improve community members’ participation as employees, suppliers, and small business operators to develop linkages for common benefits. This intervention was considered by many to be one of the most successful in the ESTDP and helped generate an estimated 1,151 permanent jobs, many for individuals in marginalized communities. Furthermore, the creation of 54 business ventures is an important achievement given that it contributes to the growth of the private sector and the sustainability of the jobs created. 24 63. The value of a sustained project that crowds in other partners and gives confidence to the private sector to invest and operate. Part of the rationale for the project was not only established through the direct contributions to employment and exchange earnings, but also through the capacity of the sector to drive linkages that foster indirect employment, entrepreneurship and investment. This, in turn, created increased awareness and response to demand, which had the effect of catalyzing further investment and generating confidence. 64. Importance of developing PDO-level indicators that are attributable to project interventions and that consider outcomes at project closure. For tourism projects, it is difficult to establish clear attribution of project activities because of the often complex nature of project interventions and the number of unpredictable factors that impact outcomes. Careful consideration must be given to the external factors influencing achievement of outcomes as well as the time lag between when results are achieved and when information is spread and the market has time to react. Projects that build key interventions early on and allow time for them to take root (by allowing for tour operators to react to changes in physical improvement and accounting for needed promotion/booking) are more likely to be able to demonstrate outcomes. As evidenced in other cases, it can take years for the tourism industry to consolidate and yield results after interventions, and in this respect the PDO-level indicators selected for the intervention should consider the project timeline (that is, making the distinction between short-term and long-term outcomes). 65. Undertaking a realistic assessment of capacity constraints, risks, and demands of implementation during the design phase. While projects should endeavor to create an impact through an integrated set of interventions that address binding constraints, design should also be realistic and focused with an appropriate understanding of the capacity constraints and risks involved for implementation. This includes assessing implementation capacity and developing timelines that account for delays and setbacks in such contexts. It also includes front-loading of expertise and training to help mitigate project risks before they arise. While the project eventually hired the relevant expertise to ensure the smooth operation of activities, expertise and training provided earlier in the course of project implementation might have helped avoid unnecessary delays as well as enabled the transfer of capacity to the PCU and SMU. It is also essential to recruit competent staff who fully understand and know the Bank policies and processes. 66. Importance of broad coordination and communication with stakeholders at the community, regional, and federal levels. Community-level involvement in the project, especially during the resettlement process, was strong and instrumental to ensuring a smooth resettlement program. The establishment of a resettlement committee and grievance hearing committee was highlighted as critical to ensuring participation of the local community. Project implementation also required communication and coordination at different levels of the Government, some with unaligned priorities. Ensuring early commitment from the municipal, regional, and federal administrations was important as was participation of relevant authorities in the Project Steering Committee (PSC). More broadly, an understanding of the political economy and its changing nature was critical for the implementation of destination-level tourism activities. 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 67. The Borrower provided comments to a draft version of the ICR which have been incorporated in the text, though the GoE’s assessment of the project ratings differ from the Bank’s assessment. Annex 6 provides a more detailed discussion of the government comments and ratings. 26 Annex 1. Project Costs and Financing (a) Project Cost by Component (US$, Millions Equivalent) Appraisal Estimate Actual/Latest Estimate Percentage of Components (US$, millions) (US$, millions) Appraisal Component 1- Destination Development Subcomponent 1.1: Rehabilitation and 10.3 17.4 168 enhancement of basic infrastructure Subcomponent 1.2: Visitor services 4.0 0.96 24 enhancement in selected destinations Subcomponent 1.3: Tourism products 1.1 1.1 100 development Component 2 – Market Development Subcomponent 2.1: Branding, positioning 2.7 0.79 29 and marketing Subcomponent 2.2: Demand-driven linkages 5.15 2.87 56 program Component 3: Institutional Development and Capacity Building Subcomponent 3.1: Institutional 3.9 2.57 66 development Subcomponent 3.2: Capacity building 1.6 0.53 33 Component 4: Implementation Support Financing the set-up of a PCU and SMUs 2.7 3.9 144 and results monitoring Total Baseline Cost 31.5 30.13 96 Physical contingencies 1.9 0 0 Price contingencies 1.6 0 0 Total Project Costs 35.0 30.124 96 (b) Financing Actual/Latest Appraisal Type of Disbursement Percentage Source of Funds Estimate Cofinancing Estimate of Appraisal (US$, millions) (US$, millions) Borrower – 1.50 2.2 137 IDA – 35.00 34.7 99 Local sources of the borrowing country – 3.0 2.1 70 (c) Summary of Budget Reallocation by Category (in Line with Project Restructurings) Historical Original IDA Category First Second Financing Disbursed as Category Restructuring Restructuring (US$, of June 20, (US$, millions) (US$, millions) millions) 2016 (US$, millions) Service 7.67 3.82 4.0 4.7 Goods 2.22 4.28 4.4 3.1 Works 14.29 15.83 16.7 19.9 Training 2.27 1.99 2.3 1.4 Matching grant scheme 3.0 2.2 2.3 2 24 Figure is taken from the most recent quarterly Interim Financial Report dated February 2016, which does not reflect the latest disbursements. The exchange rate used is US$1 = ETB 21. 27 Historical Original IDA Category First Second Financing Disbursed as Category Restructuring Restructuring (US$, of June 20, (US$, millions) (US$, millions) millions) 2016 (US$, millions) Operating cost 1.37 3.18 3.0 3.3 Project preparation facility 0.70 0.29 0.28 0.2 Unallocated 3.5 3.41 1.9 0 Total 35.0 35.0 35.0 34.7 28 Annex 2. Outputs by Component (a) Overview of Project Activities and Outputs Table 2.1. Project Results Chain Inputs Activities Outputs Outcomes  Project funds  Destination  Historic sites/monuments and (a) Increase in  Project staff development access roads rehabilitated international  Equipment  Market development  Visitor service centers visitation  Facilities (marketing and established and staffed (b) Increase in promotion,  Tourism product development spending (and matching grant and plans prepared and marketing hence in foreign CIP intervention) promotion strategies adopted exchange  Capacity building  Beneficiary firms utilizing earnings) matching grant financing and (c) Increase in business ventures established tourism-related  Tourism strategies, grading jobs regimes, tourism statistics systems established  Individuals trained in the tourism industry Component 1: Destination Development Subcomponent 1.1: Rehabilitation and enhancement of basic infrastructure Table 2.2. Intermediate Results Indicators for Subcomponent 1.1 Intermediate Result Indicator Baseline Target Actual Status Indicator 1: Number of historic sites/monuments 0 6 6 Achieved rehabilitated in all the three targeted destinations Indicator 2: Kilometers of access/town roads 0 4 21.1 Exceeded rehabilitated (kilometers, custom supplement) Table 2.3. List of Outputs for Subcomponent 1.1 Planned at Appraisal Actual Completed Lalibela Conservation of historic  This activity was dropped early on in project implementation due to the churches cost, time/skills required, and overlap with UNESCO mandate. Delineation of World Heritage  This activity was dropped early on in project implementation due to overlap Site boundaries with UNESCO.  Eight public toilets were built in three satellite localities—four in Development of satellite sites in Yemrehane Kristos Church, two near the Monastery of Na’akuto La’ab and Lalibela two in Asheton Maryam.  12 km paved along Bilbila-Yemrehane Kristos Road Improvement of basic  3.6 km paved along the bypass road infrastructure (bypass roads)  1.4 km cobble stone repaving/maintenance of streets in the Lalibela core zone Improvement of water supply  Improvement of the quantity and quality of potable water through sanitation and solid waste installation of three boreholes, providing an additional 22 l/s management 29 Planned at Appraisal Actual Completed  Construction of liquid and solid waste treatment site, purchase of a vacuum and skip loading truck to transport liquid and solid waste, and construction of 12 public toilets in Lalibela and water reservoir Axum  3.5 km and 50,000 m2 of paved road in the street and squares, respectively, Improvement of the stelae field (including Dearo Ela Piazza, Italian Piazza, the stelae square, Geza Egumay street, and along the street of Maishum and complementary areas)  The outer walls of the Archaeological Museum of Axum were restored. Archeological museum Roof leakage was fixed and the electrical system was replaced.  Improvements were made to the Tsion Church, including repairing of roof structure and construction of a fence. Monuments and attractions in Axum  Four structures (houses) were renovated, including one archeological museum, one ethnographic museum, and two houses in a craft center compound. Addis Ababa and Environs Rehabilitation of the Addis  Rehabilitation of the Addis Ababa Museum was completed, including a Ababa Museum visitor entry, restaurant/café, public toilets, and landscaping.  A new visitor museum was built in Melka Kunture that includes a main space for exhibitions/cultural artifacts, a workshop, storage room, rooftop Museums in Tiya and Melka terrace, and six toilets. Kunture  A new archeological museum and ethnographic museum as well as community center were completed in Tiya that include exhibition space along with eight public toilets and two showers. Subcomponent 1.2: Visitor services enhanced in selected destinations Table 2.4. Intermediate Result Indicator for Subcomponent 1.2 Intermediate Result Indicator Baseline Target Actual Status Indicator 3: Number of visitor service centers established 0 3 3 Achieved and staffed in all the three targeted destinations Indicator 4 (dropped): Number of trails with signage in 0 1025 n.a. Dropped targeted destinations Table 2.5. List of Outputs for Subcomponent 1.2 Planned at Appraisal Actual Completed Axum, Tiya, Melka Kunture  A new visitor center was built in Axum including a main reception area, café area, administration office space, gift shop, audiovisual room with Rehabilitation and upgrading of seating, exhibition venue, and toilets. Eight staff members from the tourist/visitor facilities Tourism Bureau are expected to work in the center.  Visitor facilities were built and staffed in Lalibela and Addis Ababa.  64 signage and 30 interpretation panels were completed in Lalibela. Interpretive services, signage/trail  12 signage and 20 interpretation panels were completed in Axum. enhancement/documentation, and  The indicator related to trails with signage in targeted destinations was exhibitions at historic sites dropped during the midterm review in light of financial and project timeline constraints 25 Original target value from approval document 30 Subcomponent 1.3: Tourism products development Table 2.6. Intermediate Results Indicator for Subcomponent 1.3 Intermediate Result Indicator Baseline Target Actual Status Indicator 5: Number of tourism product development plans prepared in secondary destinations 0 3 3 Achieved Table 2.7. List of Outputs for Subcomponent 1.3 Planned at Appraisal Actual Completed Tourism products development utilizing vernacular architecture, landscape features,  Four vernacular houses have been refurbished to use for tourism activities including historic district assets, for restaurants/craft center in Lalibela. and intangible heritage in Axum and Lalibela Tourism products development  Product development plans for the southern, western, and eastern routes and capacity development plans were prepared with extensive consultation. The preparation of the plans has for the southern, western, enabled identification and planning for the sector to continue to expand and eastern routes beyond the target sites in the project. Component 2: Market Development Subcomponent 2.1: Branding, positioning, and marketing Table 2.8. Intermediate Result Indicator for Subcomponent 2.1 Intermediate Result Indicator Baseline Target Actual Status Indicator 6: Number of marketing and promotion strategies adopted with at least three international tourism market 0 3 3 Achieved segments Indicator 7 (dropped): Number of independent small and medium enterprise travel service providers using the e- 0 10026 n.a. Dropped travel platform Table 2.9. List of Outputs for Subcomponent 2.1 Planned at Appraisal Actual Completed  A national strategy for tourism, including a five-year tourism marketing plan, and a brand strategy were completed. Ethiopia’s new brand ‘Land of Development of a positioning Origins’ differentiates Ethiopia from other African competitors. and marketing strategy and  Five marketing strategies were prepared for five regions including Amhara, implementation of a Tigrai, Addis Ababa, Oromia, and Southern Nations, Nationalities, and communication and promotion Peoples. The marketing consisted of a portfolio of five key marketing plan programs: ‘Encounter the Origin’, ‘Great Origin Routes’, ‘Tena Yistilign’, ‘Ethiopia ‘i’ Network’, and ‘Meet Ethiopia.’  A package of materials was developed that included printed materials (15 types of brochures, including general content, but also presenting routes Development of promotional and promoting specific sites), including multimedia and digital content. materials  One national website was created: http://www.ethiopia.travel/ with information focused on the core markets: the United Kingdom, Germany, 26 Original target value from the approval document 31 France, Italy, the United States (East and West Coast areas), Saudi Arabia, and the domestic market. Participation in trade fairs  This activity was not completed as part of the project. Development of a tourism e-  This activity was dropped following the MTR. It was agreed that the sector commerce online platform had not yet developed enough to manage or sustain a technical system. Subcomponent 2.2: Demand-driven Linkages Program Table 2.10. Intermediate Result Indicator for Subcomponent 2.2 Intermediate Result Indicator Baseline Target Actual Status Indicator 8: Number of beneficiary firms utilized matching 0 50 79 Exceeded grants financing Indicator 9: Number of community-based and tourism- 0 45 54 Exceeded related business ventures established Table 2.11. List of Outputs for Subcomponent 2.2 Planned at Appraisal Actual Completed  A matching grant scheme with an allocation of US$2.4 million (size reduced following the MTR) was implemented in Lalibela, Axum, and Lalibela. A Implementation of a cost- total of US$2.1 million was disbursed27 to 79 firms (hotels, restaurants, tour sharing (50-50 percent) operation, tourism promotion and marketing, tourism training, agribusiness, matching grant scheme with an crafts, and training). Grants were used for product/service improvement, allocation of US$3.0 million system improvements, training, and marketing. The matching grant scheme provided an estimated 457 permanent jobs and training to 1,781 employees of matching grant scheme beneficiary firms.  The CIP was implemented in Lalibela, Axum, and Addis Ababa with the goals of generating new employment opportunities in tourism businesses, improving supply of goods and services to tourism enterprises and direct sales, and establishing community-based businesses in the area. A total of 54 Implementation of a US$2.15 new business ventures were established as a result of the intervention, million CIP to optimize providing 1,151 permanent jobs in the targeted destinations. The total benefits for the poor allocation for the CIP was approximately US$1.1 million, with most disbursements being used for training and in-kind support and the remaining funds (approximately 9 percent) disbursed for procurement of materials for Lalibela and Axum handicraft centers. Component 3: Institutional Development and Capacity Building Subcomponent 3.1 Institutional development Table 2.12. Intermediate Results Indicators for Subcomponent 3.1 Intermediate Result Indicator Baseline Target Actual Status Indicator 10: A strategy with a detailed 3–5 year action plan to support the implementation of the national 0 1 1 Achieved tourism strategy developed 27 According to the latest available data in client connection on June 29, 2016 32 Indicator 11: A lodging grading regime compatible to 0 1 1 Achieved international standards developed and pilot tested Indicator 12: An upgraded national tourism statistics system that is capable of capturing basic tourism 0 1 1 Achieved indicators Table 2.13. List of Outputs for Subcomponent 3.1 Planned at Appraisal Actual Completed Policy implementation frameworks, including development of sector strategic plan, and  A national strategy was developed through extensive consultations that has competitiveness enhancement, guided the Government (national, state, and local level) in tourism including development of a development and decision making sector strategic plan based on the national tourism policy and competitiveness enhancement packages  A series of short-term trainings were delivered to public and private sector participants in five key areas: (a) tourism sector overview (policy approaches, promotion and accreditation, product development, and tourism development); (b) integrated destination planning and management (natural and cultural resource protection and development, site and destination management, marketing and promotion); (c) tourism marketing MoCT and and branding (applying technical and practical tools for market research, regional tourism bureau formulating cost efficient promotional activities); (d) tourism planning and implementation (providing key public sector officials with planning and implementation skills, risk management, M&E); (e) management and technical skills (for top and mid-level managers to support improvement and manage of the sector.  Office equipment was also provided by the project to the regional tourism bureaus in Lalibela and Axum.  A total of 378 hospitality facilities in 7 regions have been assessed/graded. Establishment of standards and Two additional regions finalized the assessments and some properties in rating schemes for tourism Addis Ababa required reassessments. A total of 54 grading experts have enterprises been trained and the final workshop was conducted.  A system containing data from different sources based on a survey of domestic tourists (1,740 questionnaires) and international tourists (2,810 questionnaires) was implemented in 2014. This was the first Tourism data collection and comprehensive attempt to gather reliable tourism data, resulting in the first statistics strengthening database of statistical tourism data. The consultant hired for the assignment provided training in Addis Ababa on how to use and run the system for the persons in charge of the ETSMS Platform. Finally, the ETSMS Platform was migrated from the consultant’s server to the MoCT server.  The ETO was officially established in 2013 and started its operations in 2014 with a double mandate: fostering destination and product development and promoting Ethiopia as a tourism destination. Its Establishment of and support to a membership includes both the private and public sectors. tourism board to foster public-  The ETO is expected to play a pivotal role in building the capacity of tour private partnerships in the area of tourism promotion operators, hotels, travel agencies, and stakeholders to fully exploit the country’s tourism resources and thereby generate worthwhile income. In relation to the ESTDP, the ETO has been responsible for monitoring the marketing plan and branding strategy design, including production of the 33 Planned at Appraisal Actual Completed new promotional materials, and will be in charge of implementation of the marketing strategy in the coming years. Subcomponent 3.2 Capacity building Table 2.14. Intermediate Results Indicators for Subcomponent 3.2 Intermediate Result Indicator Baseline Target Actual Status Indicator 13: Number of instructors/trainers trained on training curricula developed under 0 40 44 Exceeded the project at vocational schools Indicator 14: Number of people in the tourism industry trained in Addis Ababa/environs, 0 200 1661 Exceeded Axum, and Lalibela Table 2.15. List of Outputs for Subcomponent 3.2 Planned at Appraisal Actual Completed  An assessment of tourism strategic needs was carried out, which focused on public institutions in the target destinations, namely, Technical Vocational Education and Training Centers in both Lalibela and Axum and Assessment of the the Catering and Tourism Training Center in Addis Ababa. Special tourism education and training attention was given to the skills of trainers and teachers to provide high- strategic needs level tourism training. Five comprehensive reports were drafted that identified actions and steps needed to expedite the planned activities, and, thereby, achieve quality tourism education. Action plans were also outlined with key roles for the MoCT, ESTDP, and private sector.  In November 2015, the project financed training of 44 instructors from the Catering and Tourism Training Center in Addis Ababa and Technical Vocational Education and Training Centers in Lalibela and Axum. Topics included food preparation and presentation, food safety and Improvement of tourism training hygiene, housekeeping, tourism marketing, tour planning, and operation of institutions’ programs Oracle database and Structured Query Language database. The training was conducted in Mauritius in two cycles and participants interviewed during field visits found the courses useful and practical, contributing to an enhanced ability to deliver training.  A capacity needs assessment and an intervention plan were developed for (a) the MoCT (b) regional culture and tourism bureaus of Amhara, Tigray, and Addis Ababa, and (c) three local culture and tourism offices of Axum, Lalibela, and Addis Ababa and environs. The gap assessment report Skills enhancement at the MoCT identified five training areas: tourism planning and implementation; and other sector stakeholders integrated destination planning and management; tourism marketing and branding; tourism sector overview; and management and technical skills. Trainings in these topics were conducted for 163 people from the federal, regional, zonal, and woreda-level culture and tourism offices during a one- year period. 34 Core indicators Table 2.15. Intermediate Results Indicators for all Components Intermediate Result Indicator Baseline Target Actual Status Indicator 15: Direct project beneficiaries 0 500 3,837 Exceeded Indicator 16: Female beneficiaries 0 100 1,704 Exceeded (b) Detailed Assessment of Achievement of PDO-level Results Key Outcome 1: Average increase in the number of international visitors in the targeted destinations of Axum and Lalibela Rating: Substantial Achievement of Outcome and Associated Indicators 1. Data on international arrivals derived from hotel occupancy rates show that average international visitation targets have been fully achieved in Axum and partially achieved in Lalibela. In Axum, the average growth rate of international visitation (from 2012 to 2015) of 61.4 percent is well above the targeted average growth rate of 23 percent. In Lalibela, the average growth rate of almost 24 percent is below expectations of 33 percent average growth (see table 2.16), but still strongly positive. The methodology used for this calculation was based on primary data collected through a survey administered to all hotel establishments in Axum and Lalibela in 2015.28 The calculation for this indicator captures an average of the growth rates over the project duration rather than annual growth (see section 2.3 on M&E for further discussion). Table 2.16. Derived Estimate of International Arrivals and Growth Rates in Lalibela and Axum (2012–2015) from Terminal Evaluation Lalibela Axum Growth from 2011 Baseline Growth from 2011 Baseline Arrivals Arrivals (%) (%) 2011 41,158 – 28,460 2012 44,572 8.3 37,169 30.6 2013 49,503 20.3 40,837 43.5 2014 46,862 13.9 50,222 76.5 2015 62,420 51.7 55,450 94.8 Average growth – 23.6 – 61.4 rate (2012-15) 28 A total of 504 tourism establishments (all known establishments in Axum and Lalibela) were interviewed as part of the Terminal Evaluation. The methodology for the calculation of arrivals included (a) identifying the opening year of all the accommodation establishments; (b) multiplying occupancy rates by capacity (rooms); (c) adjusting by average length of stay (from tourists’ survey conducted in the Project’s Terminal Evaluation); and (d) adjusting by the international tourist ratio (for the international arrivals figures). 35 Source: November 2015 Establishment Survey, derived through hotel occupancy rates. Note: Growth rates are calculated from the initial 2011 base year and then an average is calculated covering the period from 2012-2015. The 2011 base year is chosen due based on the availability of data and the partial suspension which was lifted in June 2012. 2. Growth in tourist arrivals is supported by a steady increase in hotels and a small increase in hotel occupancy rates (see figures 2.1 and 2.2) in both destinations. From 2010 to 2015, hotels grew from 9 to 43 in Axum and from 8 to 55 in Lalibela. Occupancy rates at the hotels also grew (although marginally), which suggests substantial positive growth in the number of arrivals for both destinations. Figure 2.1. Growth in Hotels in Lalibela and Figure 2.2. Average Occupancy Rates in Axum, 2010–2015 Lalibela and Axum, 2010–2015 60 44 43 40 42 41 20 40 0 39 2010 2011 2012 2013 2014 2015 38 37 Axum Lalibela 2010 2011 2012 2013 2014 2015 Source: Axum Town Office of Culture and Tourism, Trade and Industry 2015, and Lalibela Source: ESTDP Terminal Evaluation, Hotel Town Office of Culture and Tourism, Trade and Establishments Survey. Industry 2015. Attribution 3. The attribution of arrivals to Lalibela and Axum is established through the efforts made to transform both towns into modern tourism destinations, which, in turn, provided the foundation to attract tourism. The growth in various establishments that peaked in 2014 (see figure 2.4) provided some evidence that the project may have influenced visitation given that the growth in tourism establishments would need to be supported by growth in tourists. The growth in 2014 coincides with disbursement for the project at 86 percent. 4. For tourism projects, attribution is usually not immediate due to the time required for the market to react to new opportunities (for example, international tourists learning of new and improved sites and accommodations and suppliers offering new products and services to meet demand). The delayed disbursement of the project, especially under Component 1, should be considered, especially as visible improvements (with regard to infrastructure and quality and variety of tourism products) did not take place in many of the destinations until 2014 with a number of them not being fully finished until the end of 2015. The International Tourism Survey, included as part of the project’s terminal evaluation, provides some positive indication of the plausibility of 36 the destinations attracting increased arrivals in the near future. Of the 42029 international tourists surveyed in November 2015 who visited Axum, Lalibela, and Addis Ababa, 93 percent would recommend a visit to Ethiopia and 59 percent will plan to repeat their visit in the next five years. Key Outcome 2: Increase in average of foreign currency spending per international tourist per day in all the three targeted destinations of Addis/Environs, Axum and Lalibela Rating: Substantial Achievement of Outcome and Associated Indicators 5. Based on a survey conducted in November 2015 of international tourist arrivals in the targeted destinations, the target average expenditure per person per day in Addis Ababa, Lalibela, and Axum has been exceeded.30 The total average expenditure of US$361.86 is above the target figure of US$350 (see table 2.17) from a baseline of US$310. If the expenditure per person per night at origin is also included, this figure is increased to US$519.31 Table 2.17. Expenditure per Person per Day by Destination Expenditure per Person per Expenditure per Person per Night (including Expenditures at Night (US$) Origin) (US$) Addis Ababa 604 653 Lalibela 230 515 Axum 252 390 Average 362 519 Attribution 6. Increased expenditures by tourists can be directly attributed to project activities that have enabled the growth of tourism-related businesses, products, and services in each of the destinations and provided the capital, skills, and TA to capture opportunities in tourism. These include investments in infrastructure and direct financing that specifically targeted the strengthening of tourism products/services. The matching grant scheme, for example, financed projects that focused on improvement in firm-level diversification and quality of products/services. In a survey administered of the matching grant beneficiaries, around half of the 54 firms who participated in the survey confirmed that their investment resulted in improvement in their product/service 29 This is a statistically significant sample size above the 95 percent confidence level (calculated as 383 international tourists for the three destinations). 30 These data were collected from the previously mentioned international tourism survey. The survey was administered to a sample of 420 international tourists at the airport who had previously visited one of the destinations (Axum, Lalibela, and/or Addis Ababa). The tourists were asked to report how much they had individually spent at any one (or all) of the destinations. The reported figures were then averaged for each destination. 31 The ESTDP baseline survey included flight expenses in calculating the foreign tourists per capita per day expenditure. However, this expenditure is not used to evaluate the achievement of this indicator because it may remain in the tourist’s country of origin as loss of added value. 37 diversification and a similar proportion indicated their investment resulted in an enhancement of the quality of their products/services. 32 The CIP also specifically contributed to product diversification, especially in the handicraft, agribusiness, and small catering/coffee subsectors where in-kind assistance and training contributed to better packaged and better quality products. The new businesses generated by the CIP (56 business ventures in total) resulted in expanded offerings that provided greater opportunities for tourists to spend money compared to the years before project implementation. The project’s terminal evaluation found a positive correlation with project activities and the number of complementary services established as a result of the project (transport, tour guide, coffee, and craft businesses). In particular, craft and transport openings can be strongly attributed to the project based on road activities and direct interventions as a part of the CIP.33 Key Outcome 3: Number of jobs created in tourism-related industries in the targeted destinations of Axum and Lalibela Rating: High Achievement of Outcome and Associated Indicators 7. The total number of jobs (permanent and temporary) in the tourism sector in the target destinations reached 5,553, well above the target figure of 1,440, starting from a baseline of 1,090.34 Figure 2.3 provides a breakdown of the jobs created in Axum and Lalibela based on a survey taken of establishments in Lalibela and Axum. Figure 2.3. Employment Distribution in Lalibela and Axum (2015) Lalibela Axum Temporary  Temporary  female female Temporary  6% Temporary  8% male male Permanent  4% 9% male Permanent  33% male 46% Permanent  female Permanent  44% female 50% 32 This is taken from the Evaluation Report on Business Performance of Matching Grant Scheme in which a survey of beneficiary firms was undertaken reporting on level of product/service diversification and quality of products/services. ‘Good’ improvement was defined as 51 percent–70 percent of the agreed business plan. 33 Under the CIP intervention crafts businesses and coffee/small catering businesses accounted for around 30 percent and 15 percent, respectively, of 54 supported enterprises. 34 These data are taken from the ESTDP Terminal Evaluation of the November 2015 Establishments Survey, in which all tourist establishments was interviewed in Axum and Lalibela and were surveyed on employment number of employees in 2015 and in the period from 2010 to 2015. 38 Source: ESTDP Terminal Evaluation taken from November 2015 Establishments Survey. Attribution 8. Job attribution is established from the interventions supported by the project with clear linkages to job generation, including the matching grant and community involvement investments, which together created more than 1,600 (permanent) jobs.35 The 457 permanent jobs linked to the matching grant intervention were the result of new jobs added by firms in the years following implementation of their projects (these were tracked on a bimonthly basis). The remaining jobs generated through the CIP are the result of 54 new, formally registered business ventures. 9. Attribution is further strengthened through structural time series analysis that reveals a large number of new establishments that grew every year at an increasing rate in accordance with the disbursement under the project. The time series reveal that, starting from 2014, the number of new establishments grew substantially. These new establishments are likely to be a source of new jobs. The timing coincides with accumulated disbursement for the project at 86 percent of the total budget amount in addition to the implementation of many tangible activities, such as civil works and implementation of the matching grant and community involvement investments. Furthermore, this indicator was revised to only include jobs in subsectors (accommodation, catering, tourist transportation, and tourist guides) that were relevant and linked to project interventions. Figure 2.4. Number of Establishments by Type in Lalibela and Axum (1990–2015) Source: ESTDP Terminal Evaluation taken from November 2015 Establishments Survey. 35 These data were collected and updated by the PCU. The livelihoods restoration scheme under the resettlement is estimated to generate another 632 jobs. 39 Annex 3. Economic and Financial Analysis 1. At appraisal, the project’s ERR and net present value (NPV) were calculated for the destination development, matching grant, and capacity-building interventions using a cost-benefit methodology considering the economic inflows from the project’s main outcomes (including tourist visitation and income generation through tourism spending and jobs) and costs. All of these considerations remain valid and relevant at project completion. The calculations and assumptions, however, have been adjusted based on project implementation. Table 3.1 summarizes the benefits and costs of the project calculated at appraisal compared to a revised estimate calculated at project closure. Table 3.1. Summary of Benefits and Costs Fiscal Impact Fiscal Impact Present Values of Present Values (at Appraisal) (at Closure) Flows (at of Flows (at Taxes (US$, Taxes (US$, Appraisal) Closure) millions) millions) Benefits (US$, millions) 108.1 79.5 Costs (US$, millions) 19.8 16.4 Net Benefits (US$, 32.4 23.8 88.3 63 millions) ERR (%) 43 37 Main Assumptions 2. The detailed benefits and costs per intervention are provided by component in table 3.2. Using the same basic methodology and framework for the recalculation of the NPV and ERR applied at appraisal, the following general assumptions were made:  The discount rate is unchanged, calculated at 12 percent based on the standard opportunity cost of capital at 10 percent for most Bank projects.  The 12-year time horizon used for the original calculation of the NPV is unchanged although cash inflows and outflows have been adjusted in line with implementation as well as actual disbursement levels. Component 1: Destination Development  Under Component 1, the number of tourist arrivals in the ‘with’ project and ‘without’ project scenario have been updated based on actual figures reported from the government. Given that some part of the growth in tourism arrivals may be attributed to events unrelated to the project, the number of international tourists in the ‘with’ project scenario is estimated to be only slightly higher (by 5 percent) than the number of international tourists in the ‘without’ project scenario starting from 2015 when all the civil works activities were completed. Assumed growth in international tourists from the end of the project period onward is 9 percent per year in both destinations. 40  Average expenditures per arrival have been adjusted to total US$361.86 per person from 2014 onward in the ‘with’ project scenario as estimated through the International Tourism Survey in the project’s terminal evaluation.  Given the lack of available information on domestic tourist expenditures and visitation, these figures are unchanged. It is estimated that a total of 403,000 domestic business tourists visit Addis Ababa; for Axum and Lalibela the estimate is 17,961 and 9,644, respectively. The estimates of average length of stay and average spending per person per day are based on Overseas Development Institute estimates calculated at appraisal of approximately 9 days (with slight growth projected over the project duration) and spending of US$15/day.  Project overhead costs (that is, project implementation) were allocated at a rate of 60 percent of the total cost of the PCU. Component 2: Matching Grant and Community Involvement  Under the matching grant and CIP interventions, the total number of firms supported, jobs created, and funding disbursed have been updated based on real data. The assumed increase in economic output is 2 times the total grant amount, in line with the findings of the project’s matching grant performance evaluation report.  Figures reflecting total job count (457 total jobs created through the intervention) have been updated in the spreadsheet. The average assumed yearly salary per job created is US$1,286.  Project overhead costs (that is, project implementation) were allocated at a rate of 20 percent of the total cost of the PCU. Table 3.2. Detailed Economic Cost-benefit Analysis by Component (at Appraisal in Comparison to Closure) Present Value of Flows Present Value of Flows Calculated at Appraisal Percent Calculated at Project Closure Percent (US$, millions) (US$, millions) Net Net Benefits Costs ERR Benefits Costs ERR Benefits Benefits Destination development and 94.34 14.06 80.28 46 72.31 13.12 59.19 39 marketing Matching grant 5.71 2.01 3.70 43 2.34 1.56 0.78 20 scheme Institutional development and 8.01 3.74 4.28 27 4.84 1.76 3.08 27 capacity building Component 3: Institutional Development and Capacity Building  Under this component, it was assumed that capacity-building support provided to various public entities would result in an increase in output at 2 times the amount of 41 funding provided with a lag effect of 2 years. This overall assumption has been maintained for the calculation at project closure.  Project overhead costs (that is, project implementation) were allocated at a rate of 20 percent of the total cost for the PCU. 42 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending John A. Boyle Senior Environmental Specialist AFTN1 - HIS Environment Yeshareg Dagne Program Assistant GFMDR Operations Edeltraut Gilgan-Hunt Environmental Specialist AFTN1 - HIS Environment Roxanne Hakim Senior Social Development Specialist GSURR Social Shaun Mann Senior Investment Policy Officer GTCDR Tourism Francois Nankobogo Lead Operations Officer ECADE - His TTL Jonathan David Pavluk Senior Counsel LEGES Legal Mulat Negash Tegegn E T Consultant AFTME - HIS Tourism Abiy Admassu Temechew Procurement Analyst AFTPE - HIS Procurement Senior Private Sector Development Menbere Taye Tesfa AFTFE - HIS Tourism Specialist Supervision/ICR Hannah R. Messerli Team Leader GTCDR TTL Senidu Fanuel Team Leader GTC01 TTL Tesfaye Ayele Senior Procurement Specialist GGODR Procurement Abiy Demissie Belay Senior FM Specialist GG025 FM Yeshareg Dagne Program Assistant GFMDR Operations Eneida Herrera Fernandes Senior Private Sector Development GTCDR Tourism Edeltraut Gilgan-Hunt Environmental Specialist AFTN1 - HIS Environment Hemam Girma Program Assistant LLILS Operations Roxanne Hakim Senior Social Development Specialist GSURR Social Guido Licciardi Senior Urban Development Specialist GSURR Social Yasmin Tayyab Senior Social Development Spec GSURR Social Meron Tadesse Techane Sr Financial Management Specialist GGODR FM Senior Private Sector Development Menbere Taye Tesfa AFTFE - HIS Tourism Specialist Lemlem Workalemahu Program Assistant GTC01 Operations Chukwudi H. Okafor Senior Social Development Specialist GSU07 Social Efrem Tiruneh Getane S T Consultant GTCDR Engineering Maria Ariano S T Consultant GFM02 MG/CIP Alex Pio S T Consultant GTC01 M&E Shimelis Woldehawari Senior Procurement Specialist GGO01 Procurement Binyam Bedelu Senior Procurement Specialist GGO01 Procurement Private Sector Development Specialist - Jade Ndiaye GTC01 ICR ICR Dileep Wagle S T Consultant - ICR GFM01 ICR Khalid Aloune Program Assistant - ICR GFM01 ICR (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, Thousands (including Travel No. of Staff Weeks and Consultant Costs) Lending FY07 5.9 32.97 43 Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, Thousands (including Travel No. of Staff Weeks and Consultant Costs) FY08 22.23 103.50 FY09 72.43 323.44 Total: 100.56 459.91 Supervision/ICR FY10 21.95 94.02 FY11 39.85 207.4 FY12 35.98 179.00 FY13 34.94 175.42 FY14 34.55 202.20 FY15 41.33 178.26 FY16 20.53 113.10 Total: 229.13 1149.4 44 Annex 5. Stakeholder Workshop Report and Results 1. Several surveys and discussions were carried out with beneficiaries and stakeholders as a part of the project’s terminal evaluation. This section summarizes the general findings and conclusions of the surveys and focus groups. Focus Group Discussions with Beneficiaries 2. Focus group discussions included over 100 participants from Axum and Lalibela who were interviewed in November 2015. In summary, the focus group participants highlighted the visible contributions of the project to the overall appearance and preservation of the towns and the historic sites. Most participants also confirmed that the project brought tangible employment opportunities in both destinations for men and women involved in construction. With respect to the impact of the project on tourist flows, most participants commented that the intervention is likely to contribute to a future increase of tourist flows, although the project could not be held accountable for recent events that have negatively affected growth (including the dip in visitation during the outbreak of the Ebola virus and marketing/promotion of destinations by tour companies in Addis Ababa that have not consistently included Axum and Lalibela in their packaged offerings). Axum: Focus Group with Community Beneficiaries 3. The meeting included discussions with beneficiaries of the CIP, namely, in the handicraft production/sales and traditional coffee subsectors. The participants were among the businesses sponsored by the project community grant and received training in addition to in-kind support. Participants reiterated the need for continued support and confirmed the overall positive impact of the project, particularly the investments in infrastructure that have made the visiting tourists more comfortable in their stay. The majority of the participants confirmed that investing heavily in tourism is much more beneficial than any other type of investment in Axum. Regarding visitation, the discussions confirmed that the travel and tour companies in Addis Ababa have had an impact on the declining length of stay of tourists because they are not listing Axum as one of the destinations in their packaged offerings. 4. Several issues concerning the future sustainability of the ESTDP interventions were discussed by the group, including the quality and durability of the materials purchased by the project for the beneficiaries. The Axum tourism office indicated to the community beneficiaries that the materials/equipment would need to be handed over after the phase-out of the project36. Furthermore, the participants observed that the investments in greenery (landscaping) were being spoiled due to lack of proper maintenance. 36 The government disagrees with this comment and notes that there is not any arrangement which indicates handing over of materials to the Axum Tourism Office in the funding agreement with CIP beneficiaries. The government further notes that the beneficiaries directly participated in the procurement process for materials/equipment and also deployed experts to check the quality of the materials in the respective sectors, as stipulated in the ESTDP/SMU procurement manual. 45 Axum: Focus Group with Souvenir Shop Owners/Shopkeepers 5. The meeting included discussions with souvenir shop owners/shopkeepers. The participants had an overall negative impression of the project due to the perception of increased competition as a result of training interventions in the tourism industry, specifically under the CIP. While the participants had not directly benefited from any of the trainings themselves, they confirmed that the project engaged many young people in the souvenir business, which resulted in more competition for them. The participants felt that the project was not participatory and transparent. Despite these negative views, the majority of the participants agreed that investing heavily in tourism is much more beneficial than any other type of investment in Axum because it is the most valuable resource that the area possesses. Axum: Focus Group with Tour Guides 6. The meeting included discussions with tour guides in Axum who had organized themselves under a tour guide association. The tour guides specifically benefited from French language training offered by the project in addition to in-kind and TA (overcoats, advertising boards, as well as assistance in developing a website). The group agreed that the project had contributed to positive visible improvements in Axum and transformed the city into a more modern international tourist destination while also maintaining its cultural heritage. Notwithstanding the positive contributions of the project, some of the participants felt the project lacked accountability and transparency. The beneficiaries gave as an example communication regarding the project’s allocated budget for tour guides, which was later revised downward. They also pointed out that there were problems with the cobblestone road built behind the main stelae field which did not provide adequate drainage. Nevertheless, the tour guides reiterated that tourism sector support should continue and allow for some type of funding for the tour guides association. Axum: Focus Group with Air Transport Service Providers 7. The meeting included discussions with minibus owners/drivers, who had not been offered any assistance by the project but who were considered general project stakeholders as a result of their participation in the tourism industry. This group was considered to be important because they are dedicated to shuttling tourists to and from the Axum airport and are, therefore, in a better position to understand the increase/decrease of tourist flow to Axum. According to the transport service providers, with the exception of 2014, tourist flow has shown an increasing trend in the last five years and most stays are either half day or full day. The participants explained that the cobblestone roads built by the project had contributed to improving Axum and making it more pleasant for visiting tourists. Lalibela: Focus Group with Souvenir Shop Owners/Shopkeepers 8. The meeting included souvenir shop owners/shopkeepers in Lalibela. The majority of the participants were owners or family members of owners. The participants were not supported by the project in matters related to their specific business of souvenir production, marketing, or selling. However, some of the participants benefitted from training and in-kind assistance from the CIP that resulted in increased competition for some of the souvenir shops. Overall, participants observed that the project interventions resulted in visible improvements to the towns, including 46 improved sanitation, drainage, cobblestone roads, and removal of solid waste, which were major problems in Lalibela before the project. Participants confirmed that investing heavily in tourism was much more beneficial than any other type of investment in Lalibela given the limited productivity of agriculture, lack of other natural resources in the area, and lack of alternative livelihood choices for the inhabitants. Participants noted that in the last five years, there has been an increase of foreign tourist flows to Lalibela except in 2014 due to the outbreak of the Ebola virus in other African countries and the travel ban imposed by many countries following this event. The participants indicated that in any given year the months of January, July, and August are the high season while May and June are the low season. The majority of tourists visiting Lalibela are citizens of Germany, France, and Spain. It is the perception of the participants that tourists spend relatively more money now than five years ago, and possible reasons may be the increasing cost of hotels, the revised entrance fee to the churches, and other factors. 9. Finally, they indicated that the ESTDP should continue, but it needs to include specific activities that benefit the souvenir business through training in marketing, branding, salesmanship, and other relevant topics. Lalibela: Focus Group with Resettled Households 10. See annex 8 for a summary. Results International Tourism Survey and Establishments Survey 11. Two surveys were carried out in November 2015 as part of the project’s terminal evaluation.  The international tourist’s survey consisted of a questionnaire administered to a sample of 420 international tourists by enumerators in the airports in Axum, Lalibela, and Addis Ababa. The survey focused on descriptive information, including on spending and overall satisfaction with the destinations covered by the project. Over 75 percent of the respondents came from North America and Europe (including Germany, the United Kingdom, the United States, Belgium, Italy, France, and Canada), and 70 percent of tourists had previously visited Ethiopia.  The Tourism Establishment Survey included a questionnaire administered by enumerators to all 504 tourism establishments in Axum and Lalibela. The survey focused on collecting information including occupancy rates, the number of people employed over time, and perceptions of destination quality. The full results of the survey are presented in the project’s terminal evaluation, while the data given in table 5.1 capture overall satisfaction and awareness of the activities undertaken under the project. Overall Satisfaction of Tourists and Establishments with the Activities Undertaken 12. As shown in table 5.1, overall, tourists’ satisfaction is high for the Axum stelae field and Lalibela’s landscaping and improvement of heritage sites. Satisfaction is relatively low for the Lalibela bypass road, water supply, and sanitary schemes. Establishments are similarly satisfied with all the activities except the visitor center (note that the visitor centers had not yet been completed and open to the public at the time of the survey in November 2015). Of the international 47 tourists surveyed, 93 percent recommend a visit to Ethiopia and 59 percent will repeat a visit within the next five years. Table 5.1. Satisfaction among Tourists and Establishments with the Activities Undertaken Lalibela Axum Addis Ababa Tourists Establishments Tourists Establishments Tourists Establishments Tsion Church 4.2 – 4.8 4.0 5.0 – Archeological 4.2 – 4.7 4.3 5.0 – Axum museum Stelae field 4.7 – 5.0 4.2 4.9 – Walking routes 3.8 – 4.5 3.8 5.2 – Visitor Center 3.3 – 4.5 3.6 5.0 – Landscaping and 4.8 4.9 5.3 – 4.7 – improvement of heritage sites Lalibela Gravel road 3.7 5.3 4.5 – 4.6 – Bypass road 3.3 4.9 3.7 – 4.3 – Water supply and sanitary 3.4 4.8 2.9 – 4.3 – schemes Visitor center 3.6 3.0 3.7 – 4.4 – Source: ESTDP Terminal Evaluation, November 2015 International Tourists Survey. Note: Ranking measured from 0 to 6, with 0 indicating low satisfaction and 6 indicating high satisfaction. Figure 5.1. International Tourist Satisfaction by Activities and Destination (Aggregate Percentage) 300 250 200 150 100 50 0 Not in charge of… Quality of… A new country to… Quality of… Price/quality ratio Adventure Close destination Climate Heritage Quietness Language Culture Food Hiking Landscape Friendliness Safety Rural tourism Night life Sports Addis Ababa Lalibela Axum Source: ESTDP Terminal Evaluation. Overall Awareness of the Activities Undertaken 13. In the surveys of international tourists and establishments, the interviewees were asked about their awareness and satisfaction with the new activities undertaken in the destinations, as summarized in Table 5.2. In Axum, tourists are aware of the activities undertaken in Tsion Church, the archeological museum, and the stelae field. However, they are not as aware of the walking 48 routes and the visitor center. The situation is similar in the case of establishments although not as marked. In Lalibela, the landscaping and improvement of heritage sites are well known to tourists, whereas the rest of the activities are not as well known. Establishments are aware of all the activities, except the visitor center, which is remarkably low. This is to be expected given that the visitor centers were not completed when the survey took place in November 2015. In any case, some complementary information such as signposts, brochures, or maps will be necessary to identify the key points of the destinations. Table 5.2. Awareness among Tourists and Establishments of the Activities Undertaken Lalibela Axum Addis Ababa Tourists Establishments Tourists Establishments Tourists Establishments (%) (%) (%) (%) (%) Tsion Church 46 – 82 83 57 – Archeological 46 – 92 81 55 – Axum museum Stelae field 52 – 95 78 57 – Walking routes 39 – 66 73 55 – Visitor Center 34 – 60 73 53 – Landscaping and 87 100 17 – 67 – improvement of heritage sites Lalibela Gravel road 61 85 10 – 62 – Bypass road 60 98 10 – 59 – Water supply and sanitary 66 92 20 – 57 – schemes Visitor center 65 16 7 – 62 – Source: ESTDP Terminal Evaluation, November 2015 Establishment Survey. 49 Annex 6. Summary of Borrower's Comments and ICR (a) Summary of Borrower’s Comments 1. The Borrower provided comments to a draft version of the ICR which were incorporated in the text; however, some comments remain outstanding (as summarized in Box 1 below). Box 1: Summary of Outstanding Borrower Issues/Comments Ratings:  The MoCT contends that the project should be rated Satisfactory based on the delivery of all the intermediate results within the implementation timeline (and with excellent financial utilization).  The MoCT disagrees with the rating provided for Government performance and supports a Satisfactory rating. The points supporting this rating include: improvement in expediting the approval process for contracts following the MTR, delays being attributed to lengthy procurement process of the Bank (rather than the Ministry) and turnover of leadership not having a significant impact on project implementation. Other Issues Raised:  Opinions of Beneficiaries - The government expressed concern that discussions with beneficiaries relied on personal opinions (for example, on the Resettlement compensation and quality and durability of the materials for the CIP beneficiaries) without full verification37.  Environmental Issues – The MoCT disagrees with the findings of the environmental safeguards assessment and the assertion that the project office did not consult with the relevant environmental authorities38. (b) Borrower’s ICR The full text of the Borrower’s ICR (excluding the data sheet and Annexes 1-3 covering Project Activity Implementation, Results Framework and Evaluation of the Matching Grant Scheme and Community Involvement interventions) is presented below. I. Executive Summary 2. The Ethiopian Sustainable Tourism Development Project ESTDP) has been claimed to be the first tourism project of its kind in Africa, perhaps on the ground of its scope of intervention. The inception of the project followed the paradigm shift made by the government of Ethiopia 37 The Bank’s ICR has sought to take into consideration both the views of the ESTDP beneficiaries (as documented through focus groups discussions) and the government. In some cases it was not possible to reconcile the differing viewpoints and revised language/footnotes are provided that provide further explanation. 38 The environmental assessment has relied on the “Back to office report, Environmental Performance of ESTDP” which highlights the issues referenced in section 2.4. 50 which was triggered after the publication of the article by the World Bank in titled” In Makeda’s Foot Step” which highlighted the potential and prospect for tourism development in the country. 3. The ESTDP was designed as a focused and strategic intervention attempting to address the critical bottlenecks of the tourism sector of the country. The scope of the project was limited to three key selected destinations of the country, vis., destination development, market development and capacity building. Lalibela, Axum and Addis Ababa and its environs. However, as a secondary intervention purpose, the project also envisaged supporting the development of other destinations in the country. 4. The ultimate goal of the project was to help Ethiopia diversify and broaden its economic base, increase its competitiveness and strengthen inclusive growth prospects and eventually contribute to poverty reduction as well as the realization of millennium development goals. The PDO was to enhance the quality and variety of tourism products and services in targeted destinations so as to increase the volume of tourism, foreign exchange earnings, and job creation. 5. The project has been concluded, after implementation period 2010-2015, with a high rate of implementation performance measured in terms of schedule maintenance and delivering on planned outputs. The project is also successful in its financial utilization 99.7% of its earmarked 35 Mill USD has been disbursed. Additional 28 Million ETB is also disbursed from the treasury of Government of Ethiopia. 6. With all implementation challenges and a partial suspension of the main component for 18 months, there are high achievements in the increase of visitor spending and increase in the number of tourism related jobs while there is also good improvements although it will not be achieved by the single effort of a project is beyond in tourism arrivals. Moreover, all the intermediate results of the project are significantly achieved or surpassed targets set. Therefore the project has been successfully implemented in a satisfactory rate. II. Project Context, Development Objectives and Design 7. The Ethiopian economy structure and performance along with development potential and policy of the country provides the context to the ESTDP project. Moreover, the significance of the tourism sector and the performance of the sector form another aspect of the context to the project. Ethiopia in recent years had one of the fastest growth rates in Africa. Yet, it is faced with an urgent need to accelerate sustainable growth and create jobs. 8. Accordingly, the Ethiopian economy is characterized by a predominantly agrarian based economy and at the same time with expanding GDP particularly during the past decade. The Ethiopian economy is also showing a strong trend of dominance of the service sector. According to the report published by the Ethiopian Economic Association, the hospitality sector is one of the few growth driver sub-sectors of the economy. 9. It is in recognition of the potential for the service sector of the country; particularly it is an appreciation of the relatively huge potential of the tourism industry of the country that the government of Ethiopia has increasingly attached strong importance to the tourism sector. The ESTDP project is conceived and implemented in line with this context of the project and envisaging a development objective “To enhance the quality and variety of tourism products and 51 services in targeted destinations so as to increase the volume of tourism, foreign exchange earnings, and jobs”. 10. The design of the ESTDP followed a programme theory approach and hence a logical system of intervention activities, outputs and long term results has been articulated. The Result Framework of the project well articulates the logical framework of the project. The design of the ESTDP project followed iterative steps of forming background information about the tourism sector of the country, identification of sector development problems and outlining of intervention areas aiming at formulating an integrated pilot level tourism sector development programme. III. Key Factors Affecting Implementation and Outcomes of the Project a. Partnership and Participation of key stakeholders 11. Partnership entails implementation arrangement whereby all key stakeholders work in coordination playing their respective role in the planning and execution of project activities with the aim of addressing the stake of each party and also to use the resource and competencies of members of the partnership. 12. Regarding ESTDP, the need for ensuring partnership emanates not just from the fact that it is just a development project, but primarily from the situation that tourism as a sector, and hence tourism development project, is a multi-institutional undertaking. In another words, tourism sector activities are normally undertaken by both the public and the private sector institutions. 13. The private sector institutions like the accommodation sector and the travel agencies, both represented by their respective associations, had been working as partners along with their public sector counterparts like the MOCT, in the conceptualization of the project idea and the follow up and support of the ESTDP intervention activities. The top management body of ESTDP with a role of providing strategy implementation guidance and oversight was called the steering committee. The committee constituted representatives from both the public, the Civil Society and the private sector at both the federal and regional administrative levels. 14. Similarly, at destination level, a Destination Implementation Committee (DPIC) was established for the each of the three project designated destinations, with the aim of ensuring coordination and monitoring of project activities at destination level. The DPIC committee constituted representatives of the local government offices and the private sector tourism service providers. 15. Regarding participation of key stakeholders, even though participatory implementation arrangement was not instituted by design like the partnership arrangement, however, the project office had shown commitment to involve key stakeholders by way of engaging them in the implementation of activities like ToRs preparations for consultancies and specification design for works and goods and eventually involving representatives of the concerned institutions in the selection process of consultants, suppliers and contractors. b. External Support and Oversight 16. For this particular project, external support and oversight was supposed to come from two key institutions, vis, the Ministry of Culture and Tourism and the World Bank. Therefore, the support offered and oversight done by the two institutions should be reviewed separately. 52 Borrower’s Performance 17. The Ministry was an official owner of the project and it was supposed to establish a project office to execute and coordinate project activities on a daily basis. While the project office was established to compensate for the implementation capacity limitation of the Ministry, however, the officially responsible government body for the provision of strategic supports and undertaking an oversight to project activity implementation and ensuring the project achieves its intended results remained to be the Ministry. 18. Accordingly, the Ministry was supposed to make certain institutional and organisational arrangements so as to ensure that the project gets the expected support (including advisory, guidance and informational…etc.) and that the Ministry carries out its oversight role. However, there was no such a formal and organized arrangement including assignment of responsibility within the organizational system of the Ministry. The obvious and regular engagement of the Ministry with the activities of the project office was when the Ministry reviews proposals for selection of consultant, supplier and contractor in the execution of project procurement activities. The other important engagement of the Ministry was when it coordinates and hosts the steering committee of the project and the Matching Grant steering committee for approving eligible matching grant beneficiary projects. 19. In summary, the most important gaps in the provision of support and providing oversight to the project implementation was that it was not institutionalized and that it was irregular. The ramifications of this gap was that identification of implementation problems and decision making on those problems were made after taking longer time than it should have been. Lender’s Evaluation 20. Regarding the Bank’s team, there was an extensive engagement in the project activities ranging from reviewing procurement activities to project activity schedule management. The team routinely and periodically followed up project activity implementation and this obviously helped the project implementation team to keep momentum when there was good performance and to take corrective measure when there was deviation from schedule. There are some indications that the project team thought that the engagement of the Bank’s team had been excessive and that there was sometimes negative perception. c. Key Issues Faced (focusing on safeguard issues and restructuring) 21. One most important newly emerged issue that came after the project implementation started was the resettlement programme of Lalibela. The resettlement programme of Lalibela was originally envisaged as a subsidiary activity for the project implementation whereby the project would be responsible only to oversight safeguard issues and to technically support the government to implement the resettlement programme. However, following the imposition of a partial disbursement ban by the Bank on the project, claiming mishandling of safeguard issue pertinent to the resettlement, the project office had to fully engage itself in the day-to-day follow up of the resettlement programme. This in turn strained the capacity of the project office and also partly diverted the attention of the project office away from implementation of its main activities. Despite all the challenges the RAP for the Lalibela project was planned and implemented timely and in participatory manner involving the various cross sections of affected community members. 53 22. In another front of emerging issues, the project had been subjected two times to resource re-allocation situations. The first budget restructuring was done during the mid-term evaluation of the project (2013) and the second as a project extension (2014) see Figure 1 below. The impact of these budgetary restructuring exercises were that they took considerable time for both preparing the budget restructuring proposals and for approval by the Bank’s higher management body. It is worth noting that once the budget restructuring process is initiated, the project office had to wait the final result and hence disbursements were disrupted during the whole budget restructuring preparation and approval period. In addition to the effect of loss of implementation time due to the budget restructuring processes, there was also another negative effect as a result of misallocation of budget for some activities by way of earmarking either surplus or deficit budget. d. Project Management Operational Performance 23. The project management operational performance should be measured in terms of the two most important factors, vis., project schedule maintenance and effectiveness in terms of executing all of the planned activities. 24. So as to evaluate the project management in terms of project schedule maintenance, one has to consider the fact that while the project was originally planned to be executed within a five year period, however, the actual time taken for the project implementation was six years. The project was granted one year extension time justified on account that one of the three components of the project was subjected to one year and seven months disbursement ban pertinent to the safeguard issue. Given the above background situation, the additional one year of implementation period taken by the project can be seen as a compensation for the lost implementation time due to the disbursement ban imposed. 25. Therefore, in view of the fact that the project office managed to execute almost all of its planned activities (except one which is the capacity building Training for tourism private sector workforce) and that the project time extension was a compensation for the lost time caused by a disbursement ban, it is proper and fair to suggest that the project management is successful in terms of project schedule maintenance. Here it is important to note that, in terms of effectiveness in maintaining the schedule of individual project activities, it is found that most of the project activities had taken longer time than they were supposed to take individually. 26. According to the project appraisal document and the subsequent plan revisions made, particularity during the mid-term review, the project office was supposed to execute 28 core project activities (considering resettlement programe support as one single core activity). Regarding the 54 implementation performance of these activities, at the end of the implementation period, it is found that the project office failed to execute only one activity. Therefore, in this second measure of operational performance of the project too, the project’s management is found to be operationally effective. IV. Implementation Performance (assessment of Outcomes against Project Objective) 27. The following discussion focuses on the performance of project in the major intervention areas of the project and the evaluation attempts to inform whether the project has achieved outcome level results and if this has eventually contributed to the achievement of the project development objective (for details see the Annexes). a. Destination Development 28. The project envisaged to develop three destinations, vis., Lalibela, Axum and Addis Ababa and environs with more focus given to the first two destinations. The strategy adopted to bring about destination development were rehabilitation of access roads and walkways, landscape development of the sites, installation of lighting system, improvement of water supply scheme (only for Lalibela) and sanitary facilities, establishment of visitors centers and undertaking product development activities. 29. Accordingly, a total of 21km road has been rehabilitated in Lalibela and Axum and the roads are also provided with improved landscape and lighting system. The water supply system of Lalibela has been enhanced and improved to serve the local and tourism population of the city sufficiently for the coming 15 years. The current total population of Lalibela is estimated to be 19,500. Provision of sanitary facilities has been also made particularly in Lalibela which include construction of land fill and construction of a number of toilets (more than 18 toilets). 30. The improvement of stelae field and Paving of plazas, rehabilitation of Aksum Tsion Church and refurbishing of old houses in Aksum have been implemented. Two Museums have been rehabilitated in Aksum and Addis Ababa destinations along with historical church rehabilitation in Axum. The new visitor center in Aksum will serve as a model building design, aesthetics, functionality, location and connection with other tourist services for other tourist destinations in the country and furniture were provided to enhance the visitor centers in Aksum, Lalibela and Addis Ababa. Product development plans preparation for western, eastern and southern routes were also done to create bases to develop new tourism areas in the country. The feasibility study of Entoto City- View has been conducted to develop and diversify the tourism offer around Addis Ababa which have caused the establishment of project implementation office by Addis Ababa City administration. Tourism development plans for Tiya Megalithic world heritage site and Melka Quntre prehistoric archaeological sites have been developed and Museums and Community centers have been constructed. Two solid waste trucks and two liquid waste trucks have been given to Lalibela and Aksum town administrations. 31. With regard to outcome of the achievement of the above destination development outputs and the contribution of these outputs to the achievement of the project development objectives of the project, it is not possible to observe the achievement of these higher level results due to the fact that most of these outputs are delivered at the end of the project implementation period. 55 b. Market Development 32. A marketing strategy design and branding proposal of Ethiopia as tourism destination and marketing strategies for five regions have been prepared and a tourism promotion system and package of materials was developed that includes printed materials (15 types of brochures), multimedia and digital content and dynamic and interactive webpage development. Although these activities will have significant impacts on the success of promotional campaigns and repositioning Ethiopia as a destination, all these activities have been executed at the end of the project life time and hence it would be premature to discuss the outcome of these outputs or their contribution at this stage. Familiarization trip- international study tour were done in two groups: the first group consisted of 12 officials visited Malaysia and Singapore, and the second group consisted of 19 officials have visited Turkey. This study tours provided opportunities for the tourism sector officials to benchmark other successful tourism destination in Asia. c. Institutional Development 33. A number of intervention activities have been executed under this component. With the aim of enhancing the institutional and human capacity of the of the tourism sector governing and regulatory institutions at all level, technical and managerial skill development trainings have been provided to 163 staff of the Ministry of Culture and Tourism (MOCT) and similar institutions at regional and local levels. The training provision is also couple with a provision of office equipment. For institutional capacity development, system of tourism statistics has been developed and deployed for MOCT. A grading scheme has been designed and prepared and has been also used to grade 378 hospitality facilities in the country using the expertise of the UNWTO and around 54 local assessors trained in hotel grading to transfer the international experts skills and knowledge to their local counterparts. Preparation of the national Tourism Implementation Framework has been developed. Supports in training 44 instructors from four tourism training center and TVETS in the areas: Food Preparation and presentation, Food safety and Hygiene, Housekeeping, Tourism Marketing, Tour Planning and Operation of Oracle Database and SQL database, and kitchen equipment and reference books have also been given to selected tourism training institutions to enhance their capacity. Moreover, the project supported the establishment of the tourism transformation council, the tourism board and the Ethiopian Tourism organization. By the same token, the outcome of the achievement of the above outputs has not matured by the time the project ends due to late hour implementation of these activities. d. The Matching Grant Scheme 34. The Matching Grant scheme of the ESTDP is introduced in response to the fact that tourism sector is basically a private sector undertaking whereby most of the services rendered to tourists come from the private sector operators. Tourism services, particularly the hospitality sector and the tour and travel services are delivered at a poorly rated standard in Ethiopia. The ESTDP project through its matching grant scheme attempted to enhance the tourism services rendered by the private sector operators in the three project designated destinations through the provision of technical and financial supports. 35. The matching grant scheme includes critical processes: promotion of the scheme, handholding/ business advisory service to beneficiaries, review & evaluation of business proposals, approval of proposals, implementation support and monitoring, and payment of reimbursement claims. In total, 139 applications were received for the matching grant scheme in the life time of the project. Out of which 91 projects were accepted for further assessment and 79 projects were 56 on track until the end of the project and able to completed majority their agreed activities with the ESTDP- Matching grant scheme. 24 (30.38%) are hotels, 20 (25.32%) are restaurant,14 (17.72%) are tour operators,7(8.86%) are tourism promotion and marketing, 6(7.59%) are agribusiness,5(6.33%) are hotel and tourism training, and 3(3.8%) are others like crafts, entertainment. Reimbursement Status ETB USD % Vs total fund Total MGT fund 45,671,280 2,400,000 100 Paid 40,461,003 2,126,203 89% Committed but not paid 2,252,717 118,379 5% To be Committed ( projects under approval process) 0.00 0 0% Uncommitted 2,957,560 155,418 6% Total committed 42,713,720 2,244,582 94% 1 USD =19.0297 ETB 36. The outcome evaluation of matching grant was carried out on 54 firms who had completed their projects and get reimbursement. The evaluation result demonstrates the improvement in the firm’s products/services diversification and quality, management system, business performance, and socio-economic contribution. The findings of the evaluation also reveals that most of the MGS beneficiary firms who developed websites, and tour operators and some craft centers who participate in national and international trade fairs has get chance to promote their local destination and/or nation along with their products/services. Furthermore, the evaluation report identifies upgrading employee skills as a further important aspect of MGS intervention. In total, 1,781 employees of MGS beneficiary firms received training, including both management and employees. Of these, 983 were women. e. Community Involvement 37. Communities living in areas of tourism destinations are key stakeholders of tourism activities because they are either affected by or they affect tourism activities in one or another way. Depending on the management of destinations in general and the tourism industry in particular, tourism could potentially either improve or aggravate poverty. The communities living in the ESTDP designated destinations, particularly those living in Lalibela are predominantly poor. Hence cognizant of this situation, the project envisaged to in tourism related economic activities. 38. The implementation of community involvement package had critical process of promotion, hand-holding service in the preparation of proposals, evaluation of business plan, and approval of business plan preparation and signing of the Community Involvement Funding Agreement (CIFA), monitoring performance, and training and in-kind support. 39. Totally, 54 Funding Agreements were signed and implemented within life time of the project. The Funding Agreements were signed with 54 business ventures (20 from Lalibela, 26 from Axum and 8 from Addis Ababa & environs). The business ventures were composed of 57 different business types. The largest number of business venture is that handicraft with 16 (30%) of the total established business ventures. Then followed by agri-business with 13(24%), catering with 8(15%), local tour guides and ecotourism with 7(13%), cultural entertainment with 4(7%), tourism promotion 3(6%), Sanitary service & greenery works 2(4%), and local transport 1(2%) respectively. 40. A survey was done on the outcome of community engagement package. The evaluation focused on job creation and strengthening livelihood, strengthening tourism products/services, and growth and sustainability of the business ventures. The findings of the evaluation are summarized here to imply short term outcomes of the intervention 41. The established 54 community based business ventures are running their businesses related to the tourism industry. The supports provided to the community members included training, materials/equipment provision and provision of backstopping services a preliminary evaluation also indicated that the program has positively impacted the life of the community members. f. Resettlement and Livelihood Restoration in Lalibela 42. The Implementation Completion Audit done by an independent individual consultant confirmed that the Project has managed to implement all its project activities and achieve its intended results as envisaged in the proposal and in line with its agreement with the World Bank. The pre-relocation phase of the RAP implementation, which had involved land acquisition, budget allocation for resettlement site preparation works and design of the procedure for compensation payment was successfully accomplished with demonstrated strong collaboration of the Project office, the Government of Ethiopia and the communities at large. 43. All the houses of 547 PAPs, who used to live in the church compound (core Zone), were demolished and the residents from the core-zone of Adishadie, Chfirgoch, and Gebrielsefer were relocated to Kurakur. It was evident for every visitor that the relocation of these households has created more open space, increased the sanctity of the churches and the cultural heritages. The avoidance of the encroachment of the compounds of the churches undoubtedly reduced the direct physical threat to the sustainability of the structures of the centuries old rock-hewn churches. The construction of basic infrastructure work and the upgrading of the social services within the cultural Zone have contributed positively to improve the environmental setting of the churches and their surroundings, which in the long-run will increase the duration of stays for both domestic and international tourists. 44. Community involvement was very strong and instrumental to create sufficient understanding and implement smooth resettlement program. The establishments of a resettlement committee and a grievance hearing committee, which composed of genuinely elected members of affected communities and representatives of local religious leaders / clergies, were found instrumental to ensure the participation of the project affected people. Therefore, unlike many urban planning and settlements projects elsewhere in Ethiopia, the Resettlement Action Plan for the Lalibela project was planned and implemented timely and in participatory manner involving the various cross sections of affected community members. 58 45. The implementation of the land acquisition process for the ESTDP in Lalibela had been guided by the relevant government laws and procedures, which mostly were in line with the policies of the World Bank, especially, OP 4.12. 46. The property valuation method which was adopted for the project pursued the cost replacement method using costs for each unit rates for components estimated according to market value. The Project Affected People have managed to utilize the compensation payment for the re- construction of their houses. The amount of the compensation was felt by the beneficiaries themselves as fair and realistic to the then market prices of construction materials. 47. The Project has relied upon community level structure to address Grievance issues. The presence of community level GHC was possible to address a total of 328 cases of complaints, free of charges. Out of these, 257 cases were resolved, whereas 71 cases were rejected since the claims were found inappropriate in light of the rules and the procedure for resettlement. The key lesson from the successful deliberation of the grievance hearing process was the fact that the election of trusted members of GHC, the presence of strong team spirit among the committee members, the creation of trustful working relationships between the GHC and local city administration bodies and the devolving of decision making power at grassroots level were fundamental to successfully handle grievances and ensure maximum benefits for people affected by the project. 48. The Project has managed to implement both individual and groups- based business activities to restore the livelihoods of persons who were engaged actively in trade, agriculture, service and construction. It was learned that beneficiaries that have had business experiences and that have continued or upgraded the business that they knew earlier have achieved greater success than those who have had no experience or those who were unemployed during the pre-Project period. Group based income generating operators were generally found to have less success stories than those individual business operators. This was partly attributed to lack of strong team spirit, limited experiences in groups business and the administrative complications associated to the management of cooperative organizations. 49. The Project had concrete strategy to support socially vulnerable persons and safe guard their livelihoods. It has managed to enter a contract agreement with Save Your Holy Land Association /SYHLA/ and supported 160 older people and people with disabilities. It is advisable that the Project, the Lalibela Town Administration and SYHLA explore opportunities for reducing the risks and assuring the continuity of food supplies and other basic social care and support services for these destitute people. V. Disbursement Performance 50. The overall project financial performance is considered high. Although there was low disbursement during the partial suspension period, there has been significant progress after the lifting of the suspension. Hence, the budget utilization at the end of April 2016 is 99.7% disbursed. Moreover, another additional 28 Million Birr were secured from the treasury of the GoE. VI. Implementation Challenges Faced 51. The implementation challenges observed during the implementation period of the project which constrained the smooth implementation proceeding of the project are outlined as follows: 59  Too much engagement in the follow-up of the resettlement programme over stretched the capacity of the project and at least partially diverted the attention of the project staff away from the main business of project activity implementation.  The partial disbursement ban imposed on component one of the project, for which larger chunk of the project resource was allotted, had strong negative impact on maintaining project schedule at activity level.  The low organizational capacity of partner organizations, from which collaboration and support was expected by the project office, had created a gap in project activity implementation support system.  The basic project document of ESTDP did not provide sufficient information and articulation of the concept, content, scope and cost of the intervention activities of the project and this information gap created a situation that there were a lot of discussions between the Bank’s team and the project office during ToRs preparation and consequently a lot of time was lost before clearance was obtained for the ToR supposedly contributing to delayed procurements.  Managing the Inflated expectations of the stakeholders that this project will solve all the pertinent problems exist in the tourism sector. VII. Lessons Learned and Proposed Arrangement for Sustainability of Results The most important lessons learned from the implementation and result of the ESTDP project is:  If implementation of project activities is pushed to the end of the project implementation period, either some of the activities will not be executed or at least the impact of the executed activities cannot be observed at the end of the project.  The preparation of further detailed activity strategy is critical to understand the knowhow of the strategy designers in the PAD and those responsible for the execution.  The deficiency in the planning stage has implication on implementation as it has been witnessed in ESTDP with low emphasis given to resettlement programme during planning and also the limited information given in the PAD about the project activities scope, content and cost.  The capacity of a project office should be augmented through a system of partnership and implementation support.  Managing inflated expectations must be considered as an important element to maintain the project’s vitality, scope and interest in its development and implementation.  The necessity of smooth institutional collaborations is critical. The presence of strong working relationships and smooth institutional coordination among the various locally instituted community structures, city administration bodies and a project office is fundamental to successfully handle resettlement and infrastructural development program.  Strong and well organized contract management system had to be developed to manage quality, the cost and time overrun in delay of implementation.  Even though the matching grant scheme is a demand-driven intervention it would have been more efficient and effective to have detail gap analysis and opportunity identification of the private sector in the tourism value chain of the target destinations before implementation. The gap analysis and opportunity identification would help the private sector to come up with more feasible business plan based on realistic gap in 60 their respective business operation. It would also provide prior information about the level of capacity of the private sector to implement the matching grant scheme.  There was no clear direction on how the community engagement should have been implemented at the initial stage of project implementation period. An international consultant was commissioned to undertake rapid gap assessment and opportunity identification as well as Implementation Manual after two years of initiating the community involvement package. It would have been better if this has been done during the design stage of the project. 61 Annex 7. List of Supporting Documents  ESTDP Terminal Evaluation  Matching Grant and CIP Performance Evaluation Reports  Promotional Materials Report  Solimar Training Report  Hotel Grading Report  Implementation Completion Audit for the RAP  Project Implementation Supervision Reports  Project Aide Memoires  Back to Office Report, Environmental Performance of Ethiopia’s Sustainable Tourism Development Project 62 Annex 8. Overview of Lalibela Resettlement Background and Context 1. At the time of project appraisal, an RPF was prepared that included a profile of the community in Lalibela highlighting the main social safeguards issues, in particular the need for relocation of households in the core zone of the Lalibela church site. The rationale for the relocation was to protect the gradual encroachment of the church compounds by an increasing number of inhabitants (posing a threat to the physical structure of the cultural heritages and resulting in underground leakage of salty water from septic tanks and the flow of dirty water). 2. Before the project, the Lalibela town had already taken some administrative measures to reduce the pressure on the physical structure of the physical heritage by suspending the construction of houses within the cultural heritage sites and imposing a housing construction moratorium on the residents of the core zone. Under the implementation of the town plan and administrative measures of the Lalibela city (backed by the Amhara National Regional State), the Lalibela City Administration had relocated 138 households in 2009 from the compound of the churches in Michael Gibi. 3. During the design phase of the ESTDP, the task team supported the view that the terms of reference for the RAP should be expanded to cover a wider range of social issues than is normally expected for a RAP, rather than commissioning a separate social analysis study for Lalibela. Figure 8.1. Lalibela Resettlement Timeline Partial  Project  Start  suspension  Lifting of the  closes  February  November  suspension  December  2010 2010 June 2012 2015 Bank team  RAP approval and  RAP completed  learns of  disclosure and ICA finalized  displacement of  September 2011 November 2015 138 households  April 2010 Project Suspension 4. Following effectiveness of the operation on February 25, 2010, a project launch mission was held in April 2010 when the Bank team learned that 138 households (98 of these households were homeowners and 40 households were kebele tenants) had been displaced without an appropriate RAP, contrary to the safeguards provisions in the Financial Agreement (Schedule 2) and PAD. As a result of noncompliance with the Bank’s social safeguards provisions, the Bank decided to suspend disbursements in November 2010 in relation to Component 1, Destination Development, until the recipient submitted evidence of compliance with its obligations under the Financing Agreement, including submission of a satisfactory RAP. 63 5. A RAP was prepared and approved in September 2011, 10 months after the project had been partially suspended and was fully disclosed in Ethiopia and Washington, D.C., from September to October 2011. At that time the estimated cost of financing the RAP was US$5,212,670, which, it was agreed, would be funded by the client and the project (US$1,193,270 to be funded by the project and US$4,019,165 to be funded by the client). 6. Following efforts at the municipal, zonal, and federal levels, the Government met the agreed conditions required for lifting the suspension39 and through a June 1, 2012, letter to the Bank, the MoCT announced completion of the activities and requested that a Bank safeguards specialist view the progress and provide the necessary technical recommendations for the lifting of the partial suspension. Following the assessment that took place in June 2012, the suspension was lifted. The Resettlement Process 7. The Lalibela resettlement included a wide range of actions that were carried out from September 2011 until 2015, including land acquisition, property valuation, compensation payments to people affected by the project, grievance redressal, provision of basic infrastructure, and granting of land and building permits. Table 8.1 summarizes the number of PAPs and resulting payments/actions. Based on the ICA, the project resulted in a total of 2,915 newly settled people at Kurakur. The RAP divided the Lalibela core zone residents into three main groups: (a) real property owners, (b) kebele house tenants, and (c) tenants. Group (a) was entitled for compensation for loss of property and assets (mostly dwelling houses, business premises, fruit bearing trees, and other timber) and for the loss of income and livelihood. Group (b) was entitled to a replacement house or plot of land for house construction, while group (c) was entitled to a disturbance allowance. Table 8.1. Overview of Lalibela Resettlement Actions Total Number of PAPs Action Affected and Payments Made Approximately US$2.2 million (ETB 44,727,809) was paid to 307 PAPs Compensation payment entitled for compensation. Land titles were issued and new houses were built on the relocation site of Physical relocation of PAPs and Kurakur by PAPs according to building permits issued by the Office of assistance with land titles and Municipal Services. A total of 547 households moved from the Lalibela core building permits for new houses zone, and 531 eligible PAPs obtained land titles and building permits and constructed new houses in the resettlement site. A total of 670 PAPs benefitted from support in the form of materials/equipment provisions, business skill development/upgrading, and provision of working and marketing premises. This scheme was meant to ensure that the livelihoods of Livelihood and income restoration people affected by the project were restored at least to levels prevailing before the project. Of the 670 PAPs, 510 people were organized into cooperatives (245 individuals) and individual business owners (265 individuals). 39 The conditions included (a) launching of the RAP in Lalibela at a gathering of stakeholders; (b) compensation to the host farmer for the acquisition of farm land for the resettlement site to be paid; (c) clearing, demarcation, parceling, and allocation of plots through a lottery and issuing of land titles. 64 Total Number of PAPs Action Affected and Payments Made The project entered into a contract agreement with Save Your Holy Land Safeguarding livelihoods of Association to support 160 persons identified as vulnerable (including persons socially vulnerable persons who live with disability and older people ages 65 and above). 8. In addition to the abovementioned actions, the resettlement process included provisions for basic infrastructure in the resettlement site and basic amenities for the relocated PAPs. Activities undertaken included basic physical infrastructure and utilities that included the construction of gravel roads, water supply, electricity, storm water drainage, retaining walls, and sanitation facilities. Findings of the ICA and Lessons Learned 9. The project commissioned an ICA of the resettlement in November 2015 to evaluate the implementation and achievements of the resettlement/safeguard instruments, identify challenges, and document lessons learned from the project for future design. The audit confirmed the completion of all the planned resettlement activities and achievement of intended results in line with the Bank policies (especially OP 4.10, OP 4.11 and OP 4.12). The audit further confirmed the participatory manner in which the resettlement was carried out and highlighted the reduced threat to the sustainability of the structures in Lalibela. A review of the audit was carried out by the Bank safeguards team and found to be acceptable. 10. The following are some key lessons learned, as summarized from the audit:  Importance of community involvement and community-based structures/committees. Community involvement and extensive public awareness creation activities were fundamental to the relocation process. The establishment of a community-based resettlement committee and grievance committee were instrumental to resolve disputes and ensure smooth implementation.  Effectiveness of ‘cost replacement’ model of compensation. The cost replacement model of compensation, relying on market values for lost materials or assets in old settlements, considers the opportunity cost of relocation and takes into account the market value of the new site. This was found to be effective in helping PAPs reinstate or replace their houses.  High level of institutional coordination. The coordination of local community leaders, city administration bodies, the project office, and the Government was essential for implementation.  Effective targeting and gestation time for business support. The income generation support program required preparation time to identify the appropriate target members who were both active and committed to start up new businesses or upgrade their existing businesses. Beneficiaries who demonstrate initiatives such as concrete business plans, regular compulsory or voluntary savings, experience in undertaking some sort of business activities, social acceptance or personal integrity, and other basic minimum requirements should be targeted for these schemes. 65 PAP Views on the Resettlement Process 11. In line with the completion of the project’s terminal evaluation, focus group discussions were held in November 2015 with the resettled households to better understand views regarding the implementation process and overall satisfaction with the resettlement area. The participants included 13 heads of households that were resettled due to the project interventions. A summary of the findings of the discussion is as follows:  In general, the group interviewed agreed that there was a net benefit resulting from the resettlement, especially in consideration of the state of the resettlement communities’ previous homes, which were cluttered, vulnerable to wind and rain, and lacking sanitation. The group agreed that they now enjoy new houses (though unfinished in most cases) with better sanitation, clean water, and a better neighborhood. The neighborhood is further from the main town, making it more difficult to access the main market place, hospitals, and schools and resulting in increased transport costs.  Regarding compensation (both in kind and in cash). Most of the participants felt that the compensation was not enough and indicated that the project had not adequately considered rising prices, due to the crowding out effect of the resettlement in the market. Some participants complained that the land plots were smaller than initially expected and that they were expected to incur an additional cost of ETB 12,000 (approximately US$650 per year to pay the land lease).  Infrastructure. Respondents indicated that the infrastructure built in the new village were of low quality. For example, the PVC pipes are destroyed by pressure causing occasional water shortages.  Livelihood restoration. Participants confirmed that they were given the material support they required to generate income for their livelihood; however, there were some problems with the quality of the equipment/materials purchased by the project. Some of the equipment malfunctioned and was later rendered unusable.  Overall planning and implementation. Despite the consultative process during the resettlement, participants pointed out that there was a lack of transparency in the planning and implementation of activities. On the other hand, all participants agreed that the project had visibly changed the tourism landscape of Lalibela and should continue in the future. 66 Annex 9. Detailed Assessment of Implementation Factors Table 9.1. Summary of Key Implementation Factors for the ESTDP Key Implementation Factors for the ESTDP  Recruitment of new and competent staff for the matching grant, strengthening/hiring of more personnel assigned to the CIP and financial management and procurement roles within the PCU and SMU  Capacity building and TA eventually built into project interventions, including for the PCU and SMU Factors contributing to  Responsiveness to the challenges of implementation, including two (level II) implementation success restructurings in 2013 and 2014  Close monitoring and supervision of project activities by the Bank team, resulting in adequate identification of needs and staffing to resolve problems  Destination committees (in addition to the overall PSC) made up of diverse stakeholders (community members, private sector, association members, church priests, etc.) at each of the sites  Resettlement in Lalibela, resulting in partial suspension of Component 1 for a year and a half  Capacity of the MoCT and regional governments and varying levels of government commitment at different project sites among different stakeholders (local, regional, and federal)  Slow approval for procurement and budget, resulting in delay of project activities Factors contributing to and timelines implementation delays  Weak contract management that contributed to lack of effective planning for escalating real prices and costs  Protracted and inefficient procurement processes  High staff PCU/SMU turnover, resulting in loss of technical expertise  Institutional weakness of the implementing agency/lack of experience managing projects of this nature and specialized staff to deal with the different areas of the intervention Key Factors Contributing to Implementation Success/Achievement of the PDO  MTR and first level II restructuring. The MTR carried out from October 22 to November 2, 2012, sought to address the project’s implementation challenges and to realign resources with the remaining time and budget to inform the planned restructuring of the project. The main findings of the MTR were that, while the main components of the project did not need to be significantly modified, the disbursement rate (at 17 percent) was too slow and the project funds needed to be realigned with greater realism regarding what could be done with the remaining time and budget. The MTR further identified the need to increase participation and effectiveness of the PSC through a clarification of roles, delegation of responsibility to the PCU, and addressing workforce capacity shortfalls in the SMU of Addis Ababa and its environs. The recommendations of the MTR were taken forward during the project’s first level II restructuring in March 2013, in which the disbursement categories and outcome indicators were realigned. The key features of the restructuring included an increase in the allocation for goods, works, and operating costs, a reduction in services, and a revised allocation for the matching grant scheme. Furthermore, the restructuring resulted in the elimination of two of the project’s intermediate results indicators to 67 correspond with project activities that were dropped.40 Additionally, the PDO-level baselines and targets were revised as were the targets for the intermediate results indicators. This was largely a result of the baseline survey carried out in 2012.  Second restructuring. The second restructuring in March 2014 provided for a one- year extension of the project closing date from December 30, 2014, to December 30, 2015. An action plan and timeline were submitted for the remainder of the implementation period as a condition for approval of the extension by the Bank.  Recruitment of new and competent staff. New staff for the matching grant and the financial management and procurement roles within the PCU and SMU, in addition to specialists embedded by the Bank (including procurement and tourism specialists) helped mitigate high turnover and capacity risks encountered during implementation.  Destination committees. Destination committees (in addition to the overall PSC) were established and made up of diverse stakeholders (community members, private sector, association members, church priests, and so on) at each of the sites. Active participation of the committees helped ensure the resolution of the project’s key challenges (in particular for the resettlement in Lalibela). Key Implementation Challenges  Partial suspension of the ESTDP. On November 6, 2010, due to the resettlement of 138 households in Lalibela before the completion of a RAP, a partial suspension of project activities and disbursement of Bank funds (for Component 1) was put in place (annex 8 covers the Lalibela resettlement and the events leading up to the suspension). Although physical works interventions were suspended across all project destinations, the project was granted an exemption to continue with eight technical design and services consultancies supporting the main rehabilitation and enhancement of sites and visitor services in the three target destinations. The lifting of the suspension and subsequent implementation of the RAP was met with some delay, as the initial RAP had to be resubmitted, the physical preparation of the resettlement site was slow, and there was a shortage in the availability of government funds to enable the smooth progress of activities. The resettlement was eventually successfully implemented. However, the main consequence of the partial suspension was the shortening of the delivery time frame for a complex set of civil works activities at all sites and diversion of PCU/GoE attention away from the implementation of the other project components.  High turnover and capacity of PCU and SMU Staff. During the course of implementation, seven PCU/SMU staff and consultants resigned, passed away, and/or had to be replaced, resulting in a loss of technical capacity and experience.41 Moreover, the need for technical/capacity training of project staff was underestimated during 40 Specifically on the number of trails with signage in selected destinations and on number of travel service providers using the e-travel platform. 41 This included the PCU project coordinator, PCU procurement specialist, SMU manager in Axum, SMU coordinator in Lalibela, Lalibela safeguards specialist, matching grant and community intervention specialist, and international tourism consultant. 68 project design, as it became evident that project staff required additional training related to the Bank’s guidelines, processes, and procedures in addition to procurement. Training was eventually carried out for the PCU/SMU and a senior tourism consultant was identified as critical and on-boarded to support the project staff.  Delays resulting from procurement and contract management. Limited capacity in contract management and procurement resulted in project delays, including in finalizing the consultancy service for the development of a marketing strategy and baseline study and completing the marketing and promotion materials for the matching grant scheme. Delays persisted even after the lifting of the project suspension up until project closure. The key challenges identified are summarized further in section 2.4.  Escalating real prices and underestimation of budget to complete activities. In the face of escalating real prices, the budget established in the PAD had to be revised and readjusted based on the project’s planned activities. Furthermore, the designs for project works were more complicated and more expensive than initially envisioned. As a result, some activities had to be eliminated due to the limited budget allotted for the project. 69 MAP Source: World Bank Map Design Unit 70