Document of The World Bank Report No: ICR2719 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-93591) ON A GRANT IN THE AMOUNT OF US$ 2.903 MILLION TO THE CLEAN AIR INSTITUTE (CAI) FOR A GEF SUSTAINABLE TRANSPORT AND AIR QUALITY PROGRAM December 20, 2013 Sustainable Development Department Latin America and the Caribbean Region Sector CURRENCY EQUIVALENTS Currency Unit = US$ FISCAL YEAR [January 1 – December 31] ABBREVIATIONS AND ACRONYMS APL Adaptable Program Loan BRT Bus Rapid Transit CAI Clean Air Institute CAI-LAC Clean Air Initiative for Latin American Cities CO2 Carbon Dioxide CPS Country Partnership Strategy FM Financial Management GEF Global Environment Facility GHG Greenhouse Gas ICR Implementation Completion and Results Report NMT Non-motorized Transport OP Operation Policy PD Project Document PDO Project Development Objective PIA Project Implementing Agency STAQ Program GEF Sustainable Transport and Air Quality Program TDM Travel Demand Management TOR Terms of Reference VKT Vehicle Kilometers of Travel WRI World Resources Institute Vice President: Hasan Tuluy Sector Director: Ede Jorge Ijjasz-Vasquez Sector Manager: Aurelio Menéndez Project Team Leader: Anca Cristina Dumitrescu ICR Team Leader: Anca Cristina Dumitrescu LATIN AMERICA GEF Sustainable Transport and Air Quality Program CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Global Environment Objectives and Design ................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 3. Assessment of Outcomes .......................................................................................... 12 4. Assessment of Risk to Development Outcome......................................................... 19 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 22 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 23 Annex 1. Project Costs and Financing .......................................................................... 24 Annex 2. Outputs by Component ................................................................................. 25 Annex 3. Bank Lending and Implementation Support/Supervision Processes ............ 32 Annex 4. Summary of Borrower's ICR ......................................................................... 34 Annex 5. Knowledge Platform Information…………………………………………..45 Annex 6. List of Supporting Documents ...................................................................... 46 A. Basic Information LAC Regional Country: Latin America Project Name: Sustainable Transport and Air Quality Project Project ID: P096017 L/C/TF Number(s): TF-93591 ICR Date: 12/20/2013 ICR Type: Core ICR CLEAN AIR Lending Instrument: APL Borrower: INSTITUTE Original Total USD 2.90M Disbursed Amount: USD 2.86M Commitment: Revised Amount: USD 2.86M Environmental Category: B Global Focal Area: C Implementing Agencies: Clean Air Institute Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 04/08/2005 Effectiveness: 04/14/2009 06/25/2009 05/28/2009 Appraisal: 03/06/2008 Restructuring(s): 09/10/2012 Approval: 10/21/2008 Mid-term Review: 06/30/2011 10/12/2011 Closing: 12/31/2012 06/30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Global Environment Outcome Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Not Applicable Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Not Applicable Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): GEO rating before Moderately Closing/Inactive status Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Public administration- Transportation 100 100 Theme Code (as % of total Bank financing) City-wide Infrastructure and Service Delivery 10 10 Climate change 60 60 Environmental policies and institutions 10 10 Pollution management and environmental health 20 20 E. Bank Staff Positions At ICR At Approval Vice President: Hasan A. Tuluy Pamela Cox Country Director: Ede Jorge Ijjasz-Vasquez Laura Tuck Sector Manager: Aurelio Menendez Jose Luis Irigoyen Project Team Leader: Anca Cristina Dumitrescu Paul Procee ICR Team Leader: Anca Cristina Dumitrescu ICR Primary Author: Melanie Glass Katya Kidba ii F. Results Framework Analysis Global Environment Objectives (GEO) and Key Indicators(as approved) Regional project development objectives will be: (1) to establish a network of local and national government stakeholders, international organizations and private sector entities to promote policies and actions leading towards more energy efficient and cleaner urban transport systems in Latin American cities; (2) to assist cities to develop sustainable urban transport strategies that integrate climate change and air quality components; and, (3) to improve the capacity of cities to quantify the impacts of transport policies on climate change and air pollution emissions. Revised Global Environment Objectives (as approved by original approving authority) and Key Indicators and reasons/justifications (a) GEO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Formal support and a work plan financed by different sources to coordinate Indicator 1 : activities at the regional level, involving local and national governments, international organizations and private sector Endorsement of the Formal support work plan by 34 Value No formal support and a and a work plan entities, and (quantitative or work plan financed by financed by financial support Qualitative) different sources different sources contributed by 5 of them. Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments Target partially achieved: only 5 out of the 34 endorsements from network (incl. % members are accompanied by in-kind contributions, raising questions on the long achievement) term sustainability of CAI. Cities with a strategy to reduce CO2 through sustainable transport strategies, including regulatory and financial frameworks that foster sustainable transport Indicator 2 : systems at local and national level Cities with a 11 city case studies, No city with a strategy to Value strategy to reduce reports and reduce CO2 through (quantitative or CO2 through guidelines were sustainable transport Qualitative) sustainable produced to help strategies transport strategies reduce CO2 Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % Target achieved (100%) achievement) iii Better understanding of the local and global environmental impacts of different Indicator 3 : transport policy and technology strategies in cities Better No associated understanding of Value No good understanding of indicators or the local and (quantitative or the local and global targets; deficiencies global Qualitative) environmental impacts in the selection of environmental this indicator impacts Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % Target could not be objectively measured. achievement) Cities applying assessment tools to quantify greenhouse gas and air quality Indicator 4 : impacts of select transport options Value More than 3 cities No city applies Cities applying (quantitative or applied the assessment tools assessment tools Qualitative) assessment tools Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % Target achieved (100%) achievement) Availability and actual application of tools for local and national decision makers Indicator 5 : and project developers to assess and compare transport policies and investments in terms of climate and air quality benefits Tool available Value Availability and through the CAI No availability and actual (quantitative or actual application Helpdesk, training, application of tools Qualitative) of tools technical assitance and workshops Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % Target achieved (100%) achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Number of agreement letters from local and national governments, international Indicator 1 : organizations and private sector formally endorsing the program Value (quantitative or 0 30 34 Qualitative) Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments Partially achieved: the number was exceeded, but only 5 of the 34 agreement iv (incl. % letters came with financial or in-kind contributions. achievement) Amount of additional resources leveraged by the GEF Program Indicator 2 : through the CAI Value (quantitative or 0 US$ 2,093,000 US$ 2,200,000 Qualitative) Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % 100% achieved achievement) Indicator 3 : Number of staff working at CAI to manage the work plan Value (quantitative or 0 4 4 Qualitative) Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % 100% achieved achievement) Number of case-studies, reports and guidelines on strategies to reduce C02 by Indicator 4 : sustainable urban transport program disseminated in the region 11 case studies, reports and Value guidelines were (quantitative or 0 11 produced and Qualitative) disseminated by CAI Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % 100% achieved achievement) Number of trainees that have taken the sustainable transport, Indicator 5 : climate change and air quality courses offered by the program Value (quantitative or 0 500 1,333 Qualitative) Date achieved 09/25/2008 06/25/2009 06/30/2013 Comments (incl. % 100% achieved achievement) Number of cities applying tools and methodologies available to assess transport Indicator 6 : policies and investments in terms of climate and air quality benefits at city and project level Toolkit and Value Toolkit developed methodologies (quantitative or 0 and applied in 3 applied in more Qualitative) cities than 3 cities Date achieved 09/25/2008 06/25/2009 06/30/2013 v Comments (incl. % 100% achieved achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. GEO IP Disbursements Archived (USD millions) 1 10/02/2009 Satisfactory Satisfactory 0.30 2 05/21/2010 Moderately Satisfactory Moderately Satisfactory 0.46 3 02/08/2011 Moderately Satisfactory Moderately Satisfactory 1.08 Moderately 4 07/27/2011 Moderately Satisfactory 1.52 Unsatisfactory 5 12/05/2011 Moderately Satisfactory Moderately Satisfactory 1.80 6 06/20/2012 Moderately Satisfactory Satisfactory 2.12 7 10/19/2012 Moderately Satisfactory Satisfactory 2.33 8 03/01/2013 Moderately Satisfactory Satisfactory 2.54 9 06/23/2013 Moderately Satisfactory Satisfactory 2.90 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made GEO Change GEO IP in USD millions To ensure immediate effectiveness of the Grant Agreement. Changes were: modification of Section 5.01 of 05/28/2009 N 0.00 the GA; modification of disbursement table; modification of Section 5.04; and addition of Section 5.05 to establish on-demand TA. Second level restructuring allowing the extension of 09/10/2012 MS S 2.33 closing date and reallocation of grant proceeds. vi I. Disbursement Profile vii 1. Project Context, Global Environment Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative) 1.1 Context at Appraisal (Brief summary of country and sector background, rationale for Bank assistance) Introduction 1. At the time of appraisal, the transport sector was responsible for more than 20 percent of the carbon dioxide (CO2) emissions in Latin America, and was the fastest growing emitting sector. The International Energy Agency projected that CO2 emissions from vehicles would increase by a factor of 2.4 (or 140%): from about 4.6 gigatons in 2000 to 11.2 gigatons in 2050. The vast majority of this increase was expected to take place in developing regions, especially in Latin America and Asia, as a result of increased motorization and vehicle use. 2. At the same time, Latin American cities were growing rapidly. Approximately 75% of Latin Americans lived in urban areas, and this was where most of the vehicle kilometers of travel (VKT) occurred. Urban transport, therefore, represented a key sector for long-run greenhouse gas (GHG) mitigation efforts in Latin America. In addition, cities in Latin America were expanding and sprawling rapidly as the mobility needs were being primarily satisfied by a growing reliance on individual motor vehicles. 3. The GEF Sustainable Transport and Air Quality Program (STAQ Program) was proposed to reduce the rate of growth of GHG emissions from transport in Latin America through the promotion of less energy intensive and cleaner modes of transport. The US$79 million STAQ Program 1 was designed as a horizontal Adaptable Program Loan (APL). It was divided into four projects – a regional umbrella project and three country projects in Argentina, Brazil, and Mexico, respectively. The Regional Project and the Country Projects were expected to contribute towards the development/adaption of sustainable transport policies in selected cities and to support national policies to significantly lower CO2 emissions in the long run and increase the person-trips in more efficient transport modes. 4. The Regional Project supported the goals of the overall STAQ Program by ensuring (i) the expansion of the knowledge base in the region on climate change and transport related issues; (ii) increased knowledge-sharing across countries and cities in the entire Latin American Region; (iii) coordination and cooperation amongst GEF implementing agencies, financing institutions, and other organizations engaged in sustainable transport investments in the region; and (iv) policies and guidelines to promote more energy efficient and climate friendly transport systems in the cities of the region. 5. Each of the Country Projects included three or four city-level sub-projects with the aim of developing sustainable urban transport investments and reducing GHG emissions by promoting investment in the following thematic windows: (i) Freight 1 Including US$20.8 million of GEF funding 1 Transport Management, (ii) Integration of Land Use, Transport and Environmental Planning, (iii) Modal Shift to Public Transport, (iv) Non-motorized Transport (NMT), and (v) Travel Demand Management (TDM). These sub-projects were expected to provide valuable lessons to inform and develop appropriate policies at a national level. With the support of the regional project, they were also expected to address the most common barriers to sustainable urban transport practices at city and national level in Latin America, including institutional, regulatory, financial, technical, and cultural barriers. 6. It was decided that the four projects would be evaluated as four separate projects, as they have different loan agreements and task team leaders, as well as independent timelines. The separate evaluation is also due to the following considerations: (i) these operations have separate PADs; (ii) there are different Global Environmental Objectives (GEOs) and results frameworks; (iii) the projects are implemented by different agencies, (iv) the projects have different closing dates; and (v) the Implementation Completion and Results Report (ICR) template is not appropriate for reporting on four separate operations. Therefore, the focus of this ICR is the Regional Project, which is the first of the four projects approved and completed. Since this ICR was prepared when the country projects were still ongoing, no reporting on the results of the overall STAQ Program is included. Separate ICRs will be prepared for each Country Project, which will provide detailed descriptions of these projects and the respective results. Rationale for Bank Assistance 7. The main factor for the Bank’s involvement in the STAQ Program was to address transport and environment issues in a coordinated way at the regional level and to ensure that Latin American cities were better positioned to meet the growing demands posed by the climate change agenda. At the time of appraisal, the link between the transport sector and climate change mitigation was poorly understood at the local level, hence the need to develop common working frameworks and methodologies and to generate platforms for effective South-South exchange. The Bank’s presence in the transport sector across various Latin American countries provided it with a strategic advantage to facilitate a region-wide knowledge exchange process and ensure the involvement of governments, private sector and other development agencies in the quest to mainstream climate change in urban transport. 8. Additionally, the World Bank was already actively involved in the Clean Air Initiative for Latin American Cities (CAI-LAC). CAI-LAC was created in the late 1990s as a network-based partnership managed by the World Bank, acting as its Technical Secretariat, to engage Latin American stakeholders and facilitate a number of activities including information exchange, capacity building, and knowledge creation on air quality and transport issues. In 2003, the World Bank and CAI-LAC members decided to move the Technical Secretariat away from the Bank. The idea to “spin-off” CAI as an independently run network, through the STAQ Program, was at the heart of the original discussions on Program design, and inspired in part by the successful experience of CAI- Asia. Through a competitive process, launched in 2005, Breakthrough Technologies Institute (BTI) was selected in April 2006 to host CAI, providing a platform for its establishment, including the provision of administrative and financial services. CAI was to execute the Regional Project for the STAQ Program based on this history and the fact 2 that CAI was found to be the only non-profit organization, at that time, with regional representation whose mandate included reducing greenhouse gas emissions and improving air quality. 9. Originally, the STAQ Program aimed to involve as many Latin American cities as possible, creating synergies, visibility, and wider political support for sustainable transport policies, and providing cross-fertilization for innovative solutions. Funding under the Program was to be allocated through a region-wide competition, open to all countries except Colombia, given that there was a separate GEF-funded project active in the country at the time. However, for implementation simplicity reasons and to avoid an overly complex and ambitious operation, the decision was eventually taken to limit participation in the Program to 11 cities from three countries - Brazil, Argentina and Mexico. 10. The STAQ program was approved under the GEF III Strategic Priorities. It was consistent with the programmatic goals of the GEF Operation Policy (OP) 11 and the GEF Strategic Priority in the Climate Change focal area CC-6. GEF grants under OP11 aimed at promoting the implementation of low carbon technologies, modal shifts to less polluting forms of transport, and interventions related to bus rapid transit systems, non- motorized transport, traffic management, and land use planning. The STAQ Program was expected to contribute to these goals by financing the incremental costs associated with awareness generation, policy adjustment, regulatory initiatives, and climate friendly technology options towards playing a crucial role in overcoming key barriers to the adoption of climate friendly development in the urban transport sector. Strategic Approach 11. The STAQ Program was designed to directly link with ongoing or planned Bank transport initiatives in cities within the three participating countries, thus leveraging significant investments to attain global environmental benefits while ensuring the attainment of other developmental goals. The products and processes of the Regional Project were open to the direct participation of all interested Latin American cities, allowing them access to lessons learned through the Program. 1.2 Original Global Environment Objectives (GEO) and Key Indicators (as approved) 12. The Regional Project had three Global Environment Objectives 2 (GEO)/Project Development Objectives (PDO): i. To establish a network of local and national government stakeholders, international organizations and private sector entities to promote policies and actions leading towards more energy efficient and cleaner urban transport systems in Latin American cities; ii. To assist cities to develop sustainable urban transport strategies that integrate climate change and air quality components; and 2 In this document, the term GEO and PDO will be used interchangeably. 3 iii. To improve the capacity of cities to quantify the impacts of transport policies on climate change and air pollution emissions. 13. Progress towards achievement of the GEO/PDOs was measured through a set of five Project Outcome Indicators, used in turn as Intermediate Outcomes, and detailed below: a. Formal support and a work plan financed by different sources to coordinate activities at the regional level, involving local and national governments, international organizations, and the private sector. b. Cities with a strategy to reduce CO2 through sustainable transport strategies, including regulatory and financial frameworks that foster sustainable transport systems at the local and national levels. c. Better understanding of the local and global environmental impacts of different transport policy and technology strategies in cities. d. Cities applying assessment tools to quantify GHG and air quality impacts of select transport options. e. Availability and actual application of tools for local and national decision makers and project developers to assess and compare transport policies and investments in terms of climate and air quality benefits. 14. Progress towards achievement of these Project Outcome Indicators/ Intermediate Outcomes, was evaluated through a set of 6 Intermediate Outcome Indicators, also detailed below: a. Number of agreement letters from local and national governments, international organizations and private sector formally endorsing the program; b. Amount of additional resources leveraged by the GEF Program through CAI; c. Number of staff working at CAI to manage the work plan; d. Number of case studies, reports and guidelines on strategies to reduce CO2 by sustainable urban transport program disseminated in the region; e. Number of trainees that have taken the sustainable transport, climate change and air quality courses offered by the program; f. Number of cities applying tools and methodologies available to assess transport policies and investments in terms of climate and air quality benefits at city and project level. 1.3 Revised GEO (as approved by original approving authority) and Key Indicators, and reasons/justification 15. The PDO and the key indicators were not revised. 1.4 Original Components (as approved) 16. The Regional Project carried out a coordinating function for the STAQ Program. It provided general technical supervision and assistance. It also included standalone 4 activities. In this way, it was expected to have an impact beyond the individual projects and sub-projects and integrate other cities and countries. The Regional Project had the following four components: 17. Component 1: Evaluate methodologies to assess climate change, air quality and other benefits of transport interventions (US$538,000). This component aimed at gathering, assessing and enhancing existing methodologies and models to quantify GHG and air quality impact of transport interventions. It included (i) the identification and review of existing transport demand, emission and climate change assessment methodologies and models, (ii) the evaluation of commonly used or promising methodologies in cities, (iii) the identification of strengths and weaknesses of different models at different scales, (iv) the organization of online forums and technical workshops to disseminate and discuss models and methodologies with governments and stakeholders at regional and city level, and (v) the compilation of regional scenarios with climate change and air quality baselines, alternative transport scenarios, and potential GHG and air quality benefits as a result of sustainable transport programs in Argentina, Brazil and Mexico. 18. Component 2: Strengthen the planning and implementation capacity of local and national level institutions (US$811,000). The component aimed at assisting cities and strengthening their capacity to maximize GHG benefits of investments and helping them to implement comprehensive urban transport policies, which effectively also address air quality and GHG concerns. It included (i) the delivery of hands-on assistance and training to local staff and stakeholders on transport, air quality and climate related topics, (ii) the creation and maintenance of a virtual network of regional trainers, experts and institutions to deliver activities in Latin America in the long run, (iii) the compilation and dissemination of a hands-on manual to develop sustainable transport strategies and projects, and (iv) the taking on a clearinghouse function for strategies, options, instruments and experiences to design and implement transport programs and projects and their linkages to climate change and air quality. 19. Component 3: Foster the development of regional national and local comprehensive climate change, air quality, energy, transport and land use policies (US$764,000) to build awareness and foster policy dialogue at the national and local levels to promote more rational uses of motor vehicles and facilitate the introduction of more fuel efficient and cleaner technologies. This included (i) the organization of high- level annual conferences and workshops for targeted dialogues, (ii) the development of hand-outs and easily accessible information for decision makers on key environmental and climate change impacts of transport policies and possible measures to reduce impacts, and (iii) the maintenance of a coordinated policy dialogue with the countries on the implementation of national policies and economic incentives to promote sustainable transport, energy efficiency, clean technologies and fuels and facilitate implementation of local action plans. 20. Component 4: Project implementation and STAQ Program monitoring and support (US$790,000) to provide tailor made technical assistance to participating cities and continuous and structured monitoring, supervision and evaluation of the program’s impacts. This included (i) the overall management of the STAQ Program, (ii) technical assistance to cities in project design, preparation, implementation, supervision and 5 evaluation, (iii) timely policy and technical support when requested by participating cities, (iv) gathering and processing of data and information from participating cities to prepare baseline and progress in project implementation, (v) the preparation of data monitoring tools for reporting and dissemination, (vi) the development of guidelines and the proposal of policies to institutionalize indicators for periodic and public reporting at city level, and (vii) training activities for indicators measurement, monitoring and assessment. 1.5 Revised Components 21. The project components were not revised. 1.6 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) 22. The Regional Project underwent an initial Amendment to the Grant Agreement in June 2009. This Amendment: (i) modified Section 5.01 of the Grant Agreement, which stated that the Agreement would not be effective until CAI executed a Cooperation Agreement with one of the National Executing Agencies (NEAs) of the associated GEF Country Projects, due to delays in the preparation of Country Projects which were holding back the effectiveness of the Regional Project; (ii) modified the disbursement table included in Section IV, Schedule 2 of the Grant Agreement; (iii) modified Section 5.04 to ensure that the Grant Agreement could become effective immediately; and (iv) included Section 5.05 establishing a disbursement condition (for categories 3 and 4 of the new disbursement table) linking said activities to the establishment of a cooperation agreement between CAI and one of the respective NEAs. 23. Additionally, in September 2012, the Regional Project underwent a Second Level Restructuring with the objective of: (i) extending the closing date from December 31, 2012 to June 30, 2013; and (ii) reallocate grant proceeds as required. The extension to the closing date was required due to: (a) substantial effectiveness delays of the Country Projects, which started approximately 1 year later than the Regional Project, hence enabling CAI to fulfill its coordination role vis-a-vis the Country Projects, and to provide the required technical assistance; and (b) effectiveness delays of the Regional Project itself, of approximately 6 months, shortening the implementation period for CAI’s activities, specifically the establishment of a network of local and national government stakeholders, international organizations and private sector entities. In terms of fund reallocations, US$48,000 was reallocated from Categories 1 and 3 (Consultants Services) to Categories 2 and 4 (Training and Operating Costs) to enable CAI to perform its functions during the 6-month closing date extension period. 1.7 Main Beneficiaries (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the GEO, as well as any other individuals and organizations expected to benefit from the project) 24. Under this operation, CAI’s mandate included providing services to 11 predefined GEF cities, as well as to the 3 national executing agencies (NEAs) under the respective 6 Country Projects. These were the formal primary beneficiary groups, as outlined in the PAD. Additionally, and although not formally established in the PAD, it is important to note that one of the primary beneficiaries of this operation was CAI itself, as implementing agency. Most of the initial activities undertaken under this Regional Project were directed towards the institutional set-up, staffing and training of CAI. As a result, CAI is today a recognized institution in the realm of air quality and climate change, providing policy advice and technical assistance to multiple countries, cities and specialized institutions within the LAC region. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) 25. Soundness of background analysis. Project preparation was characterized by an exhaustive consultative process, requiring substantial input from participating cities. That allowed project proponents to identify and prioritize needs – for example: cities identified on-demand technical assistance as a particularly valuable and critical resource enabling the team to adequately reflect this needs assessment in the actual Regional Project design. However, this consultative process was also extremely time-consuming, leading to delays in actual project preparation and causing some of its elements to lose momentum (see paragraph 28 below). 26. Additionally, project preparation drew on lessons learnt from prior Bank interventions in the region, including the need to approach the project holistically, incorporating regional synergies, highlighting the importance of linking GEF projects with existing and/or future investments, ensuring the participation of the local NGO community, and giving priority to freight transport initiatives – all of which were incorporated into the actual project design. 27. Finally, the Bank’s involvement in the urban transport sector in a wide range of Latin American countries provided the Bank with a unique cross-reference perspective, allowing it to play a critical role in regional coordination, promoting the replication of good practices, fostering south-south learning and drawing on lessons learnt and past experience. 28. Overall, project preparation proved to be much longer than expected, spanning over a 4-year period. One critical factor explaining this delay, beyond those already mentioned was the lack of certainty regarding the funding for the Regional Project, given that the original allocation was much higher. That was later reduced due to lack of availability of GEF funds. This prolonged preparation phase had significant repercussions, as original plans/ideas were no longer relevant in new contexts, and needed to be redefined (especially true for training and methodology linked components). 29. Assessment of Project Design. The STAQ Program was an innovative and pioneering experience in the transport sector in Latin America, with ambitious objectives and a highly integrated vision of the sustainable transport and climate change agenda. 7 Specifically, in the case of the Regional Project, the objective of creating a self-funded network, requiring the establishment of a new institution that would eventually become self-sufficient, was extremely ambitious, particularly when moving forward with the Regional Project activities required a strong and active implementation agency from project on-start. The newly created CAI was expected to provide services to 3 NEAs and 11 GEF cities, requiring enormous efforts in terms of communication, information exchange, coordination and monitoring, and making the Regional Project extremely labor intensive. In practice and despite what is inferred in the PAD - whereby CAI was spun off from the CAI-LAC network, created in 1998 by the World Bank and member cities to improve air quality and combat climate change - the fact is that an operational institutional framework did not exist prior to the launch of the STAQ Regional Program, and there was no central body to serve as an implementation agency of this Regional Project at effectiveness. That initial lack of an adequate institutional structure and capacity to deal with project implementation inevitably led to delays in the initial phase of implementation, with time being devoted initially to getting CAI up and running, ensuring that the institution had the necessary staff and skills to act as an effective implementing agency. Although the CAI-LAC initiative had existed and had been hosted by the Bank, as an institute CAI had to play new roles and functions and get acquainted with Bank’s operational rules and procedures. That required new staff to be trained and become familiarized with such rules and procedures. As a result, the Project Coordination Unit (PCU) was not fully operational during the first phases of implementation, despite the delays seen between project approval and effectiveness. 30. In terms of the actual content of the Regional Project, the wide range of analytical and outreach activities contemplated under the Grant Agreement were also over- ambitious and led to a dispersion of efforts by CAI. The general impression was that CAI intended to be active on too many fronts simultaneously, causing the Regional Project to lose focus on core original activities at times. This multiplicity of small activities moving in parallel not always led to a clear “multiplier effect” type impact but rather to a loss of orientation during Regional Project implementation. For example, Component 2 of the Regional Project had more than 65 activities including clinics, training workshops, webinars, technical assistance, policy papers, conferences, strategies and action plans; the other components had a similar plethora of activities (see Annex 1). 31. Questions can also be raised in terms of the coordination mechanisms between the Regional Project and the Country Projects, as established in project design. The PAD highlights the “coordinating function” to be played by the Regional Project, in terms of providing technical monitoring and supervision of Country Projects. However, the fact that the latter were designed independently created a series of obstacles in ensuring that CAI could effectively play its coordinating role. Moreover, some of the contractual clauses included with the purpose of linking the Regional and Country Projects proved to be cumbersome in practice. For example, the Regional Project originally included the establishment of a Cooperation Agreement between CAI and one of the NEAs of the associated Country Projects. As mentioned previously, this was later modified through an amendment to the Grant Agreement, in order to enable effectiveness of the Regional Project despite delays seen at Country Projects. Through the same amendment, the Cooperation Agreement between CAI and one NEA was converted into a disbursement condition for specific categories, eventually also leading to delays in disbursements. Most 8 importantly, the actual gap in the timing of the Regional and Country Projects respectively restricted CAI’s capacity to effectively play a coordinating role vis-a-vis the Country Projects, even with the extension of the Regional Project closing date. 32. Assessment of risks. Although the design of the STAQ Regional Program as an APL was expected to allow the Regional and the Country Projects to move independently, nothing was considered in terms of anticipating the difficulty in coordinating Country Projects through CAI and the Regional Project, ensuring that CAI could effectively play the coordinating, monitoring and supervision role assigned to it. Additionally, since CAI was not an operational institution at the time of project initiation, delays in terms of employing appropriate staff were also not anticipated as a potential risk. Finally, the inherent risk related to establishing an entirely new network with no existing resources was not adequately identified nor mitigated in the Regional Project risk matrix. 2.3 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) 33. Initial delays in Regional Project implementation. The Regional Project was approved by the Board in October 2008; the Grant Agreement was signed in January 2009, and the Project was only declared effective in June 2009. The delay between approval and effectiveness is partly explained by problems with the effectiveness conditions established in the Grant Agreement that had to be modified through an Amendment to the Grant Agreement. However, even after the Project was formally declared effective, delays in the actual start of activities and related disbursement were also evidenced. The primary causes were the lack of an effective implementing agency at the time of effectiveness, as mentioned previously; and the fact that it took time to create, staff and train CAI to live up to its function as implementing agency. During the initial 2 years, the staff hired by CAI had limited experience in the administration and execution of Bank financed sustainable transportation projects. Additionally, in the early stages of project implementation there were difficulties in attracting and keeping qualified staff. Frequent changes in key staff members were disruptive to project progress and only resulted in further delays. 34. Mid-term review and a resulting pick-up in the pace of Project implementation. Only following the Project Mid-term Review (MTR), in October 2011, implementation truly started to pick-up, as reflected in the upgrading of ISR ratings. By 2011 CAI had a solid team of qualified professionals, familiar with World Bank policies and procedures. That was critical in explaining the turn in implementation pace towards the final years of Regional Project execution. Additionally, the MTR resulted in the establishment of an Action Plan with defined milestones, agreed upon with CAI, which was also critical in moving implementation forward in these last years. The MTR also put forward the need to extend the Regional Project closing date in order to enable CAI to play the coordinating role assigned to it, vis-a-vis the Country Projects, and to make up for the delays in the initial stages of implementation. The extension was expected to have a considerable effect on the capacity of CAI to set up a financially self-sustainable regional network. 9 35. Regional Project restructuring. Despite the objectives set forth by the project restructuring (see paragraph 23), the 6 month extension did not fully help achieving them. Firstly, Country Projects extended beyond the closing date of the Regional Project, not enabling CAI to fulfill its role of coordinating body and provider of technical assistance, monitoring and supervision support in the final stages of the country projects. This was partly due to the depletion of Grant proceeds, which did not make possible an extension beyond the granted 6 month period. Secondly, the Regional Project did not catch up with its own initial delays, and thus the establishment of the regional network was only partially achieved. 36. Despite the overambitious and complex design, the Regional Project has had a real contribution to sustainable transport policies in Latin American cities through the large number of knowledge products created and disseminated by CAI. These are perhaps the most important achievements of the Regional Project since they are increasingly used in the World Bank’s dialogue and operations with client countries in Latin America. Among the most important it is worth mentioning: “Incentive Structure in Transit Concession Contracts”; and “Planning for BRT – Oriented Development”. Paragraphs 52 to 57 (Assessment of Outcomes) provide for additional key knowledge products linked to the objectives, while Annexes 2 (Outputs by Component) and 5 (Knowledge Platform Information) describe in detail the knowledge produced and disseminated. CAI also developed an active website that can be visited at http://www.cleanairinstitute.org. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 37. The ambitiousness and complexity of the Regional Project is also evidenced in the design of the Result Framework (RF) and the selection of GEO/PDO indicators. Specifically, the first regional objective: “to establish a network of local and national government stakeholders, international organizations and private sector entities” proved to be an overambitious goal given the lack of a pre-exiting institutional setup, the goal of making it a self-funded network, and the Regional Project established time-frame. The third outcome indicator - “better understanding of the local and global environmental impacts of different transport policy and technology strategies in cities”- proved to be too broad and vaguely worded, impeding a meaningful evaluation in terms of its compliance. Additionally, in the design of the RF the selected outcome indicators are utilized, in turn, as intermediate outcomes. This is conceptually misleading and is considered as a deficiency in the design of the RF. In fact, the intermediate outcome indicators provide more tangible targets, enabling a better assessment of whether progress was made in achieving the Regional Project objectives. The above deficiencies in the design of indicators and the RF should have been corrected as part of the project restructuring. 38. Beyond the design of the RF, the Regional Project itself also included an M&E component (Component 4). Although each Country Project had its own RF, the Regional Project was envisioned to aggregate and evaluate data at Program level for wider policy analysis and dissemination. In that sense, CAI was responsible for providing technical assistance to Country Projects, monitoring and evaluating the regional level impacts, and coordinating data gathering and monitoring of overall progress of the program at the country and city levels. Originally, CAI was expected to prepare guidelines and a general methodology for baseline assessments in GEF cities. However, this proved to be 10 overambitious as well, with CAI directing its efforts to providing assistance to 4 cities in the monitoring and evaluation process, including the establishment of a baseline. Other expected outputs under this component were the definition of Program performance indicators and the elaboration of semi-annual progress reports. However, even though some baseline assessments were conducted, there was inconsistent follow up, due to the existing disconnect in timing between the Regional and Country Projects. 2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) 39. Safeguards. The STAQ Program as a whole was considered a Category B project because the city specific sub-projects in the Country Projects included small pilot investments, such as bikeways. It was anticipated that these civil works could have minor and localized negative environmental impacts. Very small social impacts were also considered possible. The Regional Project consisted completely of studies and general technical supervision activities, and had no negative social and environmental impacts. Therefore, safeguards compliance is not relevant. 40. Procurement and Financial Management (FM). Management of procurement was overall satisfactory, as evidenced by the results of all the Post Procurement Reviews (PPRs) conducted throughout project implementation. However, the Regional Project did face a series of shortcomings regarding FM. Some of these shortcomings included: (i) untimely submission of IFRs by CAI throughout project implementation; (ii) the misallocation of expenditures by CAI under Categories 2 and 4 (training and operating costs) of the Grant Agreement, leading to an overload of these categories; (iii) the borrowing of funds by CAI from the GEF Designated Account, to cover operating costs/ expenses not related to the GEF Regional Project. These shortcomings however have been satisfactorily resolved, with the reallocation of expenditures, as needed, in Client Connection, the issuing of pending SOEs for expenditures considered eligible and the reimbursement of unused funds to the Bank. CAI fully reimbursed the grant account in US$ 44,211.18, prior to the established deadline of October 31, 2013. 41. Finally, the audit reports of the Regional Project were by and large submitted on time and considered acceptable to the Bank. However, despite providing unqualified (clean) opinions, all audit reports signaled weaknesses in terms of the internal control system utilized by CAI. The last audit report, covering the period from January 1, 2012 to June 30, 2013 was completed timely and submitted to the Bank at the end of September 2013. 2.5 Post-completion Operation/Next Phase (Including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) 42. To a large extent, the prospects of ensuring the sustainability of project results, beyond project closure, depend on CAI’s ability to sustain and perpetuate itself beyond the STAQ Program life, as originally intended. This implies ensuring its financial self- sustainability, the availability and adequacy of staff, in terms of the required technical skills, and the ability to generate an ongoing pipeline of new projects/ initiatives to justify 11 its existence in the medium to long run. This becomes particularly challenging given the change in the overall context vis-a-vis the late 1990s, as currently multiple new actors and initiatives have emerged in the realm of climate change and sustainable transport, making the identification of a specific “niche” for CAI evermore critical. 43. Although a follow-on operation supported by the Bank is not foreseen in the immediate future, CAI has been actively involved in a series of new projects and initiatives, which build directly on the efforts undertaken and the results attained under this operation. In sum, these new activities are expected to leverage financing for about US$2.75 million in the near future. A few of these ongoing initiatives are listed below: • The preparation of a project to develop a national policy for urban sustainable mobility in cities across Mexico, with SEMARNAT (Secretaria de Medio Ambiente y Recursos Naturales), building on previous work undertaken with SEMARNAT through the Regional Project. This new project is being supported by the IDB. • The third phase of the development of an integrated environmental strategy assessment for the Sustainable Urban Mobility Plan in the metropolitan area of Medellin (Colombia), to be developed in early 2014, building on the initial baselines developed for emissions, air quality, economic and co-benefit analyses. • The formulation of project proposals aimed at reducing air pollution in the State of Mexico, building on the relationship established with Mexico through the different interventions undertaken through the Regional Project. This includes: 1) designing a Plan of Action to Reduce Black Carbon Emissions in ZMVT (Zona Metropolitana del Valle de Mexico); 2) determining the impact of the implementation of the air quality guidelines of WHO on the Proaires of ZMVM and ZMVT; 3) designing a strategy to improve the Vehicle Emissions Testing Program of the State of Mexico; and 4) designing a strategy for sustainable urban mobility. • The expansion of the ecodriving training program CONDUCE in Colombia and Mexico, designed under this operation; • Supporting the preparation of the second investment plan for the Clean Technology Fund, to be potentially implemented by CAI, following-up on the prior participation of CAI in the CTF through this Project; • Working with UNEP and the Intergovernmental Network on Air Pollution for Latin America and the Caribbean, strengthening capacities to improve air quality management and related issues in the region including a strong component on sustainable transport; • Preparing a book on air quality improvement measures in Latin America, in cooperation with the Bank, gathering experts from 7 cities across the region and members of the CAI-LAC network. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) 12 44. Project relevance is rated high. STAQ’s higher-level objective is to reduce the rate of growth of GHG emissions from transport in Latin America through the promotion of less energy intensive and cleaner modes of transport. The Program was approved in pre-appraisal stage in August 2006 under the GEF-3 Strategic Priorities, and is consistent with the programmatic goals of GEF Operational Program 11 (OP-11) and the GEF Strategic Priority of promoting energy efficient, low carbon transport and urban systems in the Climate Change focal area (CC-6). 45. The PDO remained in line with the OP-11’s broader goals of promoting the implementation of lower carbon technologies; modal shifts to less polluting forms of transport; and interventions related to bus rapid transit systems, NMT, traffic management, and land use planning. The STAQ Program contributed to these goals by financing the incremental costs associated with awareness generation, policy adjustments, regulatory initiatives, and climate friendly technology options towards playing a crucial role in overcoming key barriers to adoption of climate friendly development, transport policies, and technologies in the urban transport sector. 46. Finally, the PDO reflects the GEF-4 and GEF-5 strategic long-term climate change objectives and the strategic program for the urban transport sector. 3 The Regional Project and the three Country Projects will contribute towards the creation and adoption of policies at the national level to decrease CO2 emissions while increasing the number of person-trips taking place in more efficient transport systems. 3.2 Achievement of Global Environmental Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) 47. Overall rating: Moderately Satisfactory 48. Two of the Regional Project’s development objectives were fully met. However, some challenges were faced in meeting the first objective, which was ultimately only partially achieved, resulting in this project earning the rating of moderately satisfactory. 49. Table 1. PDO Achievement – Objective 1 Regional PDO Project Outcome Intermediate Comments Indicator/ Outcome Indicators Intermediate Outcome (1) To establish a (i) Formal support (i) Number of -The network has network of local and and a work plan agreement letters formally been national government financed by from local and established; stakeholders, different sources to national -By June 2013, 34 international coordinate activities governments, agreement letters organizations and at the regional level, international were signed (30 was private sector involving local and organizations and the end target); 3 GEF Focal Area Strategies and Strategic Programming for GEF-4 dated July 25, 2007 and GEF Focal Area Strategies dated October 1, 2010. 13 entities to promote national private sector -Only 5 out of 34 policies and actions governments, formally endorsing agreement letters for more energy international the program; came with financial efficient and cleaner organizations and (ii) Amount of or in-kind urban transport private sector. additional resources contributions; systems in Latin leveraged by the -The amount of American cities. GEF Program financial resources through CAI; leveraged through (iii) Number of staff CAI by June 30, working at CAI to 2013 was US$ 2.2 manage the work million (US$ 2.093 plan. million was the end target); -By June 2013, there was 4 staff members working in CAI (4 was the end target). Partially achieved 50. Objective One: Establish a network of local and national government stakeholders, international organizations, and private sector entities to promote policies and actions leading towards more energy efficient and cleaner urban transport systems in Latin American cities. The first objective was partially achieved. CAI was able to meet the majority of the targets, including obtaining formal endorsements, leveraging additional financial resources, and sourcing and retaining staff to manage a work plan; however, only five of the 34 endorsements from network members are accompanied by financial or in-kind contributions, and it is still uncertain if the network will be sustainable over the long-term. The objective could have been achieved if CAI had been an organization with a strong existing regional presence and resources; but as CAI was a new institution that has continued to develop over the course of the Project, both the network and the institution itself are at risk, mainly due to a lack of financial resources. CAI was able to leverage US$2.2 million in financial resources from the five aforementioned sources, surpassing the end target of US$ 2.09 million. 51. Table 2. PDO Achievement – Objective 2 Regional PDO Project Outcome Intermediate Comments Indicator/ Outcome Indicators Intermediate Outcome (2) To assist cities (ii) Cities with a (iv) Number of - By June 2013, 11 develop sustainable strategy to reduce case-studies, reports case-studies, reports urban transport CO2 through and guidelines on and guidelines had strategies that sustainable transport strategies to reduce been produced as integrate climate strategies, including CO2 by sustainable per the target; 14 change and air regulatory and urban transport - By June 2013, the quality components financial program number of trainees frameworks that disseminated in the reached 1333 (the foster sustainable region; end target was 500) transport systems at (v) Number of local and national trainees that have level; taken the sustainable transport, climate change and air quality courses offered by the program. (iii) Better -No associated understanding of the intermediate local and global outcome indicators impacts of different or targets- transport policy and Deficiencies in the technology selection of the strategies in cities. indicator. Achieved 52. Objective Two: To assist cities to develop sustainable urban transport strategies that integrate climate change and air quality component. The second objective was achieved and exceeded expected targets in terms of provision of training technical assistance and webinars. CAI has assisted STAQ cities in developing strategies integrating these components through the provision of hands-on, on-demand training and assistance through numerous training clinics, conferences and workshops; while providing other technical assistance on sustainable transport concepts. A number of STAQ cities have incorporated sustainability, environmental, and climate change components under their wider plans, policies, strategies, and/or laws since the beginning of the STAQ Program. Some notorious knowledge products enhancing cities capacity to reduce CO2 are: the Emissions Assessment Knowledge Platform (Helpdesk) created by CAI, the 11 policy papers, and the Communities of Practice/Webinars. CAI was also involved in the development of the Greenhouse Gas Protocol Mitigation Accounting Initiative, piloted in Medellin, and expected to provide the first framework for assessing GHG impacts from policies implementation, thus setting an international standard. See Annex 5 and/or visit http://www.cleanairinstitute.org for details. 53. CAI was also able to create and maintain a virtual network of regional trainers, experts and institutions, adopting a “train the trainers” approach that is expected to continue to deliver services and execute activities in Latin America over the long term, after project closure. CAI also assumed a clearinghouse function for strategies, options, instruments and experiences to design and implement transport programs and projects linking to climate change and air quality. The STAQ cities have utilized CAI’s guidance 15 to integrate climate, land use, and sustainable urban transport concepts into legal, policy, and planning frameworks. 54. Table 3. PDO Achievement – Objective 3 Regional PDO Project Outcome Intermediate Comments Indicator/ Outcome Indicators Intermediate Outcome (3) To improve the (iv) Cities applying (vi) Number of - By June 2013, capacity of cities to assessment tools to cities applying tools more than 3 cities quantify the impacts quantify greenhouse and methodologies were applying the of transport policies gas and air quality available to assess tools and on climate change impacts of select transport policies methodologies and air pollution transport options; and investments in available (the end emissions terms of climate and target was 3); air quality benefits at city and project level. (v) Availability and Tools available - Achieved by June actual application of through helpdesk 2013 tools for local and national decision makers and project developers to assess and compare transport policies and investments in terms of climate and air quality benefits Achieved 55. Objective Three: To improve the capacity of cities to quantify the impacts of transport policies on climate change and air pollution emissions This third objective was also achieved. Although it is difficult to measure concretely how and to what extent STAQ cities capacities were improved as a result of this project, it is inferred that CAI and the Regional Project have contributed to this improvement through a number of activities. Such activities include the provision of trainings on GHG impact assessments, the provision of technical assistance, and the facilitation of knowledge exchange through the dissemination of information and a number of workshops. CAI has also developed a number of tools, methodologies and other guidelines specifically designed for the region; as well as reference cases to guide cities in measuring GHG impacts, including IES, Mobile 6, and IVE models. 16 56. CAI has developed a Web site that includes a helpdesk and access to the Communities of Practice (COPs), based on the STAQ Program’s five thematic windows. These CoPs aim to disseminate the products developed in the framework of the STAQ Program, particularly the policy papers, as well as being used for wider capacity building, and exchanging of information. Additionally, the Web site aggregates experiences from throughout the LAC region in the form of guidelines, manuals, methodologies, and past webinars, specifically designed for the region. These tools and methodologies are being used, and several STAQ members use the Web site to access information on these methodologies. Guatemala developed its emissions inventory for mobile sources using CAI’s information on the helpdesk on IVE model, and the Metropolitan Area of the Aburra Valley is applying the IES approach to its Sustainable Mobility Plan using CAI’s experience in Bogota. The IES approach was used to evaluate GHG emissions, local pollutants, and to assess co-benefit assessment and other externalities. Medellin applied the methodologies and tools outside the framework of the baseline assessments supported by CAI as part of a World Resources Institute (WRI) pilot. 57. Since its creation in March 2012, CAI’s online helpdesk has received 2,137 visits, 5,152 page views, and 226 downloads. There have been 4,157 visits to the Regional scenario Web page and a total of 7,655 views of the pages. Through this website CAI also received requests for advice and support from Caracas, Rosario, and Guatemala City. Since November 2012, there have been 89 visits to the Air Quality and Climate Change Community of Practice and 545 page views. There were 415 participants in the 3 webinars on climate change and air quality. 3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) 58. Given the nature of the Regional Project which only consisted of studies, institution building and general technical supervision activities, no economic and financial analysis was carried out at appraisal. Applying economic analysis to technical assistance projects is rare and challenging. However, at the time of appraisal a quantitative assessment may have been carried-out on the efficiency of the use of a centralized capacity building service – through CAI - as compared with what each of the individual country-based project would have achieved using the same type of capacity building and assessment of appropriate GHG reduction methodologies. 59. For the entire STAQ Program as well as for each Country Project an incremental costs and benefits analysis, on the basis of with and without project scenarios, was carried out. The ex-post version of this analysis will be included in the ICR reports of each Country Project. They cannot be included in this ICR, as the Regional Project closed prior to the closing of the three Country Projects. 60. For the Regional Project, in an ex-post qualitative assessment perspective, it can be highlighted that the price of the studies carried out was generally below the respective cost estimates. The studies, once contracted, were quickly implemented and there were no cost overruns and only few contract modifications. Moreover, within a limited budget, the Project was able to produce 11 case-studies, reports and guidelines on strategies to reduce CO2 emissions through sustainable urban transport programs, to train 1333 17 practitioners (with a target of 500) in sustainable transport, climate change and air quality, and to deliver a series of innovative knowledge products expected to have significant impact in the realm of urban transport and climate change. Given the above, it can be argued that the specific capacity building methodologies that CAI innovated, as compared with a more conventional approach to training and capacity building of each Country Project were more efficient; a centralized approach to technical assistance and knowledge products resulted in reduced costs of delivering a given capacity building event, and increased number of people that were trained per capacity building event. 3.4 Justification of Overall Outcome Rating (combining relevance, achievement of GEOs, and efficiency) Rating: Moderately Satisfactory 61. The overall rating is moderately satisfactory. The PDO and the project design remain highly relevant; the Regional Project achieved 2 of the 3 development objectives, 4 of the 5 established outcome indicators, 5 of the 6 intermediate outcome indicators, and was fully disbursed. 3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development 62. The STAQ Program expects that, through the promotion of sustainable transport strategies and concepts, it will indirectly have a positive impact on poverty alleviation throughout the region. This Regional Project lays the proper foundation to encourage better decision making by relevant city planning officials by allowing them to 1) understand better the negative social and environmental impacts of current transportation systems, and 2) measure and account more effectively for such negative impacts. By doing so, project proponents expected to have an impact, down the line, in terms of improving transport conditions and overall accessibility among low income users, who typically suffer the most from inequitable access to transport and tend to be most affected by public transport shortfalls. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) 63. Cities: Institutional development and capacity strengthening were part of the core objectives of this project. Local, provincial, and national government agencies and affiliated organizations active in the area of sustainable transport participated in a number of capacity-strengthening activities, including numerous workshops and clinics integrating urban transport and land-use principles. The project as a whole, and in particular Component 2, focused on strengthening the capacity of local institutions to plan and implement comprehensive urban transport policies (for further details and results see Annex 2, Component 2). 64. Institutional building/ strengthening of CAI: Probably one of the most relevant accomplishments of the Regional Project was the transformation of CAI, from a burgeoning institution at project on-start, with no actual implementation capacity, to a fully functioning network, capable of undertaking and managing diverse projects and initiatives. Despite shortfalls and difficulties encountered in terms of ensuring financial 18 self-sustainability, CAI has managed to assemble the necessary human resources and technical capacities to act as an effective executing agency. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) 65. No beneficiary survey or stakeholder workshops took place. 4. Assessment of Risk to Development Outcome Rating: Significant 66. At present, the main risk to the Regional Project development outcome seems to be the uncertainties in terms of sustaining financial support for the CAI network in the future. This risk was not identified at the project design stage, and therefore no mitigation measures were foreseen. 67. However, during implementation CAI carried-out an aggressive outreach campaign, generating additional incentives for donors, with the ultimate objective of expanding and diversifying its membership and sponsorship network. Additionally, CAI continues to foster closer ties and collaborative efforts with other similar international initiatives, national and local governments, local organizations (universities, etc.) and actively participates in regional events, to expand its opportunities and compensate for the difficulties encountered in reaching out to the donor community. 68. CAI was conceived as a self-sustainable network following the closure of the Regional Project, relying on payments of services and donor funds to finance its core functions. That has made fundraising and self-promotion one of the top priorities on CAI’s work agenda, for evident reasons, diverting efforts and energy away from what was expected to be its core mission: the promotion of sustainable transport and air quality. This trend could eventually undermine CAI’s capacity to remain a key player in this arena. 5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 5.1 Bank (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Unsatisfactory 69. The STAQ Program can be commended for being a path-breaking and innovative operation, which adequately responded to the pressing need, at the time of preparation, of better linking the agendas of urban transport and climate change in the region. The objectives pursued by the operation remain relevant and there is no question that the Bank had a strategic advantage in moving this agenda forward, given its regional presence in the urban transport sector and its ability to mobilize the support of different stakeholders, at local, national and regional level. 70. Preparation of the Regional Project, specifically, was thorough, involving an extensive consultative process with participating cities, but also lengthy, spanning over a 19 4-year period. One of the main factors explaining this prolonged preparation phase was the uncertainty regarding availability of GEF funding, with the original provision of funds being significantly modified during the preparation phase. However, these factors were outside the Bank team’s control. 71. Notwithstanding, in retrospective, the Regional Project design did reveal a series of weaknesses, including: (i) overambitious objectives (particularly, the creation of a self- funded regional network); (ii) deficiencies in the choice or wording of some indicators in the Project Results Framework, making their actual measurement and evaluation challenging; (iii) underestimation of project operating costs, considering the initial inflow of resources required to get CAI up and running; (iv) dispersion of activities included in the Regional Project, leading to a loss of focus and orientation during implementation; and (v) problems in the design of coordination mechanisms between the Regional and the Country Projects, oftentimes generating bottlenecks in implementation. Probably one of the most significant flaws in project design lies in the choice of institutional/ implementation arrangements. The fact that CAI had to act as implementing agency of the Regional Project, when the operation, as designed, required a strong implementing agency from the on-start, resulted in unforeseen delays and problems during the project life. An alternative implementation arrangement, relying on a more established or mature organization, would have probably suited the operation better. 72. Consequently, despite the innovative nature of the project and its thorough technical preparation, due to the abovementioned shortcomings, Bank performance in terms of ensuring quality at entry is rated moderately unsatisfactory. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Satisfactory 73. The Bank team undertook an active supervision role, providing hands-on support to CAI throughout the entire implementation period, particularly in the initial stage, providing institutional support, through the staffing and training of new personnel, specifically in Bank policies and procedures. 74. The initial delays incurred between grant approval and effectiveness, and between the declaration of effectiveness and the actual initiation of activities, can be mostly attributed to the vacuum that existed in terms of project implementation capacity. In addition, the internal Bank requirements (through the effectiveness and disbursement conditions) also contributed to these initial delays. 75. Although the Bank team was proactive in undertaking the project restructuring resulting from the findings at the MTR, questions remain on whether this was a missed opportunity to revise the project RF and monitoring indicators, and whether the restructuring actually lived up to its objectives (see discussion in paragraph 37). 76. Finally, the Bank team provided adequate support both in technical and fiduciary matters, sustaining permanent communication with CAI counterparts and working side by side in solving implementation issues, as they arose. This partnership was critical in managing to attain satisfactory results despite the various obstacles encountered during 20 project execution. Safeguards policies were not triggered due to the technical assistance nature of the Regional Project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 77. The Bank performance at entry and the quality of supervision are rated moderately unsatisfactory and moderately satisfactory, respectively. The Bank’s overall performance is considered moderately satisfactory given the innovative nature of the project, its relevance to the region, and the various knowledge products created which are successfully used by Bank teams in projects and policy dialogue with clients (see paragraph 36 above). 5.2 Borrower Implementing Agency Performance Rating: Moderately Satisfactory 78. The CAI’s capacity to act as Project Implementing Agency (PIA) needs to be evaluated based on the following factors: (i) although CAI was spun off from the CAI- LAC network, no institutional framework existed prior to the launch of the Regional Project, enabling CAI to actively assume the functions endowed upon it as PIA; (ii) beyond the lack of an institutional framework, at inception CAI did not have the human resources or technical capacities/ skills required to act as PIA; and (iii) CAI’s budget relied exclusively on the Bank’s grant proceeds, with operating costs exceeding the original provisions. 79. These limitations led to considerable delays in getting a fully functional PIA, with most of the initial project efforts being devoted to getting CAI up and running. However, throughout the life of the Regional Project CAI managed to establish itself as a leading player within the clean air, climate change and sustainable transport agendas in the LAC region, with a team of 4 highly qualified and specialized professionals who acquired the skills to manage projects of this nature. 80. Beyond the initial stage, CAI faced a series of implementation challenges, including: (i) dispersion of its efforts, given the large number of parallel activities being undertaken under the Regional Project, requiring CAI to be active on too many fronts simultaneously; (ii) shortcomings in terms of project financial management (delays in submission of IFRs, charging of ineligible expenses, borrowing of funds from GEF Designated Account); (iii) difficulties in reaching out and ensuring support from the donor community; and (iv) limitations in its capacity to truly act as coordinating agency vis-a-vis the Country Projects. Budget constraints were often to blame in CAI being spread out too thin and not managing to provide effective coordination of the Country Projects. For example, representatives from the CAI were often unable to participate in supervision missions of Country Projects due to budget constraints, restricting the capacity of the Regional Project to stay informed of realities on the ground in STAQ cities, provide support as needed, and encourage collaboration amongst pilots. Beyond budget constraints, there were time constraints in trying to coordinate the agendas of the respective Bank supervision teams, making coordination meetings more the exception 21 than the norm during implementation. All of these factors considerably limited CAI’s ability to truly fulfill its coordination mission/ mandate. 81. However, in general, issues arising during implementation were adequately resolved by CAI, with support from the Bank team, as mentioned above. A clear example has been the ability to find solutions to the different issues encountered in terms of financial management, as described in the FM section of this ICR (paragraph 41). 82. A final question when evaluating CAI as implementing agency, also previously raised, is its sustainability moving forward, particularly its capacity to become and remain financially self-sustainable after the Regional Project closure. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 83. Given that Government performance is not relevant to this operation, and that CAI’s performance as PIA was rated as moderately satisfactory, the same rating applies for overall Borrower performance. 6. Lessons Learned (both project-specific and of wide general application) 84. Design of Regional Projects. The experience gained under this operation raises a series of questions regarding the most appropriate way of structuring regional or umbrella type projects: Do independent project cycles help or hinder the effective implementation of these projects? Can effective coordination be guaranteed when Project timeframes differ substantially? Is the inclusion of legal covenants to establish linkages between projects recommendable? What other form of coordination mechanisms can be put in place to guarantee connectivity and ongoing dialogue between projects? Finally, does the Bank have the appropriate mechanisms to enable proper implementation of regional operations? 85. Results Framework and relevance of PDO indicator over time. As indicated in Section 2.3 of the ICR, the definition of appropriate and easily measurable set of indicators to reflect most objectively the level of achievement of the project PDOs was a challenge. That has led to difficulties in measuring the real project performance. Project teams should pursue more active restructuring – ideally at mid-term review – when such weakness is identified, and reformulate the indicators by better aligning them with the PDOs. That is particularly critical for projects that operate for several years in a changing environment when often the original results framework, or part of it, may become less relevant to the evolving context. 86. Institutional arrangements matter. Beyond questions on the appropriateness of the institutional arrangements put in place for this operation, and the effectiveness of the adopted selection process, timing becomes a critical factor. The implementing agency needs to be capable of exercising its implementation capacity since the grant/loan effectiveness. Trying to strengthen and build the implementation agency while in parallel trying to advance execution is not recommendable and inevitably leads to delays in project implementation. The innovative features of regional projects and their potential 22 transformative capacity come associated with longer preparation timetables, higher costs and implementation challenges, given the need for constant coordination. Although regional projects are increasingly being promoted by development partners, and their value in terms of regional integration cannot be denied, it is important to have realistic expectations when moving these projects forward, acknowledging their implications in terms of project preparation and implementation. Additionally, ensuring cross- fertilization and effective coordination between projects can be extremely challenging, particularly when resources, both human and financial, are restricted. Functions endowed on the “coordinating body”, including information sharing, dissemination, institutional strengthening, monitoring and evaluation are often too broad when compared to budget availability, making operating costs often higher than expected. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 87. The Borrower ICR/comments received on November 6, 2013 were incorporated in the report (see Annex 4). The general views expressed by CAI are in line with this ICR contents. However, there is a disagreement regarding the proposed overall outcome rating of the Regional Project as “moderately satisfactory”. Although CAI recognizes that two out of the three PDOs were fully met whilst one was partially achieved, they argue that consideration should be given to the work undertaken to the completion of the activities included under the four components. They argue that compliance with the component activities was the focus of the Bank supervision team during implementation. Therefore the rating should take into account the full achievement of activities under the project components. 88. Although the Bank team understands the point raised by CAI, a Bank financed project is rated based on the level of achievement of the PDOs through the related indicators results, not on the list of activities accomplished under the different project components. (b) Cofinanciers 89. Not applicable. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 90. Not applicable. 23 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ equivalent) Actual/Latest Percentage of Components Appraisal Estimate Estimate Appraisal (%) Component 1: Evaluate methodologies to assess climate 538,000 530,284.45 98.6 change, air quality and other benefits of transport interventions Component 2: Strengthen the planning and implementation 811,000 768,798.16 94.8 capacity of local and national level institutions Component 3: Foster the development of regional national and local comprehensive climate 764,000 725,510.05 95.0 change, air quality, energy, transport and land use policies Component 4. Project implementation and STAQ 790,000 834,196.16 105.6 Program monitoring and support Total Baseline Cost 2,903,000 2,858,788.82 4 98.5 Physical Contingencies 0.00 0.00 Price Contingencies 0.00 0.00 Total Project Costs Project Preparation Facility (PPF) 0.00 0.00 Front-end fee IBRD 0.00 0.00 Total Financing Required 2,903,000 2,858,788.82 (b) Financing (in US$ million equivalent) Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (US$ millions (US$ millions Appraisal ) ) Borrower 2.09 2.15 5 103% Global Environment Facility (GEF) 2.90 2.85 98.5% Local Sources of Borrowing Country 0.00 0.00 .00 4 The grant closed with an undisbursed balance of US$ 44,211.18 5 As of June 30, 2013. Subsequent to the closing date and before November 6, 2013 CAI has received additional financing of over US$ 50,000 24 Annex 2. Outputs by Component Component 1: Evaluate methodologies to assess climate change, air quality and other benefits of transport interventions Cost: US$530,284.45 or 98.6% of the appraisal estimate of US$538,000 Activities carried out, outputs and outcomes This component was meant to gather, assess and enhance existing methodologies and models to quantify GHG and air quality impact of transport interventions. It envisaged the following activities: (i) identification and review of existing transport demand, emission and climate change assessment methodologies and models, (ii) evaluation of commonly used or promising methodologies in cities, (iii) identification of strengths and weaknesses of different models at different scales, (iv) organization of online forums and technical workshops to disseminate and discuss models and methodologies with governments and stakeholders at regional and city level, and (v) compilation of regional scenarios with climate change and air quality baselines, alternative transport scenarios, and potential GHG and air quality benefits as a result of sustainable transport programs in Argentina, Brazil and Mexico. All planned activities were substantially carried out and the expected outputs were largely achieved. This included the following: • Identified and reviewed transport demand, emission, climate change, and health impact assessment methodologies, models and tools. This included a description of their strengths and weaknesses and the preparation of fact sheets. The results from this activity are available through CAI’s website; • Organized an expert workshop on Critical Review of Methodologies to Assess Greenhouse Gas and Local Pollutant Emission Impacts of Transport Interventions on September 14-15, 2010 to shed light on the large amount of existing methodologies, models and tools to assess GHG emissions and air pollutants from transport intervention and to calculate the related air quality and health impacts; • Organization of a workshop on Applying the Integrated Environmental Strategies (IES) Approach to the Integrated Public Transport System Plan for Bogotá on September 16, 2010 to present the analytic framework for considering strategies that can yield local and global benefits as well as an opportunity to apply it to a real world case, i.e. the Public Transport Plan of Bogota and on November 29, 2011 to present the results of the GHG, local pollutants emissions assessment analysis and their impact on public health; • Evaluated non-motorized transport methodologies of the Transportation Emissions Evaluations Model for Projects (TEEMP) and the Center for Clean Air Policy (CCAP) Guidebook Emissions Calculator in Rosario and simplification of the TEEMP methodology to be used for the baseline assessments in the GEF NMT country projects; • Evaluated the TEEMP, the CCAP Guidebook Emissions Calculator, and the Methodology for Calculating GHG Emissions Reduction for The Mexican Urban Transport Transformation Project methodologies for BRT systems in Monterrey and simplification of the last one to be used for the baseline assessments in the GEF country projects; • Co-developed WRI’s Greenhouse Gas Protocol Mitigation Accounting Initiative through active participation in Technical Working Group meetings (via Webex and in-person workshops). Provided input and reviewed chapters on baseline emissions estimations, ex-ante and ex-poste emissions estimations. Drafted a transport case study for calculating emissions entitled “Expansion, improvement and promotion of public transportation through the implementation of a BRT” and reviewed and provided feedback on other transport case studies drafted by other participants. The protocol will provide the first framework for assessing GHG impacts from policies and the aim is for it to be adopted as an international standard for such assessments, supporting developing countries in the GHG impact quantification of policies. CAI is also pilot-testing an Air Quality Management Plan as part of the above mentioned initiative in Medellin. Specific areas which will be addressed using the Protocol are: fuel quality, vehicle inspection scheme, compliance with new energy efficiency laws, renewal of the vehicle fleet, improvements in the public transport system, reduction of the motorcycle fleet, and inclusion of TDM measures in the city’s plans. Dissemination of initial lessons learned is available on the CAI website. This is one of a number of pilots which will be used to improve the protocol and provide case study example prior to final publication of the protocol. • Publication of a helpdesk (http://www.cleanairinstitute.org/helpdesk/) to assist LAC countries in the 25 evaluation of GHG emissions and health impacts of transport interventions. The website includes basic guidance and recommendations as to the tools and models available to assess GHG and local pollutant emissions from urban transport interventions in Latin America and the Caribbean Region. A database is also provided of factsheets and more detailed information; • Assisted Monterrey, Leon, Ciudad Juarez, Cordoba, Rosario Posadas, Belo Horizonte and Curitiba in the assessment of GHG impacts of their transport interventions under the STAQ Program and publication of three country reports with the results (for details see component 4 below); • Prepared a technical note on GHG emission assessment from transport interventions: “Development of a Common Assessments Framework and Proposed Methodologies for Integrated Assessments of GHG and Local Pollutants of Urban Transport Interventions in Latin America and The Caribbean Region”. This document is available through CAI’s website. • Prepared a paper with guidelines on baseline and follow up assessments of GHG emissions for transport interventions, which builds upon experience and lessons learned from undertaking the emissions assessments for 14 GEF country projects of the STAQ Program. • Prepared a paper on methodologies and tools for the evaluation of freight transport policies and identification of policies to improve urban transport logistics; • Trained three local officials from the State Council of Transport and Roads (CET y V, in its Spanish acronym) of the city of Monterrey on GHG emission inventories, methodologies (CoP) (including the methodology developed by CAI) and other methodologies developed for other institutions in 2012. • Applied methodologies and tools reviewed and supported by CAI on GHG and local pollutants emissions and co-benefit assessment of Air Quality Management policies in Honduras, Nicaragua, and Peru. • Organized three webinars on methodologies and tools to conduct GHG emissions assessment from transport interventions: a) Impact assessment process in reducing greenhouse gas emissions, data needs and available tools (November 6, 2012); b) Uses of the TEEMP (December 13, 2012); and c) Proposed methodology to assess GHG emissions assessment from BRT and NMT interventions (May 2, 2013). • Reviewed and improved the Monitoring, Reporting and Verification System for the “Clean Transport” and “Vehicle Substitution and Renewal Scheme” Programs in Mexico. This activity has been developed as part of a program led by the German Agency for International Cooperation (GIZ) for the Mexican Government with the objective to support the development of Nationally Appropriate Mitigation Actions. The Monitoring, Reporting and Verification systems are a tool to collect the necessary information to monitor and report on implementation of given policies following protocols accepted by the international community. The deliverables include a review of the existing MRV structure, recommendations of changes and improvements to the structure, a tool for collecting data and calculating information to compare to indicators and GHG emission reductions targets. CAI led the team and directed the execution of the project. An international literature review was conducted by CAI as well as the construction of a conceptual framework from which the rest of the project was built upon. Through its emission experts, CAI also constructed the algorithm for the emissions tool, and provided the programmer with default emission factors, fuel consumption data, and values for the correction factors to be included. • Identified and evaluated policy options for the preparation of an Urban Freight Logistics policy in Mexico to be developed into a Nationally Appropriate Mitigation Action (NAMA). CAI led this initiative, in partnership with the Ministry of Environment and the Ministry of Communications and Transport, the GIZ, Texas Transportation Institute and USAID-MLED Mexico Program, and a number of public, private and social stakeholder organizations. • Prepared web pages entitled “Latin America – An overview of Regional Scenarios” (http://www.cleanairinstitute.org/escenarios/). The objective of this activity is to provide a source of information on current emissions projections and present scenarios for sustainable urban transport and climate change in Latin America and their potential impacts. These focus on Brazil, Costa Rica and Mexico (at the time of this analysis Argentina did not have enough information for conducting the analysis). • Developed online emissions factors database from interventions to guide CAI-LAC members in preparing emissions assessment for future sustainable transport projects. 26 Component 2: Strengthen the planning and implementation capacity of local and national level institutions Cost: US$768,798.16 or 94.8% of the appraisal estimate of US$811,000 Activities carried out, outputs and outcomes This component aimed at assisting cities and strengthening their capacity to maximize GHG benefits of investments. It also aimed at helping them to implement comprehensive urban transport policies, which effectively address air quality and GHG concerns. It envisaged the following activities: (i) delivery of hands-on assistance and training to local staff and stakeholders on transport, air quality and climate related topics, (ii) creation and maintenance of a virtual network of regional trainers, experts and institutions to deliver activities in Latin America in the long run, (iii) compilation and dissemination of a hands-on manual to develop sustainable transport strategies and projects, and (iv) taking on a clearinghouse function for strategies, options, instruments and experiences to design and implement transport programs and projects and their linkages to climate change and air quality All planned activities were substantially carried out and the expected outputs were largely achieved. This included the following: • Four hands-on clinics on land use and transport in Mexico City in 2010, Belo Horizonte in 2011, Rio de Janeiro in 2012, and Lima in 2012, with a total of 159 participants; • Seven face to face training workshops on the following topics: i) GHG emissions assessment methodologies, ii) Travel demand management strategies in Mexico, 2010; iii) Non-motorized transport in Rosario, 2011; iv) Public bike sharing systems in Rosario, 2011; v) Non-motorized transport in Rio de Janeiro, 2011; vi) Travel demand management strategies in Bogota, 2011; and vii) Integrated environmental strategies assessment for transport interventions in Bogota, 2011 with a total of 591 participants , • Launched six Communities of Practice (CoP) based on the STAQ Program’s five thematic windows and one more about fuel and technologies. These CoP aim at disseminating the products developed in the framework of the STAQ Program, particularly the policy papers, as well as being used for wider capacity building, and exchanging of information. These CoPs starts with the presentation of a topic through a webinar where the participants learn, discuss and share relevant information, documents related are uploaded to the website and everybody can reply using our blog tool; • 37 Webinars on six components: Travel Demand Management (4 webinars), Non-Motorized Transport (5 webinars), Freight Transport Management (4 webinars), Climate Change and Air quality (7 webinars) Fuel and technologies (6 webinars) and Public Transport and Land Use Planning – Transit Oriented Development (8 webinars). • 2 Brown Bag Lunches at the World Bank on the following topics: Transit Oriented Development strategies for BRT corridors and Incentive Structure in Transit Concession Contracts: The Case of Santiago, Chile, and London, England. • Established on CAI’s website of a virtual network of experts, consultants and trainers, which includes the profiles of sustainable urban transport consultants, experts and trainers (http://www.cleanairinstitute.org/AsistenciaTecnica/). This network is linked to LinkedIn. • Currently negotiating a contract with the Inter-American Development Bank’s Institute for Economic and Social Development (INDES) to use CoP to provide virtual and in person training. The training program is entitled “Leading the sustainable development of Latin American cities: Program for sub-national management teams. CAI developed a module entitled “The environmental challenge of the sustainable development,” based on CAI’s experiences from the STAQ program. CAI is in the process of developing training materials. A pilot face-to-face training session will be held in November 2013, and implementation of a one-year training program with both virtual and in-person sessions will follow in 2014. • Provided technical assistance to: (i) Rosario in the framework of a proposal for an integrated public transport system and a bike sharing system (Urbici), (ii) Posadas related to the designs and technical documents for paving works in the Uruguay Avenue BRT corridor, (iii) Buenos Aires to restructure the PTUMA program, (iv) Puebla in the restructuring and improving the bidding documents: for a road safety plan for the first BRT corridor in the metropolitan area and the design of a on NMT Plan for the metropolitan area, (v) Monterrey in preparing and strengthening the bidding documents for the preparation of technical specifications for CNG buses, preparation of technical specifications for 27 emission control and security requirements for CNG stations, revision of 2012 GHG inventory, preparation of a bicycle mobility strategy, and the concession contract for public transport supply in the Lincoln public transport Corridor – Ruiz Cortines – Ecovia), (vi) Ciudad Juarez in the preparation of a comprehensive freight plan and updating the legal framework for freight transport, (vii) Belo Horizonte in the preparation of terms of reference for the development of transit oriented development along the Contour Ring Road North, improvement of the interconnection of an existing multimodal (CBTU railway, municipal and inter-municipal buses) terminal at Vilarinho with a new BRT corridor (Antonio Carlos) including improved land use in the terminal and corresponding corridors, and to develop an integrated strategy of land use and transportation planning in a major transport corridor to include densification, other zoning and urban design policies, and financial, regulatory and tax instruments to induce the desired changes in land uses. • Designed an “Efficient Drivers Program” training program (CONDUCE). The Program has already been designed, and CAI is in the process of developing specific course material. This training program is part of a broader regional program being developed by CAI: “Integrated Environmental Strategy for a more Efficient and Low Carbon Emissions Freight Transport in LAC.” The aim of this program is to disseminate environmentally sustainable policies and strategies for the development of a green freight sector in LAC. The CONDUCE Program is currently being piloted in Mexico. The pilot was launched by CAI in May 2013 with financial support from the US- Mexico Border Environment Cooperation Commission (BECC) and the Environmental Protection Agency of the United States (EPA). This pilot phase aims to train a group of 50 new trainers in eco- driving techniques and 200 truck drivers in the first year. CAI are responsible for providing an online knowledge and training platform and working with the Ministry of Environment and • Produced 11 policy papers on the following topics: (i) Travel Demand Management: Opportunities and Challenges in urban areas of Latin America, (ii) Public Bike Sharing Opportunities in Latin America and their Emission Reduction Potential, (iii) Solar illumination of bus stops to enhance public transport quality, (iv) Experience of Contract Models in Chile for Efficient Public Transport Service Operation, (v) Sustainable and Low Carbon Transport Promotion Strategies in Latin America, (vi) CAI’s experience in supporting cities in the implementation of bike sharing schemes, (vii) Considerations for natural gas buses, (viii) Impact of the reduction of fuel subsidies on air quality, (ix) Experience of implementing logistics policies and initiatives to reduce GHG emissions from urban freight transport in LAC (x) Planning for BRT-Oriented Development: lessons and prospects from Brazil and Colombia, and (xi) Bus Rapid Transit with Corridor Densification in Belo Horizonte: Case Analysis. Component 3: Foster the development of regional national and local comprehensive climate change, air quality, energy, transport and land use policies Cost: US$725,510.05 or 95% of the appraisal estimate of US$764,000 Activities carried out, outputs and outcomes This subcomponent was meant to build awareness and foster the policy dialogue at national and local levels to promote more rational uses of motor vehicles and facilitate the introduction of more fuel efficient and cleaner technologies. It included (i) the organization of high-level annual conferences and workshops for targeted dialogues, (ii) the development of hand-outs and easily accessible information for decision makers on key environmental and climate change impacts of transport policies and possible measures to reduce impacts, and (iii) the maintenance of a coordinated policy dialogue with the countries on the implementation of national policies and economic incentives to promote sustainable transport, energy efficiency, clean technologies and fuels and facilitate implementation of local action plans. All planned activities were substantially carried out and the expected outputs were largely achieved. This included the following: • Organized a conference and exhibit on sustainable transport: “Linkages to Mitigate Climate Change and Improve Air Quality” from July 25-27, 2006 in Sao Paulo, Brazil. Although this was achieved prior to project initiation, it was conceived as part of the STAQ project, the official start date of which was delayed. There were 29 speakers at the conference, and 10 delegations from STAQ cities and 87 panelists. • Organized a conference on Sustainable Transport, Air Quality and Climate Change for Latin 28 America and the Caribbean from May 11 to 13, 2011 in Rosario, which more than 350 people from over 40 Latin American cities attended. The conference hosted 109 speakers and special guests, with 23 moderated session, two high level round tables, and 14 presentations. The event was covered in the media and close to 90% of the participants considered it as very good and excellent; • Developed a Sustainable Transport virtual library using the communities of practice (http://www.cleanairinstitute.org/cops/bd-conocimientos-gdt/). The library provides access to relevant information and publications on sustainable transport. • Prepared 8 policy notes summarizing sustainable transport, climate change and air quality issues for policy makers. CAI has disseminated two-page guidelines on a) Planning for BRT-Oriented Development: lessons and prospects from Brazil and Colombia, b) Bus Rapid Transit with Corridor Densification: Belo Horizonte Case Study , c) Bus stop lighting systems to improve accessibility to public transport, d) Incentive structure in transit concession contracts: the case of Santiago, Chile, and London, England, e) Travel demand management: opportunities to mitigate transport externalities in Latin America, f) Recommendations for the introduction of natural gas vehicles in BRT systems in developing cities, g) Recommendations for the implementation of bike share systems in Latin American cities, h) Experiences in urban freight logistic policies and initiatives to reduce emissions of GHG in LAC; • In April 2010 CAI was commissioned by the World Bank to develop a strategy for the introduction of cleaner vehicles and sustainable transport system in Colombia, for consideration by the Colombian government. The strategy encompasses three sections: 1) international practices and public policy instruments to promote clean vehicles around the world, 2) Opportunities and challenges for introducing clean vehicles in Colombia, and 3) Proposal to establish a number of regulatory instruments and policy framework. Parts of this strategy are currently being incorporated into the Ministry of Environment’s strategy to promote electric, hybrid, and natural gas vehicles in Colombia. • Assisted Colombian government in developing strategies on sustainable transport and air quality policies. In April 2010 CAI was commissioned by the World Bank to develop a strategy for a National Policy on Air Quality for the Colombian Ministry of Environment, Housing, and Territorial Development. The strategy has three sections: 1) Overview of international practices and public policy instruments to promote clean vehicles around the world 2) Opportunities and challenges for introducing clean vehicles in Colombia, and 3) Proposal to establish relevant regulatory instruments and policy frameworks. Parts of this strategy are currently being incorporated into the Ministry of Environment’s strategy to promote electric, hybrid, and natural gas vehicles in Colombia. CAI entered a technical cooperation agreement with the Metropolitan Authority of Aburra Valley (AMVA) to implement an Air Quality Management Plan between 2012 and 2013. CAI assisted the AMVA in 1) Designing technical specifications for an automated control system to track emissions testing at vehicle inspection stations; 2) Developing a technical proposal to strengthen on-road mobile-source emissions monitoring; 3) identifying improvements and quality control requirements for emissions testing at vehicle inspection stations; 4) Designing a remote sensing monitoring pilot test to enable emissions data collection without interrupting traffic flow In June 2013, the AMVA and CAI began developing the TOR for a sustainable urban mobility plan for the Metropolitan Area of Aburra Valley. • Provided input to the Peruvian government in incorporating sustainable transport, climate change and air quality considerations into a national program on sustainable development called: “Our Cities.” This program aims to foster the development 18 small sustainable cities under a new green growth approach to ensure their sustainability and avoid the accessibility problems that medium and metropolitan cities face. • Supported Honduran government in the preparation of the National Strategy to Reduce Emissions from Transport in 2011. In 2010 CAI and the Centro Mario Molina (CMM) developed an Air Quality Management National Plan for the Ministry of Natural Resources and Environment’s (SERNA) Centro de Estudio de Control de Contaminantes (CESCCO). This technical assistance was provided with support from the United Nations Environment Program (UNEP) and the World Bank. As a continuation of this effort, the World Bank hired CAI in 2010-2011 to carry out a study to assess measures to reduce air pollution from the transport sector in Tegucigalpa. In November 2010, the IDB approved a US$ 33 million loan for a Public Transportation Program for 29 Tegucigalpa’s Central District, which was part of CAI’s study. • Assisted Mexican government in the development of a National Agenda for an Environmentally Sustainable Mobility as part of the Mexican National Development Plan 2013-2018; b) Recommendations to improve the “Clean Transport” and “Vehicle Substitution and Renewal Scheme” Programs, and c) Preparation of a Nationally Appropriate Mitigation Actions for Urban Freight Logistics in Mexico. • Coordinated policy dialogue in Belo Horizonte, Brazil to integrate climate change and sustainable transport concepts into the development of an integrated strategy to address land use and transportation planning in major transport corridors. Organized 3 clinics on land use and urban transport; provision of TA on densification and transit-oriented development along BRT corridors; and reviewed Belo Horizonte’s law to mitigate climate change in May 2011, which was eventually enacted. • Developed a Draft Regional Action Plan for Intergovernmental Cooperation on Air Pollution for Latin America and the Caribbean, in collaboration with the National Institute of Ecology and Climate Change of Mexico, under the guidance of UNEP. This document aims to build on previous made during the Forum of Ministers of Environment of Latin America and the Caribbean, to: a) establish the Intergovernmental Network on Air Pollution for Latin America and the Caribbean, and b) develop an action plan to guide its operation. CAI’s participation was to ensure climate change and sustainable considerations on the development of the regional action plan. • Prepared and disseminated the first regional report on air quality across Latin America. The report was released in Mexico City on April 23, 2013, and via webinar in May 9, 2013. • CAI and Clean Air Initiative for LAC co-founded and funded the organization the Sustainable Transport Award since 2010. CAI encouraged and incentivized the participation of STAQ cities and CAI-LAC network members in this award process as part of its strategy to advance the sustainable transport agenda in the region. Mexico City won in 2012, and Rosario won third place. • CAI is an active member of the Partnership on Sustainable Low Carbon Transport (SLoCaT). The Partnership’s overarching goal is to mobilize global support to reduce the growth of GHG emissions generated by land transport in developing countries by promoting more sustainable, low carbon transport. As part of its SloCat activities, CAI serves as a member of a multi-stakeholder Technical Working Group (TWG) that was formed to advise the United Nations Department for Economic and Social Affairs (UN-DESA). The TWG’s main activity was to provide feedback on a plan drafted by SLoCaT to the Executive Office of the Secretary-General on how to best coordinate stakeholder engagement on sustainable transport issues, particularly ground transport in the urban context. CAI participated in two workshops and provided commentary on the draft. The TWG submitted the draft plan in November 2012 and is currently reviewing the final draft for re- submission in June 2013. CAI has used its STAQ experiences to guide the feedback provided on this work plan. • Acted as a civil society organization observer for the LAC region at the Climate Investment Fund (CIF) for the period 2012-2015. CAI proposed that the CIF’s Clean Technology Fund (CTF) include black carbon related issues into its funding options, as SLCPs are a real concern for CAI- LAC network members. CAI was invited to prepare a paper on the topic to support its proposal for a CTF meeting in October 2013. CAI also provided expert technical review for project proposals and was able to influence at least one project – the Mexico EcoCasa project. As a result of CAI’s input, this project was redesigned to integrate a model to calculate transportation needs for buyers of low-income eco-friendly housing. Additionally, CAI disseminated lessons learned from the STAQ Program, particularly on challenges faced in accessing funding, as well as those faced during project implementation. CAI conducted two surveys conducted on the role of civil society in the CIF process among worldwide civil society organizations. CAI presented the final results of the surveys at the CIF civil society forum held in November 4, 2012 as part of a presentation entitled “Finding New Opportunities and Overcoming Barriers” to 125 attendees. Topics covered include addressing transparency issues, and programming sufficient resources for stakeholder engagement activities, communication strategies, outreach, and technical assistance. Component 4. Project implementation and STAQ Program monitoring and support Cost: US$834,196.16 or 105.6% of the appraisal estimate of US$790,000 30 Activities carried out, outputs and outcomes This subcomponent was meant to provide tailor made technical assistance to participating cities and continuous and structured monitoring, supervision and evaluation of the program’s impacts. This included (i) the overall management of the STAQ Program, (ii) technical assistance to cities in project design, preparation, implementation, supervision and evaluation, (iii) timely policy and technical support when requested by participating cities, (iv) gathering and processing of data and information from participating cities to prepare baseline and progress in project implementation, (v) the preparation of data monitoring tools for reporting and dissemination, (vi) the development of guidelines and the proposal of policies to institutionalize indicators for periodic and public reporting at city level, and (vii) training activities for indicators measurement, monitoring and assessment. All planned activities were substantially carried out and the expected outputs were largely achieved. This included the following: • Project management; • Provision of on-demand technical assistance and support (for details see Component 2); • Monitoring of sustainable transport and air quality initiatives in STAQ cities; • Developed an Excel spreadsheet and a guidance note to measure ex-post assessments of transport interventions in GEF cities; • Prepared STAQ website and city Web pages displaying a brief informational overview of each STAQ city, including economic data, and a variety of transport indicators, including data on emissions by source and pollutant; • Conducted environmental impact assessments (GHG and local pollutants emissions) of Sustainable Urban Transport Interventions of the 14 GEF projects in 11 STAQ cities in Mexico, Argentina and Brazil. Assessments included ex-ante assessments on GHG and other local pollutant emissions as results of these interventions. • Conducted a survey on the impact of GEF Projects in STAQ cities. 31 Annex 3. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Environmental Paul Proce Co-TTL LCSEN Specialist Sr. Transport Andres Pizarro Co-TTL LCSFT Specialist Lead Transport Emmanuel James Co-TTL LCSFT Specialist Lead Transport Jorge Rebelo Co-TTL LCSFT Specialist Veronica Raffo Young Professional LCSFT Nicolas F. Estupinan JPA LCSFT Elisabeth Goller Transport Specialist LCSFT Nelvia Hayme Diaz Language program Assistant LCSEN Alan G. Caroll Operations Adviser LCSQE Jose M. Martinez Procurement Specialist LCSPT Alejandro Alcala Gerez Legal Counsel LEGLA Fabienne Mroczka FMS LCSFM Supervision/ICR Alejandro Alcala Gerez Senior Counsel LEGES Sr. Transport Elisabeth Goller TTL (Spn) LCSTR Specialist Sr. Transport Anca C Dumitrescu TTL (ICR) LCSTR Specialist Keigsner Alfaro Senior Procurement Specialist LCSPT Fabienne Mroczka Financial Management Specialis LCSFM Maria Catalina Ochoa Transport Specialist LCSTR Sepulveda Veronica Ines Raffo Senior Infrastructure Specialist LCSTR Jorge M. Rebelo Consultant LCSTR Lelia S. Werner Sr. Finance Assistant CTRLN Melanie Glass Consultant LCSTR ICR Co-Author Katya Kuang Idba Consultant ICR Co-Author (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ Thousands (including No. of staff weeks travel and consultant costs) Lending FY06 196.92 32 FY07 40.64 FY08 120.03 Total: 357.59 Supervision & ICR 328.87 Total: 686.46 33 Annex 4. Summary of Borrower's ICR and/or Comments on Draft ICR GEF Completion Report Sustainable Transport and Air Quality (P096017) Contribution of the Recipient Introduction and Role of the Clean Air Institute A recent report from the MDBs to the G20 has highlighted: “Some regional projects have the potential to be transformational in helping to provide the access to markets and essential services critical for promoting inclusive and sustainable growth. Effective implementation of such projects will catalyze more domestic resources and interest towards regional integration initiatives”. Also, “regional projects tend to take longer and cost more to prepare. The report highlights the need to “introduce incentives for staff to focus on leveraging rather than lending resources and to undertake complex catalytic and regional projects.”6 This report describes the experience of the Clean Air Institute (CAI) in executing the Regional Project of the Sustainable Transport and Air Quality Program (STAQ Program), implemented by the World Bank with funding from the Global Environment Facility. The STAQ Program has been an unprecedented effort for addressing urban transport and climate change issues at a Latin American scale, combining local, national and regional efforts. The role of CAI was to provide day-to-day coordination and technical support for the STAQ Program including, among other responsibilities, implementation of regional activities and taking necessary steps to ensure sustainability of relevant program activities. As part of this story, there is a need to provide some background information about the historic relationship between the World Bank and the Clean Air Institute (CAI). In the 1990s, the World Bank provided support to Mexico City and Santiago de Chile to implement successful transport and air quality projects that led to major interventions for addressing urban air pollution issues. Realizing the importance of disseminating lessons learnt and catalyzing actions to face raising air pollution challenges, the World Bank and six of the largest cities of the region launched the Clean Air Initiative for Latin American Cities (CAI-LAC) 7 . The World Bank took responsibility to operate CAI-LAC’s Technical Secretariat, which for several years was instrumental in channeling technical assistance, training, regional workshops and other resources to support CAI-LAC members to prepare and implement air quality improvement plans and projects. In 2003, the World Bank and CAI-LAC members decided to move the CAI-LAC’s Technical Secretariat away from the World Bank, aiming to strengthen its institutional framework and regional ownership, as well as to attract new funding for ensuring financial sustainability. In July 2005, the World Bank called for proposals to host the “Clean Air Initiative Center” and as a result of a competitive process, the Breakthrough Technologies Institute (BTI) was selected. In April 6 Infrastructure Action Plan. 2012 Follow-Up Report. Submission to the G20 by the MDB Working Group On Infrastructure. April 2012. 7 Original CAI-LAC members include Buenos Aires, Lima, Mexico City, Santiago, São Paulo and Rio de Janeiro. Bogota also joined later. 34 2006, BTI facilitated the incorporation of the Clean Air Institute as a non-profit 501 (c) 3 organization under the District of Colombia’s laws. Although this process of moving CAI-LAC’s Technical Secretariat didn’t imply any transference of budget nor resources to BTI, the World Bank issued on July 2006 a press release stating: “The World Bank will continue to be engaged with the Clean Air Initiative agenda through investment operations related to urban transport, energy, and environment; and through development of policy lending operations directly related to enhancing air quality management in the region.”8 To this end, in 2006, the World Bank selected CAI to execute the Regional Project of the STAQ Program, which was based on discussions with the National Executing Agencies, as well as with participating cities. Key technical agencies and donors in the region also endorsed the Clean Air Institute to execute regional activities. However, original expectations from the World Bank staff to get the STAQ Program approved in the second half of 2006 didn’t materialize. Finally, the World Bank and the Clean Air Institute were able to sign a Grant Agreement in February 2009 for executing the Regional Project. However, CAI was still required to sign a memorandum of understanding (MoU) with one of the National Executing Agencies (NEA) as a condition to declare the grant effectiveness. None of the NEAs had formalized their own grant agreements by then, which was out of the control of CAI. The Project became effective on June 25, 2009 and received its first disbursement of US$ 300,000 on August 3, 2009. However, CAI was not allowed to work on components 2 and 4 until signing a cooperation agreement with at least one NEA, which was achieved in December 2009 with Mexico. Subsequently, the memorandums of understanding with Argentina and Brazil were signed in March 2010 and October 2010. Conditions for a full implementation of the overall STAQ Program were met in late 2010. As a result of the prolonged delay of the STAQ Program approval and start up, interest from potential donors in the Program rapidly diminished. In addition, it also resulted in changes of priorities at the national and local level in participating countries and cities which later on impacted project implementation. Despite these delays and obstacles, the Regional Project has significantly contributed to foster sustainable transport and climate change at a regional level, in compliance to the objectives of both the Regional Project and the overall STAQ Program. In addition, CAI became a fully independent and internationally recognized organization that has shown a strong resilience and taken important steps to ensure sustainability of its program and the achievement of its mission. Implementation of the Regional Project and the Role of the STAQ program This and following sections highlights key achievements, and summarizes important lessons learnt, challenges, and barriers experienced by the Clean Air Institute during the implementation of the STAQ Program’s Regional Project. 8 http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRIES/LACEXT/0,,print:Y~isCURL:Y ~contentMDK:20993269~menuPK:258568~pagePK:2865106~piPK:2865128~theSitePK:258554 ,00.html 35 Component 1: Evaluation and improvement of existing methodologies to assess and demonstrate benefits of transport interventions. Key achievements As detailed in Annex 2, CAI successfully identified, evaluated, and improved access to methodologies, and approaches to assess impacts and demonstrate benefits of transport interventions, engaging multiple partners and leveraging additional sources of funding. CAI implemented a broad dialogue with key experts and organizations involved in developing and implementing transport intervention assessment tools. In addition, CAI associated with various organizations to implement methodologies in particular projects and locations, including a co- benefits approach that went beyond the original scope of the Program that was limited to assess greenhouse emissions. Significant highlights from this Component were the implementation of an Integrated Environmental Strategies approach to assess emissions, ambient air, health and economic impacts of transport interventions from a co-benefits perspective, linked to assessments of major transport projects in Bogota and Medellin. This included partnerships with the Inter-American Development Bank, the Government of Bogota, local academic institutions, the Aburrá Valley Metropolitan Authority, and the US Environmental Protection Agency. CAI was also involved in an international collaboration coordinated by WRI to develop the Greenhouse Gas Protocol Mitigation Accounting Initiative (under final review at time of writing). CAI provided technical input and reviewed chapters on baseline emissions estimations, ex-ante and ex-poste emissions estimations and drafted a transport case study for calculating emissions from the implementation of a BRT. CAI also pilot tested the Protocol in Medellin. The protocol will provide the first framework for assessing GHG impacts from policies and the aim is for it to be adopted as an international standard for such assessments, supporting developing countries. Lessons learnt During the STAQ Program preparations in 2006, the Bank originally detected a lack of coherent and consistent methodologies for estimating GHG emission reduction from transport measures. By 2009/2010 when the project implementation finally began this need had evolved and changed significantly since the initial inception of the Program in 2006. However, CAI took advantage of multiple existing discussions and developments among the numerous entities interested in addressing this methodology gap, as well as the absence of consistent data to evaluate climate and air pollution impacts of transport investments and land-use strategies. CAI activities have provided valuable outputs for the identification and implementation of “internationally recognized and demonstrated methodologies suitable for project assessment and integrated city- wide assessment of benefits and co-benefits of urban transport investments in Latin American cities”, as well as a clearer understanding of their scope and limitations. Furthermore, these lessons learned on assessment methodologies enabled CAI to become a key partner in national initiatives for preparing Transport NAMAs (like in the case of Mexico funded by GIZ and the USAID) and a variety of projects in which an Integrated Environmental Strategies approach is needed (like in Mexico and Medellín, funded by the federal and local governments, respectively). 36 Component 2: strengthening planning and implementation capacity of local and national level institutions. Key Achievements CAI delivered a successful program of capacity strengthening which supported the specific objectives of the component as well as two key programmatic Objectives 2 and 3 (see Section 3). Significant highlights from those listed in Annex 2 were reaching more than 1,300 individuals across the region with CAI’s capacity strengthening program through a mixture of workshops, hands-on training clinics, on-line webinars, and a major regional conference, all covering a range of sustainable transport, climate change and air quality topics; and successfully provided to World Bank and participating cities with substantial specialized technical assistant requirements of the Component to help them negotiate obstacles and move forward more effectively with their sustainable transport agendas, most significantly, for example, with being instrumental in the enacting of a law to mitigate climate change in Belo Horizonte and facilitating access to Clean Technology Funds for introducing natural gas buses and refueling stations at the Lincoln Ruiz Cortines’ BRT corridor in Monterrey. In addition to the work completed the following activities are those that are continuing beyond the STAQ program and are relevant for continuing to implement the requirements of this Component whilst ensuring the longer term sustainability of CAI: • Integration of long term sustainability measures to strength and improve the Mexican Program “Transporte Limpio” by the creation of a national network of new trainers on eco- driving techniques through the implementation of the Efficient Drivers Program (CONDUCE Program) pilot phase. CAI intends to expand this initiative over the next 12 to 18 months and beyond with governmental and private funders, with targeted focus on Mexico and Colombia in the short term. • Partnering with IADB, particularly with the Inter-American Institute for Economic and Social Development (INDES) training program: “Leading the sustainable development of Latin American cities: Program for sub-national management teams”, to develop the module on “The environmental challenge of sustainable development”. • Working with UNEP and the Intergovernmental Network on Air Pollution for Latin America and the Caribbean, strengthening capacities to improve air quality management and related issues in the region including a strong component on sustainable transport. Lessons learned The training activities as envisioned by the Program had been a comprehensive “train the trainers” approach, addressing a variety of technical subjects. CAI began to work with GIZ to develop such a training program, but the Bank’s implementation team provided directions to scale down this approach and, as a result, CAI delivered a less comprehensive training program as side events to other conferences. These programs were one-off events that lacked continuity or comprehensiveness and so to provide a more structured training framework. CAI was also able to develop and launch successfully an interactive Knowledge Platform accessible through CAI’s website. With respect to the technical assistance program (TA), the approach outlined in the GEF STAQ project document was that CAI would build up a framework of experience through its TA 37 activities that would form the basis for knowledge sharing activities which CAI would deliver. In practice, however, the anticipated role of TA in fostering a unique frame of reference for knowledge-sharing was not fully materialized, for a variety of reasons. First, CAI were requested to limit TA to only the GEF-funded activities of the country projects and should not be involved in technical assistance for the IBRD components of the country projects. Second, the local authorities in charge of the implementation of the GEF country projects did not have a clear understanding of the role of the Clean Air Institute and the Regional Project to the extent that it was perceived as a duplication of the country projects activities. Third, the projects of the participating cities in the GEF STAQ program ended up being rather small. As a result, most of the TA provided was useful for specific cities and projects, but did not have a critical mass of cities to allow for the generation of a regional approach for a knowledge-sharing program. Component 3: Fostering the adoption of regional, national and local policies that mainstream climate change and air quality considerations into transport, land- use, and energy planning. Key achievements Conditions that led to the creation of CAI-LAC in the late nineties drastically changed in recent years with the emergence of a variety of organizations and movements dealing with climate change and related issues, which have attracted the attention of local and national governments, as well as the multilaterals and other donors. In addition, climate change issues increased in importance in lieu of air quality which ceased to be a substantial part of the agenda until its very recent reemergence. Despite this new arena, the Clean Air Institute has been able to expand and strengthen the CAI-LAC Network. CAI has also played a key role in pushing forward and building partnerships and collaborative efforts with other regional and global efforts. For example, CAI is working as a Non State Partner of the Climate and Clean Air Coalition partnership (CCAC); has supported UNEP in the implementation of the Intergovernmental Network on Air Pollution for Latin America and the Caribbean, including the preparation of its Proposed Regional Action Plan, which includes a substantial chapter on sustainable transport and air quality; was selected as one of the four Civil Society Observers to the Climate Investment Fund (CIF), which promote policies and actions leading towards more energy-efficient and cleaner urban transport systems in Latin American cities; and is also a member of the SLOCAT network and of the United Nation’s Secretary General Work Group on Sustainable Transport. A number of activities at regional, national and local scales are continuing beyond the STAQ program, ensuring the long term sustainability of CAI: • Preparation of a book on air quality in Latin America, which is expected to be launched in early 2014 in an international workshop to be held in Lima under the auspices of the World Bank. • Development of the National Agenda for an Environmentally Sustainable Mobility in Mexico as part of the Mexican National Development Plan 2013-2018. The CAI was instrumental in the development of this Plan and is involved in its implementation in the administration 2013-2018. 38 • Preparation of a Nationally Appropriate Mitigation Action for Urban Freight Logistics in Mexico. This project is funded primarily with MLED and SEMARNAT and is due to continue into 2014. • Assessment of TDM policies for the Valley of Mexico Metropolitan Zone and the Valley of Toluca Metropolitan Zone in Mexico. • The third phase of the development of an integrated environmental strategy assessment for the Sustainable Urban Mobility Plan in the metropolitan area of Medellin will be developed in early 2014. Lessons learned The objective was to build awareness and foster policy dialogue at the national and local levels involving key government decision makers and other stakeholders, to advance the agenda for introduction of incentives for more rational use of motor vehicles and facilitate introduction of more fuel efficient and cleaner technologies. This component faced three key constraints. First, GEF funding of the CAI-LAC network or activities that could be said to be building the CAI-LAC network was quite limited. Second, this component required the services of an outreach and communications specialist for the Program, but this was never approved by the TTL of the Regional Project. Third, the appropriate niche for the CAI-LAC network was still to be understood due to the emergence of a number of initiatives and networks in Latin America focusing on Sustainable Transport, including, the CAF mobility observatories, the Environmentally Sustainable Transport Forum for Latin America, the SLOCAT network, the EMBARQ-related Centers for Sustainable Transport, and others. Despite these constraints, the Clean Air Institute sought to expand and strengthen the CAI-LAC Network. After successful implementation of the Regional Conference on Sustainable Transport, Climate Change and Air Quality in Rosario, Argentina in 2011, CAI launched a “Call for Joint Regional Action to Improve Urban Air Quality whilst Mitigating Climate Change,” seeking to build on the foundations established by the original Clean Air Initiative, and to consolidate the efforts made over recent years to expand the CAI-LAC Network. 35 organizations responded to this call and became members of the new phase of the CAI-LAC network. Component 4, Project implementation monitoring and support. Key achievements This component was intended to provide operational stability to the overall implementation of the GEF STAQ program by providing project implementation monitoring and support for implementation of particular components. In practice Bank TTLs were already carrying out standard implementation support and supervision of the individual country projects. CAI’s initial efforts to carry out supervision activities were deemed a duplication and inefficient use of resources. It was decided that the TTLs would carry out implementation support activities, and provide regular input – in the form of Aide Memoires and Implementation Status Reports – to the Clean Air Institute to allow for a regional overview and coordination. However, this information was not supplied on a regular basis, so CAI was unable to provide consistent programmatic monitoring to the GEF STAQ program. 39 CAI provided outstanding support to the World Bank in supporting the assessment of the STAQ Program activities in each of the cities and countries. CAI conducted environmental impact assessments (GHG and local pollutants emissions) of Sustainable Urban Transport Interventions of 14 GEF projects in 11 STAQ cities in Mexico, Argentina and Brazil. This helped the cities to assess the potential impact of their projects and to report these findings to the GEF. CAI also established a web-page that provides information on the cities which took part in the STAQ Program and tracks the individual projects by country (as reported by the cities). With a view to the sustainability of CAI, the IADB’s Office of Evaluation and Oversight (OVE) has now hired CAI to provide technical expertise to evaluate impacts on greenhouse gas emissions from selected transport projects, as part of major review on the climate change impacts of IADB portfolio. Lessons learned This component suffered from a number of constraints. First, the design of the component was too similar with that of Component 2. The distinction between efforts to strengthen planning and implementation capacity of local and national institutions and the provision of “tailor made” technical assistance to participating cities was somewhat artificial and difficult to maintain, particularly given the relatively narrow scope of the windows under implementation and the distinction established by the supervising TTLs that the technical assistance to cities in project design, preparation, implementation, supervision, and evaluation, was only appropriately given to the GEF-financed portions of those projects. Second, the operating costs assigned to the CAI from project resources were too small to ensure adequate implementation and appropriate compliance with World Bank fiduciary requirements. Compliance with the objectives of the Project and key advances and developments As described in the main body of the ICR there were three objectives as well as the four operational components of the Regional STAQ Program. With no direct relationship between the objectives and the components this matrix to some degree hindered the clear implementation of the program with competing priorities across the Objectives and Components. This section briefly presents the main achievements of the regional project in accomplishing its objectives. Achievement of Objective 1. Establish a network of local and national government stakeholders, international organizations and private sector entities to promote policies and actions leading towards more energy-efficient and cleaner urban transport systems in Latin American cities. In partnership with the CAI-LAC Network the CAI has been able to position air quality and sustainable transport in the forefront of the environmental debate in Latin America, including the importance of linking air pollution reduction measures to measures designed for climate change mitigation. For example, CAI published “Air Quality in Latin America: An Overview. 2012 Edition” with an official launch in Mexico in April 23, 2013 attracting 60 participants from a range of governmental and non-governmental organizations within Mexico, international organizations and significant press coverage at a national and international level. CAI is also preparing a draft book on air quality improvement measures in cooperation with the World Bank which gathered experts from 7 cities across the region and members of the CAI-LAC Network to pull together experiences on air pollution monitoring, dissemination and air quality management 40 planning, including tackling emissions through the implementation of sustainable transport and linking emissions reductions actions to GHG emissions actions. As a civil society organization observer for Latin America and the Caribbean at the CIF 2012- 2015, the CAI has taken a leading role among the civil society observers to disseminate the lessons learned from the implementation of the STAQ Program regarding the challenges that countries and cities face accessing funding and implementing of their national and local programs. Achievement of Objective 2. Assist cities in developing sustainable urban transport strategies that integrate climate change and air quality components. The key PDO for this component was “To improve the capacity of cities to quantify the impacts of transport policies on climate change and air pollution emissions”. Despite the challenges described in Section 0, this objective was largely achieved through a series of activities under CAI’s training and technical assistance program (T&TA). This program was comprised of tailor made technical assistance, “face to face” training workshops and an interactive knowledge platform accessible via CAI’s website. Results from the T&TA program accompanied by intensive policy dialogues conducted by CAI with STAQ program cities, others in the Region, and international organizations, allowed CAI to successfully promote the integration of strategies to reduce greenhouse gases emissions and local pollutants through sustainable transport and urban development strategies in countries and cities detailed in Annex 2. Currently, the Clean Air Institute is partnering with the Inter-American Institute for Economic and Social Development (INDES) training program of IDB: “Leading the sustainable development of Latin American cities: Program for sub-national management teams”, to develop the module on “The environmental challenge of sustainable development”. The first face to face training session will be held in November, 2013 and a one year training program will be implemented in 2014 within the LAC region. Achievement of Objective 3. Improving cities capacity to quantify the impacts of transport policies on climate change and air pollution emissions. This Objective overlaps Component 1, 3 and 4. As discussed elsewhere (Annex 2, Section 00 and Section 0) CAI has developed specific methodological approaches to assess the greenhouse gas and air quality emission reduction benefits of different urban transport investments; assisted with international collaborations to develop a new assessment protocol; provided in person and online training; undertaken assessment of GHG and air pollution impacts from GEF transport projects; provided an online platform for information on emissions assessments, available tools and models and an emissions factor database amongst others. Bank performance CAI would like to thank the World Bank staff for its continuous support for implementing the regional project and its coordination. However, some aspects could have been improved to strengthen the Program. 41 The program itself was innovative, but proved to be difficult to implement, because of the need to coordinate different teams with different agendas, and due to a conflict between the original high level regional objectives and the component-based day-to-day requirements and activities which were given more importance by the Bank. The program was also steered by the Bank towards working on completing and improving day-to-day operational tasks. This meant that there were limited resources, both time and funding, for addressing and implementing the higher level Objectives and enabling the Program to foster development in a more strategic manner. Outreach and communication activities should have been key to maintaining the coherence of the program across the four different projects, World Bank, and national and international actors. The project was designed without such a sub-component or specialist, and the need for such a specialist was not recognized by the Bank. The lack of attention to outreach and communications constrained the project from generating a sense of ownership of the lessons learned, knowledge and experience gained, tools and other products developed within the participating cities and organizations of the Program. Consequently, ensuring a long term sustainability of the Program and engagement of additional funding was difficult. As requested by and agreed with the TTLs of the STAQ program, the implementation of the Component 4 of the STAQ regional project focused in the assessment of GHG emissions reductions of the STAQ country projects which was very important to fulfill the reporting requirements of the GEF implementing agency. However, the major contribution of the Regional Project should have been an overall assessment of the technical execution of the Program to allow the development of wider policy analysis, scale-up of potentially transformational projects, and a long-term strategy to design and implement sustainable transport, climate change and air quality related investments in the region. Providing such responsibility to the Program would have allowed for wider reaching impacts and greater mobilization of the private sector financial support. Furthermore, operating costs assigned to the CAI from project resources were too small to ensure adequate implementation of the project and appropriate compliance with World Bank fiduciary, administrative and technical reporting requirements. This is particularly important considering that CAI was set up external from the World Bank with no resources. Considering the size of the grant for the Regional Project we consider that this was an unnecessary use of GEF funds and could be better thought out for similar grants in the future. It is recognized that the set up of this Program was unusual for the World Bank but an amendment of administrative and technical reporting requirements to match the type of work being undertaken would have allowed more resources to be spent on implementation. CAI Performance Throughout the implementation of this Project CAI has been able to put together a highly qualified staff and an extended network of professionals in air quality, climate change, sustainable transport, project management, operations and financing. This team has proven its capabilities by driving the Project to a successful conclusion while finding new allies to provide continuity to the Clean Air Institute programs. 42 The CAI has also adapted to changes in the international sustainable transport arena since the original inception of the Program to ensure that the Program continued to provide important contributions to the region whilst avoiding duplication of effort already being carried out by others once the Program was finally realized. These professionals also managed to obtain thorough knowledge of World Bank guidelines, rules and procedures. This knowledge and experience gained has allowed the Clean Air Institute to successfully prepare proposals and negotiate Grants with donors like USAID, US-Mexico Border Environment Cooperation Commission (BECC) and the Environmental Protection Agency of the United States (EPA), the Inter-American Development Bank (IADB), United Nations (UN) and local and private partners. Conclusions The STAQ Program was a pioneering effort in the transport sector of Latin America, with ambitious objectives and an integrated vision of the sustainable transport and climate change agenda. Its implementation represented new opportunities and challenges for all partners of the Program, including the World Bank, National Executing Agencies, the cities and the Clean Air Institute. A very important lesson learned from this project is that the definition of the objectives and indicators needs to be in line with what can reasonably be expected and must show a certain degree of flexibility to allow the Project to focus on delivering the expected outcomes rather than specific indicators as well as to adapt to changes in needs of the countries and cities. At the closure of the STAQ Program CAI is well equipped with a variety of knowledge products and services, as well as with a highly qualified staff and a broad network of consultants and supporters to implement its mission and ensure its continuity and sustainability. CAI has substantially expanded its sources of funding and, for the period July to December 2013, its project portfolio represents an income of over $800,000 with new contracts already inline for completion for 2014. The Clean Air Institute is playing a leadership role on expanding air quality efforts at a regional scale. Under the auspices of UNEP, the CAI in collaboration of the Mexico’s National Institute of Ecology and Climate Change have drafted a Regional Action Plan for Intergovernmental Cooperation on Air Pollution for Latin America and the Caribbean, in consultation with the Regional Intergovernmental Network on Air Pollution. The Forum of Ministers of Environment of the Latin America and the Caribbean Region has endorsed this plan in their recent biannual meeting, setting up a common agenda for the entire region to address air pollution and related issued from a cobenefits perspective. Sustainable transport and its links to climate change mitigation and air pollution improvement is a key thematic area and major efforts are being conducted in different cities to serve as models to foster regional efforts. CAI activities are taking advantage on the experience, approaches, tools and multistakeholder network developed under the STAQ Program. CAI is keen to expand its work on Freight Management. With support from USAID through Mexico’s Program for Low Emission Development and SEMARNAT, CAI is implementing a second of three phases for the preparation of a Freight Transport NAMA focused on logistics. 43 CAI is also finalizing the design of a MRV for Mexico’s Clean Transport Program and Fleet Modernization Program, funded by GIZ. In addition, CAI is successfully implementing with the Mexico-US Border Environmental Commission and SEMARNAT a training Ppogram on Eco- driving focused in the Northern Border, which is aimed to be expanded to the rest of the country. Discussions are being held with the “Corporación del Fondo Vial” to implement this program in Colombia. Other ongoing works include collaborations with the Mario Molina Center related to city wide air quality plans and travel demand management policies, starting with the Metropolitan Areas of the Toluca and the Valley of Mexico and aiming to expand this collaboration to other Mexican metropolis. CAI is also in process to materialize collaborations with UNEP to address black carbon emissions and to provide cities with guidance to formulate air quality management plans focused on cobenefits, as well as to provide technical assistance to design and implement specific policies and projects. These and other efforts, complemented with CAI activities as a Member of the Climate and Clean Air Coalition, observer of the CIF and other worldwide initiatives, are strengthening CAI’s financial sustainability and its strategic role to develop and conduct regional processes to improve air quality and mitigate climate change, by enabling, supporting and complementing national and local efforts. Final comments/Comments of the recipient on the Bank’s ICR Two of the three Regional Project’s Objectives were fully met, whilst one was partially achieved, resulting in this project earning the rating of “Moderately Satisfactory” (section 46 of ICR). This rating takes no account of the significant work undertaken to achieve compliance with the four Components despite the challenges highlighted in this document, with a significant number of these Component activities not fitting within the framework of the Objectives. Completion of the Components, however, was the day-to-day focus of the Bank and as such the rating should include achievements within this aspect of the Program. The work within the Components made considerable contribution to advancing sustainable transport in the region, and the Project was able to identify strategic opportunities in on-going local and international initiatives and institutions to partner with leading the Project to a successful implementation. Consequently, we disagree with the World Bank final project rating of “Moderately Satisfactory”. 44 Annex 5. Knowledge Platform Information PDOs PDO 1 Knowledge Platform Information - CAI-LAC – New Governance Documents http://www.cleanairinstitute.org/ial/ - CAI-LAC – New Webpages Establish a Network http://www.cleanairinstitute.org/ial/ - Preparation of AQ in LAC Report http://www.cleanairinstitute.org/calidaddelaireamericalatina/?pag=23&sec=289 Cities with enhanced capacities PDO 2 - Emissions Assessments Knowledge Platform(Helpdesk) http://www.cleanairinstitute.org/helpdesk/ - 11 Policy Papers http://www.cleanairinstitute.org/cops/documentos-de-politica/ - Communities of Practice/Webinars http://www.cleanairinstitute.org/cops/ http://www.cleanairinstitute.org/cops/webinars/ - Data Base of Regional Experts http://www.cleanairinstitute.org/AsistenciaTecnica/ - STAQ cities webpage http://www.cleanairinstitute.org/cai-f2.php?id_sitio=1&p_idioma=ESP&idp=39 Cities applying PDO 3 - Emissions Assessments Knowledge Platform (see Helpdesk) assessment tools - EF Database http://www.cleanairinstitute.org/factoresdeemisionLAC/ - On demand TA http://www.cleanairinstitute.org/AsistenciaTecnica/asistencia_contact.php 45 Annex 6. List of Supporting Documents - Project Appraisal Document and Aide Memoires of preparation and supervision missions, and ISRs in WBDoc - CAI Reports and Papers http://www.cleanairinstitute.org/cops/documentos-de-politica/ 46