Document of The World Bank Report No: NCO00004289 NOTE ON CANCELLED OPERATION REPORT (IBRD-81510) ON A LOAN IN THE AMOUNT OF US$ 180 MILLION TO THE REPUBLIC OF UZBEKISTAN FOR AN ADVANCED ELECTRICITY METERING PROJECT December 22, 2017 Energy and Extractives Global Practice Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective November 8, 2017) Currency Unit = = Uzbekistan Som (UZS) UZS 8,080.58 = US$1 FISCAL YEAR January 1 - December 31 Vice President: Cyril E Muller Country Director: Lilia Burunciuc Practice Manager: Sameer Shukla Project Team Leader: Mitsunori Motohashi NCO Author: Koji Nishida ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank kWh kilowatt-hours AEM Advanced Electricity Meter MDM Metering Data Management AMI Advanced Metering Infrastructure MW megawatts CPS Country Partnership Strategy PAD Project Appraisal Document DOS Disk Operating System PDO Project Development Objective EDM Energy Data Management PES Regional Distribution company GDP Gross Domestic Product PIU Project Implementation Unit HV High Voltage PLC Power Line Communication IBRD International Bank for Reconstruction and Development PMU Project Management Unit ICB International Competitive Bidding RF Radio Frequency IFI International Financial Development Institution UE Uzbekenergo IsDB Islamic Development Bank The World Bank (P122773) Republic of Uzbekistan Advanced Electricity Metering Project TABLE OF CONTENTS DATA SHEET A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Ratings of Program Performance in ISRs 1. CONTEXT, PROJECT DEVELOPMENT OBJECTIVES, AND DESIGN………………………………………………………….…………...1 2. POST-APPROVAL EXPERIENCE AND REASONS FOR CANCELLATION ................................................................... 6 3. ASSESSMENT OF BANK PERFORMANCE ................................................................................................................. 9 4. ASSESSMENT OF BORROWER PERFORMANCE .................................................................................................... 10 5. LESSONS LEARNED .................................................................................................................................................. 11 ANNEX. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES ........................................ 13 MAP Page 1 of 15 The World Bank (P122773) 1. CONTEXT, PROJECT DEVELOPMENT OBJECTIVES, AND DESIGN (drawing from PAD, briefly describe the country and sector background, rationale for Bank assistance, development objectives, components, costs and funding, implementation arrangements, risk analysis, and quality at entry): Country and sector background 1. Uzbekistan is a lower-middle-income, mineral-rich country with a population of 32.3 million as of July 2017, the largest in Central Asia, and an annual population growth of 1.7 percent in recent years. Over the past decade, Uzbekistan has maintained a high and stable economic growth rate and achieved gradual diversification by following a state-driven approach to development. According to official estimates, the annual gross domestic product (GDP) growth averaged 7.2 percent over 2000–2016 and contributed to a decline in the poverty rate from 27.5 percent in 2001 to 12.5 percent in 2016, making Uzbekistan one of the fastest growing economies in the world during this period. In the decade preceding the implementation of this project, some reform measures had been undertaken with the objective of improving the functioning of the energy sector, including functional unbundling of generation, transmission, distribution, and dispatch. A majority of the country’s power generation, transmission, and distribution assets are owned and operated by subsidiaries of a single holding company, Uzbekenergo (UE), composed of over 45 subsidiary companies. Its subsidiary, Energosotish, is the single buyer and supplier in the sector. Uzelectroset is the system operator providing dispatch, transmission, and network services, and includes seven high-voltage (HV) transmission network affiliate operators. Electricity distribution is managed by 14 regional electricity distribution companies. 2. Electricity losses of Uzbekistan’s power system have been high, estimated at 20 percent of net generation. The high losses are partly attributable to inaccurate and inefficient metering and billing activities. A majority of the existing electricity meters are still of electromechanical type and were manufactured between 1960 and 1990. These meters are well beyond their design life without proper recalibration and their measurement inaccuracy is expected to exceed the allowable tolerance. Residential meters in rural areas are typically installed inside residential houses, and consumers report readings by themselves. The billing system currently used at UE was developed in the late 1990‘s and the main system runs under the Disk Operating System (DOS) with manual inputs and adjustments. It is considered that those manual-based operations pose risks of inaccurate meter reading, inadequate monitoring and control, which contributes to electricity losses or revenue loss. The absence of modern metering and billing systems is one of the main obstacles for improving financial and operational efficiencies of the distribution system. 3. At the timing of project appraisal in 2012, Advanced Metering Infrastructure (AMI) technologies had emerged in the market as a result of technological advances and price reduction in meters and telecommunication equipment. The technology was widely adopted in countries such as Italy, Spain, and Brazil. AMI typically refers to the full measurement and collection system that includes meters at customer sites, communication networks between customers and a service provider, and data reception and management systems that make the information available to the service provider. The technology enables remote monitoring of electricity consumption and remote connection/disconnection of electricity supply (in the case of low-voltage consumers). AMI is considered as an effective tool to improve the accuracy of Page 2 of 15 The World Bank (P122773) measurement and to detect and eliminate commercial losses. 4. The Government of Uzbekistan (the Government) identified the Advanced Electricity Metering (AEM) program, which aims to implement a country-wide AMI technology as a key action in the power sector to reduce power system losses and to improve transparency and accountability of the sector. It was also expected to encourage electricity consumers to use electricity more efficiently by providing transparent and fair pricing signals, and information of their electricity consumption. At appraisal, UE had already spent US$32 million to install electricity meters capable of accurately measuring and reporting interval demand for high- and medium-voltage customers, power generation plants, and transmission-distribution interfaces. Several pilot AEM projects, which UE had also already completed, demonstrated some reduction of commercial losses and increase in revenue in the pilot areas. Building on the experience of the pilot projects, the Government requested the World Bank (the Bank), the Asian Development Bank (ADB), and the Islamic Development Bank (IsDB) to finance the scaling-up of the AEM program. The overall AEM program was composed of four-phased projects: the ADB project would cover the Samarkand, Bukhara, and Jizzakh regions under Phase 1, and the Andijan, Fergana, Kashkadarya, Namangan, and Surkhandarya regions under Phase 4; the Bank project would cover Tashkent City, and the Tashkent and Syrdarya regions under Phase 2; and the IsDB project would cover the Karakalpakstan, Khorezm, and Navoi regions under Phase 3. Rationale for Bank Assistance 5. The Bank’s Country Partnership Strategy (CPS) for Uzbekistan (Report No. 65028- UZ) for FY12-15 aimed to support the implementation of efficiency, competitiveness, diversification, and social equity elements of the Government’s medium-term development strategy. In this regard, increasing the efficiency of infrastructure (energy, drainage, irrigation and water management, and, potentially, transport) was fully aligned to the key priorities under the CPS where the Government and Bank Group views on sector-specific reforms and strategy broadly converged. In line with the Bank’s CPS and the Government’s strategic priorities, the Bank’s energy sector program aimed to support: (a) improvements in energy efficiency of industrial enterprises through an ongoing financial intermediary lending operation; (b) strengthening of the power transmission system to improve operational efficiency of the power transmission networks through an ongoing investment lending; and (c) updating the technical standards and improving financial management of UE through technical assistance. The Bank was also in discussion with the Government to provide technical assistance in electricity tariff setting and regulations and finance a power distribution system technical losses reduction project. 6. The AEM project was consistent with the first results area of the CPS, increasing the efficiency of infrastructure, as well as with UE’s development strategy issued by the Cabinet of Ministers and approved by the President of Uzbekistan in December 2010. It provided the Bank with a unique opportunity to: (a) support the Government’s policy priority of improving energy sector performance by commercializing utility operations and improving energy efficiency; (b) introduce the Bank’s rich experience in reducing commercial losses and strengthening governance of power utilities in meter reading, billing and revenue collections; and (c) assist UE to significantly advance its metering, billing and revenue collection systems with the most Page 3 of 15 The World Bank (P122773) advanced “smart” metering technologies. 7. The Bank had already implemented similar smart metering projects including ones in India and Latin America. The following lessons learned in the experience were reflected in the project:  In addition to technical solutions (installation of the AMI system), it is essential to implement a management efficiency improvement component to identify and reduce commercial losses, and re- engineer the business process for the operation of the new AMI system;  The smart meters should at least have the functions of fraud detection, and remote disconnection and re-connection for low-voltage consumers;  Sufficient training should be provided by equipment suppliers to the operation and maintenance staff of UE;  For residential customers, meters should be installed outside their premises to prevent meter tampering;  Strong commitment and ownership by the implementation agency is essential for the successful implementation of the project; and  Advanced preparation of bidding documents during the project preparation stage strengthens the project ownership and speeds up project implementation. 8. A feasibility study was prepared by an international consultant in 2010 financed by ADB together with the feasibility study for the ADB project. The feasibility study confirmed technical and economic feasibility of the AEM project, and identified optimal technology options. Specifically, the study confirmed that power line communication (PLC) be the primary technology and radio frequency (RF) for the remaining meters where PLC cannot be adopted for technical reasons. The study also recommended one centralized system architecture for AMI/EDM (Energy Data Management)/Billing/Archive systems since it has cost advantages and requires fewer local technical staffs, compared with the alternative, decentralized system architecture. Project Description, Costs, and Funding 9. The Project Development Objective (PDO) was defined in the PAD as follows: “to reduce commercial losses of Uzbekenergo’s three regional power distribution companies in Tashkent City, and the Oblasts of Tashkent and Syrdarya by improving their metering and billing infrastructure, and the commercial management systems.” Key indicators for the achievement of the above objectives were: (a) an increase in the billing rate; and (b) an increase in the collection rate. To this end, the project was to support the following three components:  Component 1: Advanced Metering Infrastructures (AMIs): (US$ 183.1 million, 75.7 percent of total project cost). This component includes supply, installation, and commissioning of: (i) advanced electricity meters (AEMs) for about 1.2 million low voltage customers and at low voltage sections of relevant distribution transformers of the three regional distribution companies; (ii) data concentrators, and communication systems among various sub-systems, the centralized management system in Tashkent and AEMs of both low voltage and medium and high voltage customers of the three regional distribution companies; and (iii) hardware and software of one metering data management (MDM) system per regional distribution company. Page 4 of 15 The World Bank (P122773)  Component 2: Energy Data management (EDM)/Billing/Archive System: (US$ 17.8 million, 7.4 percent of total project cost). This component includes: (i) supply, installation, customization and commissioning of the hardware and software, as well as training and after-sale services of a centralized EDM/billing/archiving center in Tashkent and its applications for the three regional distribution companies; and (ii) migration of all the existing billing data to the databases of the new EDM/billing system.  Component 3. Management Efficiency Improvements and Project Implementation Support: (US$ 5.0 million, 2.1 percent of total project cost). This component includes: (i) assessment of electricity consumption patterns and physical connection conditions of various customers to identify opportunities for reducing commercial losses, starting from large consumers and to develop and implement a loss reduction strategy; (ii) business process re-engineering and institutional capacity strengthening for the operation of the new AMI and better utilization of rich information collected by the AMI to reduce commercial losses; (iii) project communication program to educate consumers of the benefits of the new system, how it works and the responsibility of the consumers to pay the electricity bills on time; (iv) project implementation, supervision and administration; and (v) project entity and project financial audits and technical assistance for further customization of the EDM system. 10. The total project cost was estimated at US$ 246.1 million inclusive of price and physical contingencies, tax and duties, interest during construction and front end fee, with US$ 180.0 million financed by an IBRD loan and US$ 66.1 million financed by the Government. Installation of meters would be financed by the Government and implemented by UE. Implementation Arrangement 11. The Project Management Unit (PMU) was created within UE. The project was implemented by the PMU together with the Project Implementation Units (PIUs) created within the three Territorial Distribution Enterprises (PESs) in Tashkent City, and the Oblasts of Tashkent, and Syrdarya. An Inter-Ministerial Tendering Committee was established to conduct bid evaluation and make procurement decisions. An experienced supervision consultant was hired in 2015 to assist UE to implement the project. 12. The PMU was responsible for preparing the project plans, project progress reports, applications for withdrawal of funds, and any other reports required by the Bank. The PMU also assisted the tendering committee in conducting procurement and coordinated works for project preparation, implementation, and completion. The PMU consisted of a project director, project managers, procurement specialists, project accountants, social development/public relations officers, qualified technical experts, and other staff members. 13. The PIUs were mainly responsible for installation and commissioning of meters and systems. Those regional PIUs reported to the PMU and the senior management of the distribution companies. Risk Analysis 14. The overall project risk was rated as substantial due to the decentralized nature of the project, which Page 5 of 15 The World Bank (P122773) involves several sub-project sites and various sub-implementation agencies, and the PMU’s relative lack of experiences in the Bank projects, especially in the areas of procurement and financial management. To mitigate those risks, qualified consultants were recruited to assist the PMU and procurement and financial management trainings were conducted by the Bank team for related staff of UE at the entry stage of the project implementation. 2. POST-APPROVAL EXPERIENCE AND REASONS FOR CANCELLATION. (main events leading to cancellation, steps taken to resolve problems, exogenous factors, identification of causes and responsibility if project failed, implications of failure): Post-Approval Experience 18. The loan was approved by the Board on March 27, 2012. However, the Legal Agreement was signed only on April 10, 2013, and became effective on December 6, 2013, more than 20 months after the Board approval. The 20-month period was spent for the Government and UE to prepare and approve the country’s own feasibility study as per the national legislation. During the period, an advanced procurement, which worked well for another project implemented by UE during the same period, was discussed but not accepted by the PMU. Subsequently, an inclusion of pre-qualification process further delayed the project implementation. After eight months of discussions about the inclusion of the process and ten months for its full development, the pre-qualification was concluded in July 2015. The Bank finally provided no-objection to the bidding documents for the AMI package on November 26, 2015, and for the commercial management system package including EDM, billing, and archive system on March 29, 2016. After four years from the board approval, the tenders for the major packages were finally launched. A bid evaluation report for the AMI package, which contains supply and commissioning of 1.2 million low-voltage meters, data concentrators, and communication systems, was submitted to the Bank on July 21, 2016. The Bank provided no-objection on September 19, 2016, to award the contract to the lowest bidder in the amount of US$ 103,719,916.15. The contract amount was significantly lower than the initial cost estimate of US$ 150 million. The Bank issued no objection to the draft bidding documents for the commercial management system package on March 29, 2016. Subsequently, bid opening took place on July 27, 2016. The PMU informed that a single bid was received. 19. During the prolonged procurement process, there were frequent turnover of senior management and PMU staff, and changes in the Government’s priorities, which precluded the ownership of the project and hampered the continuity in project implementation. The Chairman of UE was replaced three times, and the PMU head was replaced twice during the whole project period (and there was no permanent PMU head after August 2016). The issue was further exacerbated by the Government’s transition period and subsequent changes in the Government, ministries, and agencies in late 2016. In the meanwhile, the Government renewed emphasis to promote import substitution of manufactured equipment in order to Page 6 of 15 The World Bank (P122773) encourage local businesses and job creation and to achieve the vision of becoming an industrialized middle- income country by 2030. The urge for higher local content was also prompted by the accelerated depreciation of the local currency (on the project’s final year, the depreciation pace was three to four times higher than ones for previous years.), making import products less economically attractive. The Government also initiated and gave high priority to a distribution grid modernization project that aims to address deteriorating supply reliability and high technical losses, over the AEM project which modernizes meter infrastructure aiming to reducing non-technical losses. 20. In the context of this country’s situation, after awarding to the lowest bidder for the AMI contract before the contract signature, UE renegotiated with the contractor to increase the level of local manufacturers’ participation and promote local production of meters as per an order from the Government without officially informing the Bank. The increased participation of local manufacturers and local production of meters would represent a major change in the responsibilities of the contractor stipulated in the bidding documents. This substantial change in scope was considered by the Bank as a major violation of the Procurement Guidelines (Clause 2.59). Moreover, there was no clear official communication from the Government or UE for over eight months despite Bank’s numerous reminders to the senior government officials about the impending Project Closing Date. On June 1, 2017, the Bank declared a misprocurement for the AMI contract, as per relevant provisions outlined in the Loan Agreement and the Procurement Guidelines, and a cancellation of the portion of the loan allocated for the contract. As for the commercial management system contract, the bid evaluation report was not submitted to the Bank. 21. The Government did not request for an extension of the Project Closing Date. On the basis of the aforementioned factors, including: the cancellation of a large portion of the loan; unlikeliness of achieving the PDOs; lack of official communications and without a clear signal for project extension, the Bank was compelled not to extend the Project Closing Date. The project therefore closed as scheduled on June 30, 2017 with a disbursement of US$1,163,176.22 for consulting services and front-end fees of US$450,000, totaling merely 0.6 percent of the loan. Reasons for the Project Outcome 22. The project was closed with a cancellation of the majority of the loan amount. Main reasons of the outcome are as follows: (i) Government’s priority to increase the local contents of AMI equipment; (ii) Counterparts’ insufficient ownership and leadership to implement the project; (iii) Poor communication and coordination between the PMU and UE senior management; and (iv) the country’s contract registration process: 23. (i) The immediate reason for the cancellation of a portion of the loan was the violation of the Bank Procurement Guidelines due to a major change in the scope of the AMI contract, as the Government demanded the awarded bidder to increase the local production of AMI equipment after bidding process was completed, even though such a condition was not included in the bidding documents, and was explicitly prohibited under Clause 2.59 of the Procurement Guidelines. Without the implementation of the AMI contract, accounting for 75.7 percent of the total project cost, the PDO could not be achieved and Page 7 of 15 The World Bank (P122773) therefore the project extension was not warranted. 24. (ii) Frequent turnovers of UE senior management and PMU staff discouraged ownership of the project, and hampered continuity in project implementation. The issue was exacerbated with the prolonged project preparation and procurement processes. The project spent 20 months for the preparation and approval of the country’s own feasibility study and another almost two years for the pre-qualification process. During the preparation stage, the project initially gained strong support and ownership from UE management. Subsequent to the management and PMU staff changes, however, the project lost strong promoters, diminished momentum, and thereafter could not even process small issues, such as contract registration, resulting in further delays. Project implementation deteriorated further between late 2016 and early 2017, especially on the registration of the AMI contract. According to the information provided by UE, all investment projects implemented by UE, including AEMP, were suspended during certain period, and were subject to scrutiny and reprioritization. The project showed almost no progress during this critical period. 25. (iii) The PMU was well staffed with sufficient technical capability but their effectiveness was low in terms of bringing issues to the UE senior management and expediting decision-making process. The Bank team experienced that even small decisions cannot be made within the PMU but had to be brought to UE’s management or higher. The head of the PMU should have had a direct reporting route to the top management. This was even more important than in normal circumstances, given the frequent turnovers of UE senior management. Moreover, the project implementation could have benefited more had PMU communicated more frequently with UE’s technical experts given the project complexities in terms of technologies and implementation arrangements. The PMU’s better understanding of how the technologies work and help achieve the project PDO was crucial to facilitate decision making and speed up the implementation. 26. (iv) As per the existing country’s regulations, the Government reviewed the awarded bid and contract only after the conclusion of the bid, instead of during the bid evaluation process. The Government demanded the awarded bidder to increase the portion of the local production by organizing a joint venture with local manufacturer(s), which constitutes a material change to the original bidding documents. Furthermore, this kind of substantial scope change after contract award could damage the country’s reputation in maintaining stable business environment. However, in September 2017, after the project closing, the Government has changed the regulation, which now requires the State Investment Committee to participate in bid evaluation processes and carry out price validation before contract awards. Page 8 of 15 The World Bank (P122773) 3. ASSESSMENT OF BANK PERFORMANCE (lending process/ensuring Quality at Entry, supervision and implementation assistance role, compliance with Bank policies, justification for rating): Rating: Moderately Satisfactory 27. The project was generally well designed taking into consideration lessons from past experiences. Economic and financial assessments were carried out, and the financial capacity of the Borrower was evaluated. Perceived risks were addressed through hiring an international consultant and conducting relevant trainings. The Bank team was well staffed in terms of the availability of technical expertise. Experienced AMI specialists, including an external consultant, were extensively involved in the project design and implementation. 28. The system architecture design of AMI and Billing System, which was deeply discussed among the ADB, the Bank, and the PMU, is considered appropriate. The Bank adopted a centralized system for data collection and management, to be installed at the Headquarters and covering both Tashkent and regional PESs, due to its cost effectiveness and less intensive requirements on local technical experts. On the other hand, the ADB chose three decentralized systems (one for each region) partly because their project preparation was ahead of the Bank-financed project, and they required a system that can be operated independently from the progress in other projects. Even though the two projects adopted different types of system architectures, no compatibility issues are envisaged. The decentralized systems financed by ADB will be able to operate under the centralized system installed in the country’s capital city under the project financed by the Bank. The centralized system, which was agreed by the Bank team, is considered optimal due to the cost and maintenance advantages. 29. In addition to the local contents issue, there were several discussions regarding the procurement package design, including the adoption of: (i) a phased or a full-scale approach for the Bank-financed project; (ii) a single or two different contracts for the AMI and Billing System components; and (iii) a pre- or post- qualification. After a thorough analysis, it was concluded that the procurement strategy agreed among the PMU, UE’s Management, and Bank team is appropriate as per reasons given below. (i) Given that the feasibility study and the pilot projects implemented by UE in the early 2000s already confirmed the feasibility of the technology, which has become standard, the phased-approach is not considered appropriate and would not have helped resolve critical issues leading to the result. While ADB accepted to adopt a phased approach, where milestones were set at critical points, according to the ADB team, the approach was not for ensuring the feasibility of the technology but for a training purpose to develop and ensure UE’s capability in handling the meter installation work at a very initial roll-out stage. Full scale approach adopted for the Bank financed project is considered appropriate because the technology was already proven in the feasibility study and UE already had experience in the installation work for pilot projects. (ii) Considering the market for AMI and billing system (purely IT system), the Bank team requested, and UE accepted, that AMI (meters + concentrators + data management system) and Billing System be procured separately in two different packages. The Bank team’s recommendation was based on other country’s successful experiences, which achieved the least-cost procurement outcomes with Page 9 of 15 The World Bank (P122773) sufficient competitions and faced no issues on interface between the two systems. The Bank team’s approach is considered appropriate given the above bases. (iii) Finally, the Bank team opposed to add pre- qualification process, on the ground that the process would significantly delay the project implementation without any real benefit because the technologies were proven and standard, bid preparation would not require high costs for bidders, and the number of firms expected to bid was considered acceptable to enable sufficient competition. However, UE kept insisting on having the pre-qualification and for this reason the Bank accepted the pre-qualification process to avoid further delays caused by disagreements. The Bank team’s recommendation not to include the pre-qualification process is considered appropriate given that the technology was already standard. 30. Supervision missions were fielded on a regular basis, although missions often had multiple agenda and could only spend a limited amount of time on each project. Initial progress was extremely slow owing to the prolonged Feasibility Study approval process. The Bank team made significant efforts to speed up the implementation by trying to implement advanced procurement and to discourage the pre-qualification process. The Bank team also tried to improve technical capability on the counterpart by organizing a technical workshop in 2014 regarding the state of the art AMI technologies and global experiences. Responding to the delayed contract signature for Contract 1, the Bank fielded four missions in September 2016, December 2016, January 2017, and June 2017, and sent six letters with sequential escalation to bring the matter to the attention of senior government officials and to urge diligent execution of the contract. However, there were no clear official communications from the Government over eight months. The Bank team’s supervision efforts were considered appropriate. 31. There should have been comprehensive stakeholder analyses upfront as well as more proactive stakeholder consultations throughout the project, to avoid potential oppositions from stakeholders who would lose from the roll-out of the AMI infrastructure. Given the technology’s impacts on commercial activities and accordingly on the winner-loser structure, comprehensive stakeholder analyses would help understand the overall impacts. With clear understanding of the political economy of the AMI infrastructure, the Bank team should have more proactively facilitated consultations with stakeholders including meter readers, end-users, and companies involved in the commercial activities. 32. On these bases, the Bank team performance is rated as moderately satisfactory. 4. ASSESSMENT OF BORROWER PERFORMANCE (government and implementing agency performance, compliance with covenants, justification for rating): Rating: Moderately Unsatisfactory 33. The visionary concept to roll out AMI technologies proposed by the Government aimed at reducing commercial losses and promoting energy efficiency is considered commendable given the country context. The concept is still highly relevant even today. However, the project could not materialize the ambitious Page 10 of 15 The World Bank (P122773) vision. 34. The frequent replacements of senior management of UE discouraged the commitment and deteriorated continuity of the project, resulting in lost momentum. The lack of strong support from the Government and UE management delayed the project implementation and made the project spend long time to resolve minor issues. 35. Generally, the PMU has been well-staffed and supportive. They have improved their capability over the implementation of the project. However, as pointed out above, the PMU should have been in a more effective position to bring issues directly to the UE management and expedite decision-making process. The head of the PMU should have had direct and effective communications with the UE management, which could have ensured smoother and quicker project implementation. The multiple layers of bureaucratic structures also made every decision-making process complicated and prolonged. Various ministries were involved in every decision making including the approval of the project feasibility study and registration of contracts financed under the project. It took very long for the implementing agency to resolve all comments received from various bodies. 36. The PMU was created inside UE while PIUs belonged to PESs. Distribution operations are managed by PESs and therefore the PIUs might have better understanding of distribution business and operations. However, the PIUs had little interventions on decisions made by the PMU. Given their better understanding of distribution, the expertise of the distribution companies should have been more effectively utilized in the project design and implementation. 37. The Government’s insistence on the local contents in the contracts eventually led to the unsuccessful implementation of the project. 38. On the basis of the commendable visionary project concept as a positive side as well as negative experiences including (i) insufficient commitment from Government and UE, (ii) PMU’s ineffective interventions on decision making, and (iii) insufficient involvement of the PIUs, the Borrower’s performance is rated as moderately unsatisfactory. 5. LESSONS LEARNED (both project-specific and of wide general application) 39. Following are the key lessons to be taken away from the AEM project:  The project confirms that strong and continuous commitment and ownership of the government and management are essential for smooth implementation of an installation of AMI systems. The management must have clear understanding of how the technologies would achieve project objectives. Otherwise, a project could lose momentum for implementation and be unable to overcome even smallest of challenges during implementation. Page 11 of 15 The World Bank (P122773)  The PMU must have direct and effective communication routes to management for smooth implementation of a project. Especially this is the case for Uzbekistan’s electricity sector, where management decisions are made by a single holding company, UE, to which various subsidiaries including thermal power plants and a transmission operator belong. The PMU, together with technical experts of the distribution companies should play a key role to explain to the top management the benefits of the AMI project and the technology options available. PESs, distribution subsidiaries, must be directly involved in the project design and implementation.  It is considered that the project design and technology selection are appropriate. However, if a new AMI project is to be developed, the design and technologies need to be updated to incorporate recent technological advances and other countries experiences. The key concept to be applied is to specify the performance of AMI system in terms of functionalities and reliability levels, and accept all the communication technologies available in the market able to provide those requirements meeting the reliability levels.  The AMI and billing technologies fundamentally transform utility’s commercial activities, leading to possible negative impacts on some stakeholders. Understanding upfront of political economy of AMI infrastructures including potential winners and losers is critical. Without it, the project might not be able to cope with resistances from stakeholders who would lose as a result of the introduction of new technologies. To mitigate the risk, stakeholder analyses must be carried out up-front followed by appropriate consultations with various stakeholders including meter readers, end-users, and companies involved in the commercial activities.  Technical experts in AMI and Commercial Management systems have to be heavily involved in project design, preparation and implementation on both the Bank team and the Government and UE sides. This will ensure clear understanding of not only technologies themselves but also benefits and impacts caused by the introduction of the technologies. Capacity enhancement activities are also recommended to mitigate implementation risks by allowing all stakeholders including the Government, utilities, suppliers, commercial trade entities and even electricity consumers to understand the benefits/impacts.  Advance procurement is an effective tool to expedite tender process and project implementation. Especially, in the case of Uzbekistan, where the country’s own feasibility study has to be approved after the Board approval of international financial institutions (IFIs), advance procurement plays an important role, which speeds up the project implementation immediately after the effectiveness of a project.  Collaboration with other partners are critical if they are undertaking similar projects. Proper collaboration enables both parties to learn from other parties’ practices and to implement standardized approaches. This is applicable not only to the preparation stage such as collaboration on financial arrangement and feasibility study but also to the implementation stage. Both parties should work closely and share information on various aspects during implementation such as technology selection, technical specifications, procurement strategy, environmental and social aspects, and installation work or construction. Page 12 of 15 The World Bank (P122773) Annex: Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Leiping Wang Senior Energy Specialist ECSS2 Task Team Leader Ghada Youness Senior Counsel LEGEM Sunil Khosla Senior Energy Specialist ECSS2 Galina Alagardova Financial Management ECSC3 Specialist Fasliddin Rakhimov Procurement Specialist ECSO2 Joseph Paul Formoso Senior Finance Officer CTRLA Iskander Buranov Energy Specialist ECSS2 Stefan Rose Consultant/Metering ECSS2 Arthur Kochnakyan Financial Analyst ECSS2 Irina Tsoy Communication Associate ECCUZ Bernard Baratz Consultant/Environmenta ECSCS l Warren Waters Consultant/Social EASIS Josephine Kida Program Assistant ECSS2 Elena Klementyeva Program Assistant ECCUZ Eugenie Muminova Translator/Interpreter ECCUZ Mohinder Gulati Lead Energy Specialist ECSSD Peer Reviewer Pedro Antmann Senior Energy Specialist SEGEN Peer Reviewer Franz Gerner Lead Energy Specialist EASVS Peer Reviewer Supervision/NCO Mitsunori Motohashi Senior Energy Specialist GEE03 Task Team Leader Page 13 of 15 The World Bank (P122773) Husam Mohamed Lead Energy Specialist GEE03 Beides Sunil Kumar Khosla Lead Energy Specialist GEE02 Pekka Kalevi Salminen Senior Energy Specialist GEE03 Task Team Leader Iskander Buranov Operations Officer GEE03 Koji Nishida Senior Energy Specialist GEE03 NCO Author Maksudjon Safarov Energy Specialist GEE03 Nurbek Kurmanaliev Senior Procurement GGO03 Specialist Djamshid Iriskulov Financial Management GGO21 Specialist Elena Klementyeva Program Assistant ECCUZ Galina S. Kuznetsova Senior Financial GGO21 Management Specialist Fasliddin Rakhimov Procurement Specialist ECSO2 Blaga Djourdjin Procurement Specialist ECSO2 Arcadie Capcelea Senior Environmental GEN03 Specialist Rebecca Emilie Anne Social Development GSU03 Lacroix Specialist (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including travel and Staff Time and Cost (Bank Budget Only) consultant costs) Lending 49 307 Total: 49 307 Supervision/NCO 79 468 Total: 79 468 Page 14 of 15 The World Bank (P122773) MAP Page 15 of 15