Document of The World Bank Report No: ICR2764 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H2320 TF-56536) ON A GRANT IN THE AMOUNT OF SDR 5.563MILLION $8.0 MILLION EQUIVALENT TO MONGOLIA FOR A INFORMATION AND COMMUNICATIONS INFRASTRUCTURE DEVELOPMENT PROJECT September 25, 2013 TRANSPORT, WATER, AND INFORMATION COMMUNICATIONS TECHNOLOGIES DEPARTMENT CHINA AND MONGOLIA EAST ASIA AND PACIFIC REGION CURRENCY EQUIVALENTS (Exchange Rate Effective March 31, 2013) Currency Unit = MNT MNT 1397.5 = $1.00 $1.49 = SDR 1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS $ All dollars are in United States dollars unless otherwise Indicated CIO Chief information officer CRC Communications Regulation Commission GDP Gross domestic product GPOBA Global Partnership on Output-Based Aid ICT Information and communication technologies ICTPA ICT and Postal Authority IDA International Development Association IRR Internal rate of return M&E Monitoring and evaluation MoF Ministry of Finance MTC Mongolia Telecom Company OBA Output-based aid PDO Project development objective PHRD Policy and Human Resources Development Fund PIU Project Implementation Unit PPP Public private partnership RfP Requests for proposals SDR Special drawing rights SOE State-owned enterprise SMMS Spectrum monitoring and management system USOF Universal service obligations fund Vice President: Axel van Trotsenburg Country Director: Klaus Rohland Sector Manager: Randeep Sudan Project Team Leader: Peter Silarszky ICR Team Leader: Peter Silarszky -i- MONGOLIA INFORMATION AND COMMUNICATIONS INFRASTRUCTURE DEVELOPMENT PROJECT CONTENTS Page 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN .............................................. 1 1.1. Context at Appraisal ..................................................................................................... 1 1.2. Sector Context .............................................................................................................. 1 1.3. Rationale for Bank Involvement................................................................................... 3 1.4. Original Project Development Objectives and Key Indicators..................................... 3 1.5 Revised PDO and Key Indicators ................................................................................. 4 1.6 Main Beneficiaries........................................................................................................ 4 1.7 Original Components.................................................................................................... 5 1.8. Other significant changes ............................................................................................. 6 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES................................................. 7 2.1 Project Preparation, Design and Quality at Entry .............................................................. 7 2.2 Implementation ............................................................................................................. 8 2.3 Monitoring and Evaluation Design, Implementation and Use ................................... 11 2.4 Safeguard and Fiduciary Compliance......................................................................... 12 2.5 Post-completion Operation/Next Phase ...................................................................... 13 3. ASSESSMENT OF OUTCOMES ................................................................................................. 13 3.1 Relevance of Objectives, Design, and Implementation .............................................. 13 3.2 Achievement of Project Development Objectives...................................................... 14 3.3 Efficiency ................................................................................................................... 15 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................ 17 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE ..................................................... 18 6. LESSONS LEARNED ................................................................................................................ 22 7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS ...... 23 ANNEX 1. PROJECT COSTS AND FINANCING ................................................................................. 24 ANNEX 2. OUTPUTS BY COMPONENT ........................................................................................... 25 ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS........................................................................ 28 ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES............. 29 ANNEX 5. BENEFICIARY SURVEY RESULTS .................................................................................. 30 ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS..................................................... 31 ANNEX 7. SUMMARY OF BORROWER'S ICR AND/OR COMMENTS ON DRAFT ICR ....................... 32 ANNEX 8. COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS .................. 47 ANNEX 9. LIST OF SUPPORTING DOCUMENTS .............................................................................. 48 - ii - A. Basic Information Information and Communications Country: Mongolia Project Name: Infrastructure Development Project Project ID: P092965 L/C/TF Number(s): IDA-H2320,TF-56536 ICR Date: 09/29/2013 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: MONGOLIA Original Total XDR 5.56M Disbursed Amount: XDR 4.70M Commitment: Revised Amount: XDR 5.56M Environmental Category: C Implementing Agencies: Communications Regulation Commission (CRC) Information, Communications Technology and Post Authority (ICTPA) Cofinanciers and Other External Partners: Government of Japan Policy and Human Resource Development (PHRD) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 01/18/2005 Effectiveness: 07/26/2006 07/26/2006 Appraisal: 04/10/2006 Restructuring(s): 03/21/2012 Approval: 06/08/2006 Mid-term Review: 06/24/2009 Closing: 03/31/2012 03/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Satisfactory Overall Borrower Satisfactory - iii - Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 34 5 General information and communications sector 5 25 Telecommunications 61 70 Theme Code (as % of total Bank financing) Infrastructure services for private sector development 29 Other Private Sector Development 14 10 Regulation and competition policy 29 20 Rural services and infrastructure 28 70 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg Jeffrey S. Gutman Country Director: Klaus Rohland David R. Dollar Sector Manager: Randeep Sudan Philippe Dongier Project Team Leader: Peter Silarszky Tenzin Dolma Norbhu ICR Team Leader: Peter Silarszky ICR Primary Author: Gaurav Relhan Andrea Ruiz-Esparza - iv - F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project's long-term development objectives are to: (a) significantly increase the coverage and use of relevant ICT services among the rural population through an incentive program designed to encourage the participation of private operators in the rural segment of the ICT market; and (b) increase private sector participation in the delivery of e-Government services, thereby improving public sector utilization of ICT. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Increased number of telephony minutes originated in soum centers with about Indicator 1 : 1,000 (or more) inhabitants Value 1.2 million annual 2.7 million annual 56.5 million annual quantitative or minutes minutes minutes Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Target greatly surpassed due to the overall increase in the availability and usage Comments of mobile telecommunications services in Mongolia on the soum level. The (incl. % project financed extension of digital telephony into 90 soum centers directly achievement) contributed to this. Increased number of Internet users in soum centers with about 1,000 (or more) Indicator 2 : inhabitants Value about 12,000 11,000 Internet quantitative or 300 Internet users Internet users Qualitative) users Date achieved 06/23/2006 03/31/2012 12/31/2012 Target fully achieved due to overall increase in availability/usage of mobile data Comments services and better internet services at the soum level. Extension of mobile (incl. % services and broadband Internet services to soums funded by the project were achievement) critical for this. Increased number of telephony minutes originated in rural areas outside of soum Indicator 3 : centers (herder network) Value about 530,000 0 annual minutes in 1,200,000 annual quantitative or annual 2005 minutes Qualitative) minutes Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments The Actual Value is a conservative estimate, as precise measurement of these (incl. % minutes is not possible. The shortfall is attributed to the fast mobile networks achievement) expansion resulting in herders using the superior mobile services instead of -v- herder network. Indicator 4 : Continued additional annual investment in the ICT sector Value quantitative or $37.6 million in 2005 $140 million $395 million Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Fully achieved due to the continued strong private investments in the sector. Comments There are two basic channels through which the project contributed to this: (a) (incl. % improving the policy and regulatory environment; and (b) promoting investments achievement) in the rural areas. Indicator 5 : Number of e-Government services provided in partnership with the private sector Value quantitative or 1 4 3 Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Partially achieved. The project made significant contributions through Comments improving the enabling environment and awareness-raising. These were (incl. % essential for the acceptance of the private sector as a partner in delivery of e- achievement) Government services by Government. Number of Government ministries and agencies willing to accept the PPP model Indicator 6 : for e-Government Value quantitative or 1 10 10 Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Achieved. The project made significant contributions toward the acceptance of Comments the PPP model for the implementation of e-Government services by the (incl. % Government through improving the enabling environment, capacity building, achievement) training, and awareness-raising. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Component 1: Increased number of soum centers with about 1,000 or more Indicator 1 : inhabitants with access to digital telephony services Value (quantitative 120 200 360 or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments (incl. % Achieved—all soum centers have access to modern telephony services. achievement) Component 1: Increased number of soum centers with about 1,000 or more Indicator 2 : inhabitants with access to Internet services Value 15 100 360 - vi - (quantitative or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments Achieved. All soums have access to mobile data; 34 soums have access to high (incl. % speed internet. achievement) Component 1: Increased number of herder communities with access to Indicator 3 : telephony services Value (quantitative 0 350 192 or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments Partially Achieved. Shortfall due to the fact that mobile coverage extended much (incl. % faster than expected and, thus, the herder network was scaled back. achievement) Component 1: Approve legislation to ensure funding for the Universal Service Indicator 4 : Obligation Fund (USOF) The Law on Value Legislation is Government Funds (quantitative N/A approved approved in July or Qualitative) 2006 Date achieved 06/23/2006 06/23/2007 12/31/2012 Achieved. This indicator monitors whether USOF's sustainability is ensured by Comments appropriate funding. Since the project focused on rural ICT sector investments, (incl. % it helped to achieve this as GoM regarded the project as an incentive to approve achievement) the legislation. Component 2: Increase in percentage of spectrum applications approved and Indicator 5 : assigned within 15 working days Value (quantitative 50% 80% 90% or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments Achieved due to the spectrum monitoring and management system and improved (incl. % capacity of CRC funded by the project. The capacity of CRC was built through achievement) foreign trainings, in-country organized trainings and seminars, and study tours. Indicator 6 : Component 2: Regulatory public consultation process established Value Regulatory public Consultation (quantitative N/A consultation process established or Qualitative) process established in 2007 Date achieved 06/23/2006 06/23/2008 12/31/2012 Comments Achieved due to improved capacity of CRC. The project funded foreign (incl. % trainings, in-country organized trainings and seminars, and study tours for CRC achievement) staff. Indicator 7 : Component 2: Increase in percentage of regulatory decisions published Value (quantitative 58% 100% 100% or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 - vii - Comments Achieved due to improved capacity of CRC. The project funded foreign (incl. % trainings, in-country organized trainings and seminars, and study tours for CRC achievement) staff. Indicator 8 : Component 3: Policy guidelines on PPPs approved Parliament of Mongolia passed Value policy on PPP in Policy guidelines (quantitative N/A October 2009. approved or Qualitative) Concession and PPP law approved in January 2010 Date achieved 06/23/2006 06/23/2008 12/31/2012 Achieved due to the work on the enabling environment. The draft legislation and Comments other deliverables developed by a consultant financed under the project have (incl. % been used during the development of the newly approved Concession Law of achievement) Mongolia. Indicator 9 : Component 3: Number of CIOs of ministries, agencies and aimags trained Value (quantitative 0 70 255 or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments Target surpassed due to the capacity building subcomponent. Government (incl. % officers trained both abroad and in Mongolia. achievement) Indicator 10 : Component 3: Establishment of Government Portal Portal not Value Government portal implemented and (quantitative N/A established subcomponent or Qualitative) cancelled. Date achieved 06/23/2006 06/23/2008 12/31/2012 Comments Not Achieved. After the elections in mid-2012, the new Government decided (incl. % against the continued implementation of the Portal under the Project and achievement) cancelled the related uncommitted funds. Component 3: Percentage of target users using on-line e Government services in Indicator 11 : the areas supported by the project Value (quantitative N/A 30% 0% or Qualitative) Date achieved 06/23/2006 03/31/2012 12/31/2012 Not Achieved. After the elections in mid-2012, the new Government decided Comments against the continued implementation of the e-Government services under the (incl. % Project. Thus, there are no operational e-Government services directly supported achievement) by the Project. Component 3: Number of Government ministries/agencies with personnel Indicator 12 : trained in PPPs Value (quantitative 1 10 85 or Qualitative) - viii - Date achieved 06/23/2006 03/31/2012 12/31/2012 Comments Target surpassed due to the capacity building subcomponent. About 360 (incl. % government officials have participated in training abroad and within Mongolia achievement) and partially financed by the project. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 01/30/2007 Satisfactory Satisfactory 0.30 2 01/10/2008 Satisfactory Satisfactory 0.64 3 06/29/2009 Satisfactory Satisfactory 4.53 4 06/29/2010 Satisfactory Moderately Satisfactory 5.69 5 07/10/2011 Satisfactory Moderately Satisfactory 6.73 6 12/25/2012 Satisfactory Moderately Satisfactory 7.28 7 03/29/2013 Satisfactory Satisfactory 7.30 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions There was one restructuring on March 21, 2012 which was to extend the closing date by 12 months. In addition, the reallocation of proceeds from the IDA grant H232 was requested by GOM from 'Unallocated' and categories that did not require full utilization due to savings to categories that 03/21/2012 N S MS 7.06 were overdrawn. Several categories were overdrawn due to higher than estimated cost of the Spectrum Monitoring and Management System (SMMS), unexpected costs associated with the necessity to move the site of the fixed monitoring station, unusually high bank fees for the irrevocable confirmed documentary credit - ix - ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions for SMMS, and higher than expected cost of goods and incremental operating costs. I. Disbursement Profile -x- 1. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES AND DESIGN 1.1. Context at Appraisal 1.1.1 Country background. Mongolia is divided into 21 provinces (aimags), 342 soums and districts (urban and rural divisions), and 1,681 baghs and horoos (urban and rural communes). With a population density of only about two people per square kilometer, Mongolia is one of the most sparsely populated countries in the world. In addition, a significant share of the country’s 2.8 million citizens leads a nomadic lifestyle as herders, while the rest reside in villages, towns, and cities. 1.1.2 Over the past 20 years, Mongolia has transformed itself from a socialist country to a vibrant multiparty democracy with a booming economy. The Government has been focusing on improving economic competitiveness and productivity through attracting investments and reducing the economic and social isolation of the rural population. Driven by the exploitation of its vast mineral resources, the economy grew by 12.4 percent in 2012, compared to 6.4 percent GDP growth in 2010. GDP is expected to continue to grow at a double digit rate over the period from 2013 to 2017. This economic growth has translated into some benefits for the people of Mongolia. Poverty has been on a downward trend, having decreased from 39 percent in 2010 to 27 percent in 2012. But economic growth has also led to an increase in disparities, at the household and regional levels. There is a widening gap between urban and rural areas in terms of access to income and services. 1.1.3 Increased economic prosperity has led to an increased demand for information and communication technologies (ICT) with the market growing at a compound annual rate above 25 percent in the early 2000s. At the end of December 2005, total fixed and mobile teledensity reached approximately 26 percent, up from 13 percent in 2001. 1 However, Mongolia presented unique challenges to the further development of ICTs. As a sparsely populated country with almost half the population clustered in the capital of Ulanbaatar and a significant portion of the population living a nomadic lifestyle, it had been difficult for ICT companies to provide access to services outside of main population centers and make headway into rural Mongolian society illustrated by a 1 percent teledensity in rural areas in 2005. 1.2. Sector Context 1.2.1 Recognizing the potential of ICT infrastructure, applications, and services to enhance competitiveness, productivity, and the efficiency of institutions, the Government of Mongolia (GoM) began implementing since the mid-1990s a comprehensive telecommunications sector reform program. This led to effective liberalization of all market segments, partial privatization of the fixed line incumbent, Mongolia Telecom Company (MTC), and establishment of the Communications Regulation Commission (CRC) as an independent regulatory authority. The regulatory environment also evolved with the inception of interconnection, tariff, and licensing regimes conducive to competition. As a result, the 1 The number of telephone connections for every hundred individuals living within an area. -1- telecommunications sector received considerable foreign investments and witnessed tremendous growth. 1.2.2 During this period, GoM developed a framework and strategy for universal access to telecommunications services and for the use of the universal service obligations fund (USOF) to support the strategy. The USOF itself was established with the purpose of accumulating and disbursing funds for the construction of new networks, expanding or renovating existing networks, and to provide services to remote areas and to populations without access. The focus was on achieving an appropriate and self-sustainable level of access to telephony and public internet/information services in all underserved areas, particularly rural soums that are unlikely to see sufficient private investment without intervention in the form of financial support. 1.2.3 Through implementation of the Government’s E-Mongolia strategy (2005), the government intended to establish the foundation of a knowledge-based, information society in Mongolia by enhancing extensive application of ICT in all sectors. Some of the main strategic directions of the E-Mongolia strategy were the: • establishment of an improved legal and policy environment for ICT development, • creation of adequate and quality information and communications infrastructure, • use of ICT in government to build a citizen-centered and results-oriented government. 1.2.4 But despite the initial successes reaped in the ICT sector, GoM faced a number of challenges towards achieving its development objectives stated under the E-Mongolia strategy. These included: • Huge disparity in the availability of ICT infrastructure and services between the rural and urban areas of Mongolia. For example, in 2006, over 80 percent of existing phones and Internet users were in Ulanbaatar and other commercial areas. Teledensity in rural areas stood at about 1 percent. Virtually no services existed outside of the soum centers, at the bagh level, where the vast majority of residents are seasonally nomadic. • Inadequate capacity to proactively regulate and promote the development and implementation of a growing ICT sector agenda. With the implementation of the first phase of sector reforms, CRC, the regulator, needed to build institutional capacity to consolidate reforms, further promote competition, and address convergence in the ICT sector. However, CRC lacked the capacity or resources to do so. While regulations had been established to deal with a number of critical issues, including licensing, interconnection, tariffs, long-distance, and management of limited resources such as numbers and radio spectrum, CRC had to take steps to improve the regulatory environment in this area as well as enhance competition in the sector. • High cost of service provision and challenges of attracting private investments in rural Mongolia. Given the country’s vast and challenging geography, the cost of providing rural communications infrastructure was very high and ensuring access difficult and challenging. While MTC is responsible for provision of rural services, it had limited resources and capacity to implement the rollout of rural services in a timely and efficient manner. On the other hand, private operators are not mandated to -2- provide services in rural Mongolia and had been unwilling to expand services in areas that are commercially unviable, without support of capital investments from GoM. • Limited ICT policy leadership, implementation capacity, and resources in the government to deliver on its ICT vision and related action plan. While the GoM had an ambitious ICT policy agenda, the institutional arrangements (such as funding, coordination capacity, human capacity and skills) for delivering a comprehensive program in this sector were weak. Despite the announcement of an e-Government master plan, Mongolia lacked many of the key enablers of e-Government such as infrastructure, interoperability frameworks, standards, and an adequate legal and regulatory framework. In addition, Mongolia had a fledgling albeit growing private sector with limited capabilities and resources to fully participate in the country’s ICT growth potential. 1.3. Rationale for Bank Involvement 1.3.1 The project was deemed consistent with the FY05-FY08 CAS 2 core strategic objectives of: (a) consolidating the transition to a market economy by improving the regulatory and legal environment for increasing private sector participation in the provision of infrastructure; and (b) reducing rural vulnerabilities by supporting new approaches to improve the environment for local economic growth. The project aimed to support the latter objective by improving the country’s ICT infrastructure and thereby reducing Mongolia’s characteristic remoteness and creating new opportunities for its rural people. The project was also aligned with the government’s Economic Growth and Poverty Reduction strategy. 1.3.2 Further, World Bank support was deemed vital to: (a) leveraging increased private investments in the sector, and establishing a robust and transparent approach for public- private partnerships (PPP) in the provision of rural infrastructure access; and (b) providing resources to finance the design and implementation of key ICT policy and regulatory priorities that would foster private sector led growth in the sector and improve public sector use of ICTs. 1.4. Original Project Development Objectives and Key Indicators 1.4.1 The project's long-term project development objectives (PDO) were to: (a) significantly increase the coverage and use of relevant ICT services among the rural population through an incentive program designed to encourage the participation of private operators in the rural segment of the ICT market; and (b) increase private sector participation in the delivery of e-Government services, thereby improving public sector use of ICT. 1.4.2 The key performance indicators of the project are summarized in Table 1. 2 World Bank. 2004. Mongolia - Country assistance strategy. Washington, D.C. http://documents.worldbank.org/curated/en/2004/04/3125709/mongolia-country-assistance-strategy -3- Table 1. Key Performance Indicators Detailed Outcome Measures (to be tracked as key Project Development Objective Outcome Indicators performance indicators) Increased number of telephony minutes originated in soum centers with about 1,000 or more inhabitants Evidence of a significant increase in the use of relevant Increased number of internet ICT services among the users in soum centers with (i) To significantly increase the population of participating about 1,000 or more inhabitants coverage and use of relevant ICT rural soums services among the rural population Increased number of telephony through an incentive program minutes originated in rural designed to encourage the areas outside of soum centers participation of private operators in (herder network) the rural segment of the ICT market, Number of e-Government and (ii) Increase private sector services provided in partnership participation in the delivery of e- with the private sector Government services, thereby Number of e-Government improving public sector use of ICT. services provided in Number of government partnership with the private ministries and agencies willing sector and available to to accept the PPP model for e- businesses and citizens. Government Continued additional annual investment in the ICT sector 1.5 Revised PDO and Key Indicators 1.5.1 None. 1.6 Main Beneficiaries 1.6.1 Original: • Through extension of basic telephony services 3 into about 350 to 400 rural herder communities at the bagh level outside of soum centers that did not have access to telecommunications and information services, the project envisaged benefitting at least 45,000 people living in rural herder communities. • By increasing the number of soum centers with access to digital telephony and internet services, along with providing institutions such as schools with access to internet, the project aimed at delivering digital telephony and internet services to users (citizens, private enterprises, and public sector entities) living and/or based in about 45 soum centers. 3 Through the establishment of a herder network utilizing satellite technologies. -4- • By facilitating better spectrum management, increased capacities of public sector institutions (such as CRC and ICTPA), and e-government applications, the project would stand to benefit the entire population of Mongolia. 1.6.2 Revised: None 1.7 Original Components 1.7.1 The project had three main interrelated components aimed at achieving the project development objective: (a) Component 1: Information and Communications Infrastructure Development in Rural Areas ($ 7.04 million inclusive of taxes and duties) to support the implementation of the GoM’s universal access strategy in order to accelerate the provision of telephony and Internet services in rural Mongolia. In particular, service providers were to be awarded “least-cost’’ capital subsidies to provide public telephony services at the bagh level; and telephony and Internet services at the soum level, including Internet services for strategic vanguard institutions such as schools. Specific subcomponents were: (a) capital subsidy support to provide telephony services at the bagh level (for herder communities); (b) capital subsidy support to provide telephony and Internet services for soum centers; and (c) technical assistance support for consultant services to design the bidding documents for the OBA schemes (described above), and to manage the implementation of these programs. (b) Component 2: Communications Regulatory Development ($2.01 million inclusive of taxes and duties) was aimed at providing investment support and technical assistance to strengthen the sector regulator, CRC. Specific subcomponents were: (a) financing a radio spectrum management and monitoring system; (b) technical assistance support to finance the preparation of bidding documents for the radio spectrum management and monitoring system; (c) technical assistance to CRC in the areas of licensing, spectrum policy, monitoring of the implementation of the universal access program, and specialized regulatory advice; and (d) training for staff in areas such as regulation of competition, public consultative processes, interconnection, price regulation, resolution of disputes, consumer rights and complaints, accounting for costs and new trends in regulation. (c) Component 3: Public-Private Partnerships (PPPs) for e-Government ($2.46 million inclusive of taxes and duties) to address gaps in the enabling environment for use of PPPs for provision of e-Government services and to roll-out e-Government applications using PPPs. Specific subcomponents were: (a) technical assistance for developing the policy, legal and regulatory frameworks for PPPs in e-Government; (b) consultancy services for conducting a detailed feasibility study to identify PPP project opportunities for e-Government, rank order them and develop requests for proposals (RfPs) to roll-out those applications with the most potential; (c) technical assistance for developing a project in the area of e-Government; (d) investment support for government’s participation in implementing priority e-Government project(s) in partnership with the private sector; (e) technical assistance for developing an information security framework; and (f) training of ICTA staff and Chief Information Officers (CIOs) of government ministries, agencies and -5- aimags/soums to deal with e-Government issues and to help them use PPPs to leverage funds from the private sector for the provision of e-Government services. (d) Component 4: Project Management ($ 0.35 million inclusive of taxes and duties) for providing support to cover the operating costs of the Project Implementation Unit (PIU) that had managed the project. 1.7.2 Revised components. Components were not revised. 1.8. Other significant changes 1.8.1 Detailed design of the universal access component. Based on the results of the rural access pilot, developments in the telecommunications sector (rapid expansion of mobile coverage to rural areas in particular), and feedback from the telecommunications operators, the detailed design of the universal access component was revised in June 2007. It was decided that separate subsidy competitions will be held for voice telephony and Internet services in soum centers and the available financing was reallocated within the component. 1.8.2 The reallocation of proceeds from ‘unallocated’ and categories that would not require full use due to savings to categories that were overdrawn of the IDA grant H232 was requested by GoM in a letter dated April 6, 2011. Several categories were overdrawn due to higher than estimated cost of the spectrum monitoring and management system (SMMS), unexpected costs associated with the necessity to move the site of the fixed monitoring station, unusually high bank fees for the irrevocable confirmed documentary credit for SMMS, and higher than expected cost of goods and incremental operating costs. Table 2. % of Expenditures Allocation (SDR) Financed Category of Expenditure Original Revised Original Revised (1)(a) Goods under Part B of the Project 522,000 850,000 75 75 (1)(b) Goods under other Parts of the Project 24,000 38,000 100 100 (2) Capital Subsidies under Part A of the project 3,687,000 3,250,000 100 100 (3) Government Investment Support under Part C of the Project 661,000 661,000 100 100 (4) Consultants' Services, including audits of Project accounts 454,000 560,000 100 100 (5) Incremental Operating Cost 45,000 100,000 100 100 (6) Project Preparation Advance Financing 107.38 107.38 100 100 (7) Unallocated 169,892.62 103,892.62 TOTAL 5,563,000 5,563,000 -6- 1.8.3 The closing date for Grants No. H232 and TF 56536 (P09296) was extended by one year from March 31, 2012 until March 31, 2013 to allow for the implementation of the two e-Government applications (Public Key Infrastructure and Certification Authority and Portal System) to be completed within the extended closing date. This extension had no effect on the achievement of the original project results other than the time delay. 1.8.4 After the elections in mid-2012, the new GoM decided against the continued implementation of the two e-Government applications (Public Key Infrastructure and Certification Authority and Portal System) under the project. Subsequently, GoM requested the cancellation of the uncommitted funds of special drawing rights (SDR) 842,000 allocated for this subcomponent in a letter dated March 27, 2013. This cancellation was processed on March 29, 2013 effective March 27, 2013. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Soundness of background analysis. The proposed project was consistent with the CAS core strategic objectives of: (a) consolidating the transition to a market economy by improving the regulatory and legal environment for increasing private sector participation in the provision of infrastructure; and (b) reducing rural vulnerabilities by supporting new approaches to improve the environment for local economic growth. The latter is achieved by improving the country’s ICT infrastructure and thereby reducing Mongolia’s characteristic remoteness and creating new opportunities for its rural people. Drawing upon lessons learned from three Global Partnership on Output-Based Aid (GPOBA) financed pilot universal access projects implemented prior in Mongolia, the project was also consistent with GoM’s own ICT Vision for 2010, its e-Government Master Plan of 2005, and also earlier donor assistance in the telecom sector. The output based aid (OBA) approach enabled use of the right incentive framework to delineate the role of the public and private sectors. Reliance on private sector operators for service provision through a competitive bidding process that minimized the subsidy for up front capital expenditures was found to be an effective approach for serving remote and poor areas. 2.1.2 Assessment of project design. The project design was kept simple and adapted to the limited government capacity, primarily as a telecommunications infrastructure project implemented through public/private partnerships. Complex business systems design and major business process changes were avoided. Finally, the implementation schedule was kept realistic and there was intensive supervision in the first year (at a cost of some $250,000) which enabled the ‘hand holding’ needed at the initial stages. 4 Project financing involved an IDA grant of SDR 5.563 million ($8.0 million equivalent at the time of negotiations), a Government of Japan-financed Policy and Human Resources Development Fund (PHRD) co- financing grant of $2.0 million, and a GoM contribution of $1.85 million. The Bank budget preparation costs were relatively modest at some $120,000. The outcome indicators focused on quantifiable results for which the project could reasonably be held accountable. 4 All dollars are United States dollars unless otherwise indicated. -7- 2.1.3 Adequacy of GoM’s commitment. Preparation of the project involved extensive consultation with various concerned government agencies and received strong government commitment. GoM’s ownership of the project was partly through the ICT Vision 2010 paper, the e-government Master Plan and the participation in the design of the OBA approach for rural access. There was also clear assignment of responsibilities by Government among its various agencies. The GoM also exhibited its commitment by co-financing an amount of $1.85 million, representing over 15 percent of total project costs. This demonstrates the importance of this project for the Government as it was very rare for GoM to contribute to project costs at that time. An inter-agency/ministry steering committee and an adequately staffed PIU were also established before project launch. The PIU was responsible for monitoring and evaluation, reporting to the Project Steering Committee, GoM, and the World Bank. The only shortcoming noted was that staff of this new client was neither familiar with nor trained in Bank procedures at the beginning of implementation, which delayed start up. 2.1.4 Assessment of risk at appraisal. Specific critical risks were identified, of which that involving weak institutional capacity of policy and regulatory entities was rated substantial. Mitigation measures including capacity building to strengthen CRC and ICTA’s institutional capabilities; use of consultants to assist in the development of various policy and regulatory toolkits and assist in the bid specifications for the rural access tenders; development of in- house capacity to address policy and regulatory issues and administer the USOF; and setting- up of a PIU for management and coordination support were integrated into the project design and monitoring plans. But overall, the project was assigned risk rating of ‘Modest’ in view of the following: The objectives were modest in terms of scope- namely, expansion of telecommunications and internet services, and policies to encourage public private partnerships (PPP). For a country with a challenging geography and a highly dispersed population, the DOs were found to be relevant as the largest component focused on rural access, particularly on smaller population centers. The reliance on PPP also minimized the reliance on government institutions which, as mentioned, have limited capacity. 2.2 Implementation Implementation Review and GoM performance. 2.2.1 The project was implemented successfully, for the most part, in that it had achieved (and in many instances surpassed) its original outcome indicator targets, most significantly in the area of telecom access at the soum and bagh levels as summarized in Annex 2. The borrower was fairly effective in resolving implementation issues, although there were not many of them. A shortcoming noted during implementation was delays in collection of some of the data needed for M&E, due to nonavailability of some baseline data and difficulties in obtaining information from private operators. But this improved over the course of the project. 2.2.2 The government agencies concerned were able to keep implementation proceeding on track, except for the e-government subcomponent, with the largest component for rural access having surpassed its anticipated outcomes. Consultations with the private sector and other stakeholders became more transparent and inclusive over the project period (concerns of corruption/cronyism within government agencies had traditionally hindered private-sector collaboration). Finally the ICT project’s PPP framework was used by the GoM to enact a law embodying this PPP concept for all sectors, which is a valuable contribution. Overall, implementation progressed on track with disbursements only slightly behind the original -8- schedule. By March 21, 2012, $7.14 million (SDR 4.64 million) had been disbursed from the IDA grant, representing over 83 percent of the IDA grant proceeds, and $1.68 million had been disbursed from the PHRD grant, representing over 83 percent of the PHRD grant proceeds. Restructuring and Extension of Closing Date 2.2.3 As noted in 1.8.1 and 1.8.2, the reallocation of proceeds of the IDA grant H232 was requested by GoM from ‘Unallocated’ and categories that would not require full use due to savings to categories that were overdrawn. Several categories were overdrawn due to higher than estimated cost of the SMMS, unexpected costs associated with the necessity to move the site of the fixed monitoring station, unusually high bank fees for the irrevocable confirmed documentary credit for SMMS, and higher than expected cost of goods and incremental operating costs. The closing date for Grants No. H232 and TF 56536 were extended by one year from March 31, 2012 until March 31, 2013 in order to allow for the implementation of the two e-Government applications (Public Key Infrastructure and Certification Authority and Portal System and Portal System) to be completed within the extended closing date. See below for an explanation on the reasons behind the delay in implementing E-Government applications. 2.2.4 “Cannibalization” of the herder network. The project design didn't foresee the rapid expansion of mobile networks, which led to curtailment of the herder network (a satellite based network providing basic telephony services to rural herder communities at the bagh level outside of soum centers) expansion. This phenomenon is not without precedent as the development of mobile networks, due to technology development and competition, has surpassed even the most optimistic forecasts in most of the world. In 2006, the rapid expansion of mobile networks which brought mobile coverage to all soums before the end of 2008, in a country with the world's lowest population density, was not anticipated, given that only about 100 out of the 330 rural soums had mobile coverage in 2006. This made the herder network obsolete in many areas and made it suitable only as for deep rural areas. Ultimately, however, the herders received a superior service. 2.2.5 Inability to develop E-Government applications. While the project has contributed to the increase in private sector participation in the delivery of e-Government services, the two targeted e-Government applications (Public Key Infrastructure and Certification Authority; and Citizen Internet and Government Administrative Intranet Portals and Virtual Private Network) could not be implemented. The project design acknowledged risks and foresaw the potential problems in this direction, and therefore applied only 10 percent of the funding to try something new with potentially high benefits. 2.2.6 The reasons for inability to develop the E-Government applications are twofold: (a) Since these two applications were to be implemented as PPP, this approach was new for the Government and it took some time to finalize the PPP model to apply as well as to develop the bidding documents. (b) After the elections in 2012, the new Government decided against the continued implementation of these applications under the project (although contracts for the implementation of both applications had been awarded). Therefore, the earlier considered second extension of the closing date to allow for the implementation of -9- these e-Government applications was not processed. Consequently, the IDA funds allocated to them (SDR 842,000) were canceled before project closing and reallocated to Mongolia's IDA16 allocation. 2.2.7 Delays in project execution during 2006-2010, their effects, and implications for future projects are identified in Table 3. Table 3. Delays in Project Implementation Delay Implication for Future Cause of Delay Period Effect of Delay Projects Enabling environment Key stakeholders were unprepared preparation must precede future Delay in PPP to participate. efforts to introduce new enabling methods of doing business. If 6 Months environment Project and ICTA were perceived enabling environment consultancy by stakeholders as not having preparation is delayed, all political backing downstream activities should be delayed as well. The Land Registration system delay resulted in a wait of approximately 1 month while the consulting team was in country, Agencies and target systems before the decision was made by should be identified prior to ICTA to drop the system, and work project inception, or if in the on RFP development could case of a feasibility study continue. targeting a wide range of agencies/systems, supported by ICTA choice of ICTA claimed to have buy-in from a separate enabling two systems agency leadership and instructed environment consultancy, ALL (Land the consultants to develop potential agencies affected by Registration and specifications for the IPRMIS 6+ Months the new business model should IPRMIS) without system. Development of these be the focus of the enabling buy-in from specs took about 2 months. After environment preparation. responsible the specs were developed, it took agencies another 3 months of ICTA trying Lead agency (client) personnel to get approval from agency must not order work to proceed leadership before the system was on systems or initiatives until finally dropped. after formal (documented) buy- in has been obtained from other Because ICTA instructed the agencies which will take part consultants to develop technical specifications for the IPRMIS system, significant resources were expended in a wasted effort. - 10 - The decision to move to Two RFPs developed for Single Stage Stage procurement was bidding were redeveloped to fit necessitated not by the technical Two Stage requirements. complexity of the systems, but by the complexity of adopting Because World Bank had no the PPP model. The World standard bidding documents or Decision to switch Bank’s lack of standard bidding procedures adapted for e- from Single to document for PPPs in e- Government PPP, an inordinate Two Stage Government contributed to the amount of time was spent sending bidding to 6 Months problem. If e-Government iterations of the bidding documents accommodate PPPs are to be prioritized, either back and forth between the World complex PPP a) adapted standard bidding Bank and the consultants as all of structure documents should be produced, the implications were ironed out. or b) the bidding approach should be identified during Contract addendum was needed to project design and work on the provide for additional consultant contractual aspects of the labor outside scope of original bidding documents should contract. begin at project inception. The Feasibility Study output of Phase 1 was delivered in March of 2007, nearly a full 3 years before the Concessions Law was adopted, meaning that the absence of legal Again, the enabling underpinning for operating e- environment preparation must Delay in passage Government services under a PPP begin with enough lead time for of Concessions 2+ years was deemed risky by some parties. all necessary issues to be Law As a result, some agencies were addressed prior to attempting to unwilling to commit to a PPP solicit bids for systems. approach (e.g., Land Registration and IPRMIS), and ICTA and its consultants could not finalize the contractual conditions of the bids until the law was passed. Because agency leadership in Mongolia is extremely cautious about the risk of being accused of Until the political environment cronyism/corruption by an in Mongolia matures, it is incoming administration in case of probably best to avoid a change of ruling party, for a scheduling system Two National 1 – 1.5 period of approximately 6-8 procurements which span an Elections years months leading up to a State Great election period, i.e., project Khural legislative election, the timeframes should clearly be decision making process in ICTA contained within one election was effectively paralyzed. In 2008 cycle. and again in 2012, essentially no decisions related to the subcomponent could be made. 2.3 Monitoring and Evaluation Design, Implementation and Use 2.3.1 Arrangements for monitoring and evaluation (M&E) were overall adequate. The PIU was responsible for monitoring and evaluation, reporting to the Project Steering Committee, GoM, and the World Bank. PIU was stipulated to carry-out monitoring and reviewing the project performance every six months and provide semi-annual report no later than January 31 and July 31 of the subsequent half-year. The capacity to collect and analyze data was supported by resources from the project. While there were delays in collection of some data - 11 - due to difficulties in obtaining information from private operators in areas not supported by the project, attempts were made to reduce the burden of data provision by private operators. 2.3.2 M&E use. The M&E framework was useful in monitoring implementation progress, signaling issues and prompting the client and the Bank to make adjustments to enhance the project’s ability to realize its objectives. Data was gathered on the growth of mobile networks, and based upon this, funding was diverted from herder network (which is an inferior service) to mobile network, which is a superior service. This led to an increased number of soum centers to be provided with telecom infrastructure, than originally planned. 2.4 Safeguard and Fiduciary Compliance 2.4.1 Environment. The project was classified as Category “C,” and was thereby not expected to and did not pose significant environmental impacts. As an example of environmental mainstreaming within the project, small satellite terminals provided to herder communities for network services can be moved when herders move to new grazing grounds. 2.4.2 Procurement was organized and carried out by the PIU with oversight from the Steering Committee. Procurement of goods was carried out on the basis of international competitive bidding, shopping and direct contracting. The Bank’s oversight included prior and post reviews of procurement and contracting procedures to ascertain that the Procurement Guidelines were followed and the activities were in compliance with the legal documents. There were no major policy or institutional issues to be resolved as none had come up during project implementation. Several key capacity constraints identified at inception were resolved by the Borrower or were improved markedly over time, particularly because of the mitigating measures that were introduced, which included, most saliently, the comprehensive capacity building program to strengthen CRC and ICTA’s institutional capabilities in this regard. Various toolkits and new regulatory policies were introduced while ensuring that bid documents were properly designed and contained appropriate specifications and evaluation criteria. International consultants were provided through the Japanese grant trust fund to advise and assist on procurement related issues. 2.4.3 The effectiveness of some procurement actions remained an issue affecting the timeliness and orderly implementation of the project, but this over time also improved. The midterm review pointed out that: “the project implementation has made good progress (around 65 percent of the available funds have been disbursed and about 75 percent of the available amount has been committed). Most of the packages have been effectively committed.” No fraud and corruption issues were uncovered or reported during implementation. Despite procurement risk rated as ‘high’ at appraisal (on account of client’s limited capacity and experience with World Bank procurement policies), overall procurement experience was smooth with all of the procurement activities under the project completed. Consecutive ISRs point to a steady improvement in the procurement process. This may partially also be attributed to the fact that over the course of the project’s implementation GoM had improved its public financial management, adopted new anti-corruption legislation, and collected/ published asset and income disclosures for top officials. 2.4.4 Financial management. The PIU maintained dedicated accounts for the project and prepared financial statements on a regular basis. Annual audit reports were submitted to the Bank on time and no significant issues had been identified. It was noted that expenditures were properly supported by adequate documents, approved by authorized persons and - 12 - recorded into the project accounts accurately and timely. Supervision reports noted no shortage or delay of funds hindering fund flow or disbursement. The financial management arrangements were strengthened by separate monitoring of implementation of the main output-based components, comprising about two-thirds of project expenditures, through a technical audit undertaken by an independent verification agent. It was concluded that the project’s financial management system was adequate and proceeds of the funds were used for their intended purposes. 2.4.5 Social aspects. The project was designed to contribute to reducing the 'friction of distance' and economic and social isolation of those living in rural Mongolia, and as such was in sync with GoM’s strategy to promote inclusive growth. Particular attention was paid to meeting the needs of remote and seasonally nomadic herding communities, which account for a significant share of those in Mongolia living below the poverty line. In addition, the promotion of e-Government services as part of the project was intended to contribute to improving public access to information in remote rural communities and thereby enhancing transparency and accountability in public service provision. 2.5 Post-completion Operation/Next Phase 2.5.1 The rural access component of this project does not impose any additional future financing requirements on the government as it was based on an OBA approach where a one- time subsidy by the government is awarded to the operator in return for a commitment to provide the ICT services to the population. The rural access component of the project was designed to be sustainable based on the pre-feasibility studies carried out by GoM which estimated an internal rate of return (IRR) of 25 percent. The operators that were awarded the rural access contracts knew that their operation must be commercially sustainable after the initial subsidy from the government. In addition, the establishment and operationalization of the USOF created a mechanism for financing the major future telecommunications and ICT infrastructure needs of rural Mongolia, barring unforeseen developments or needs. 2.5.2 The project was designed to be fully self-sustainable at completion. As a result of this project, regulatory staff received training aimed at increased capacity of staff and effectiveness of government processes. Furthermore, as sector revenues increase, the amount of revenues collected by the regulator from the operators has increased. Today, the sector regulator CRC is fully self-financed. CRC has its own income from sector fees, which it uses for its ongoing operations, capacity building, and investments in regulatory tools (including co-financing of the radio spectrum management and monitoring system implemented under the project). 2.5.3 The subcomponent on PPP approach for e-Government applications was cancelled. After the elections in mid-2012, the new GoM decided against the continued implementation of the two e-Government services under the project implement these using its own sources. 3. ASSESSMENT OF OUTCOMES 3.1 Relevance of Objectives, Design, and Implementation Rating: Highly Satisfactory - 13 - 3.1.1. The relevance of the project objectives, design and implementation remains high. The project is consistent with the current CAS 5 core strategic objectives of: (a) consolidating the transition to a market economy by improving the regulatory and legal environment for increasing private sector participation in the provision of infrastructure; and (b) reducing rural vulnerabilities by supporting new approaches to improve the environment for local economic growth. The objectives are fully aligned with GoM’s e-Government Master Plan of 2005 (which emphasizes improved regulatory environment for ICT development; ICT-enabled Government for citizen-focused services; and improved ICT infrastructure), GoM’s ICT Vision for 2010 (providing a blueprint for the development of ICT in Mongolia), as well as with GoM’s 2021 ICT vision. By enhancing rural productivity and competitiveness (and thereby spurring economic development in the hinterlands) through rendering of ICT services, the project is also aligned with the government’s Economic Growth and Poverty Reduction Strategy and is also found to be coherent with the World Bank Group’s ICT Sector Strategy of 2012. The project adopted a combination of physical components, subsidies, and TA that proved effective for it to meet its objectives. 3.1.2. The provision of mobile services rendered by the project has been found to benefit various ongoing World Bank projects in improving their deliverables quality. For example, the Mongolia Livestock and Agricultural Marketing Project takes advantage of mobile services to collect the data for provision of index-based livestock insurance. The Mongolia Global Partnership Early Childhood Education project takes advantage of internet services made available in soum centers for rendering of web-based instruction and skill enhancement. 3.2 Achievement of Project Development Objectives Rating: Satisfactory 3.2.1 The project greatly surpassed its first PDO, i.e., it significantly increased the coverage and use of relevant ICT services among the rural population through an incentive program designed to encourage the participation of private operators in the rural segment of the ICT market. While mobile voice services were available only in about 100 rural soums in 2006, mobile voice and data services became available in all 330 soum centers in Mongolia within the first 3 years of project implementation and the project played a critical role in this. The project directly financed capital subsidies for the establishment of the services in 90 soums across Mongolia. In addition, the USOF established with the help of World Bank funded technical assistance during project preparations funded capital subsidies for the establishment of the services in about 50 additional soums. Finally, the pilot which preceded the project and the project itself helped the operators to realize that there is significantly more demand for services than they expected resulting in operators establishing services in the remaining soums without subsidies on purely commercial basis. Moreover, over 150 rural herder communities gained access to public telephony service at the bagh level; 34 soum centers have broadband Internet access for public and private users including schools connected at discounted rates; and a modern SMMS was successfully installed and is operating. 3.2.2 The target of increasing number of internet users in soum centers (to at least 11,000 users) was achieved. Provision of internet access in soum centers funded by the project 5 Report No. 67567-MN: Country Partnership Strategy for Mongolia for the Period 201-2017 - 14 - contributed to the rise in internet users in these soums. In addition, the extension of digital telephony services to all soum centers partially funded by the project also helped, as in almost all cases, mobile internet services are available wherever modern digital telephony services are available. An overall increase in availability/usage of mobile data services and better internet services at the soum level also helped in meeting the target. 3.2.3 The intermediate indicator associated with enhancing telephony access to herder communities was only partially achieved, owing to the fact that the herders used the faster- than-expected extension of mobile network coverage as a substitute to the herder network – but there is no way to identify and measure these minutes. 3.2.4 The indicator on continued additional annual investment in the ICT sector greatly exceeded forecasts due to the continued strong private investments in the sector. There are two basic channels through which the project contributed to this: (a) improving the policy and regulatory environment (the project supported this through Component 2 including the spectrum management and monitoring system) which typically attracts additional private sector investments; and (b) promoting investments in the rural areas which were mostly unattractive for private investments until only a few years ago (both directly by providing subsidies and indirectly through demonstrating that services can be provided sustainably also in rural areas). 3.2.5 The project also contributed to the second PDO, namely, increase in private sector participation in the delivery of e-Government services and improved public sector utilization of ICTs. This was done by improving the enabling environment, capacity building and awareness-raising for PPPs in general and for the implementation of e-Government services under PPP arrangements in particular. The project achieved this through delivering: (a) PPP framework for e-Government services and related policy, legislation and regulation; (b) demand study and identification of e-Government services suitable for implementation under PPP arrangements; (c) information security strategy and framework and related policy, legislation and regulation; and (d) capacity building and training. These were essential for the acceptance of the private sector as a partner for e-Government services by the Government, and thereby, helped to increase the number of e-Government services provided in partnership with the private sector and number of Government ministries and agencies willing to accept the PPP model for e-Government. As noted above, the project fell short of delivering on the two pilot e-Government applications due to the initial in-country unfamiliarity with PPP models, and then later (after political changes), by the decision of the new government to cancel the implementation of these services despite the fact that the associated contracts had been awarded through a competitive process. However, the e- Government applications were a relatively small part of the project (less than 10 percent of project funding) and the overwhelming success of the much larger Component 1 outweighs the failure to implement the e-Government applications. 3.2.6 Overall, the project has surpassed or fully achieved four of the total of six PDO indicators and the other two have been partially achieved. The project has also achieved nine of intermediate indicators out of a total of twelve. 3.3 Efficiency Rating: Satisfactory - 15 - 3.3.1 The economic and financial analyses were conducted for the rural access component (Component 1) at the time of appraisal. The nature of the rest of the project was not amenable to a quantitative economic and financial analysis. We replicated the analyses using the same methodology and actual post-project data (for more details see Annex 3). 3.3.2 Our financial model, using a discounted cash flow methodology, calculates that the IRR over 10 years enjoyed by the operator of the herder network (telephony services at the bagh level) is 22 percent which is somewhat lower than the 28 percent estimated at appraisal. The average IRR over 10 years enjoyed by the operators of the soum level mobile networks is 37 percent which is significantly higher than the 25 percent estimated at appraisal. 3.3.3 It is estimated that the establishment of the bagh-level herder network and the expansion of the mobile networks has reduced the distance (time and cost) to communication from an average of 25 to 39 km to access the nearest telephone of any kind to 14 to 16 km. The resulting economic benefits include reduced costs, improved and strengthened family ties, improved security and coordination associated with migration and response to harsh weather, increased economic activity, and more efficient governance and social infrastructure activities. In addition, the soum level mobile services provide economic benefits from improved access to telephony and Internet services demonstrated by strong demand for the new services. The services better integrate the soum with the aimag center and national capital and allowed social infrastructure and administration (e.g., education, health, governance) and market-based services to become more efficient at the soum level. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 3.4.1 The project’s overachievement of its objective of providing ICT services to the rural population, its contribution to increasing private sector participation in the delivery of e- Government services, its high continued relevance to economic and social development of Mongolia, and its financial soundness merit an overall Satisfactory rating for Outcomes. 3.5 Overarching Themes, Other Outcomes, and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 With the objective of providing greater availability of and access to ICT services for some of Mongolia’s poorest rural citizens, it is estimated that the project enabled about 1 million Mongolians living in rural areas, villages, and small towns to gain access to telephony services. This was achieved by both expanding mobile services and establishing a bagh-level herder network and by spurring sector-specific reform along with the requisite capacity building. The improved telecom access helped these communities overcome isolation, enhance economic activity, and promote social integration with the rest of the country. The herder public access network, combined with the mobile services, reduced the average travel distance required to make a call for herders to 15 km from 39 km. In most of the soum centers, there used to be less than 10 phone lines often limited to a payphone and a few lines in the government offices; today, there are several hundred active mobile lines in almost all, if not all, soums with most of families having at least one phone. (b) Institutional Change/Strengthening - 16 - 3.5.2 Improving capacity of CRC to better regulate the rapidly expanding sector and further empowering it through the installation of a modern SMMS was an important project component, which was achieved. Several consultancy assignments have been successfully completed providing transaction support for the acquisition of the Spectrum Monitoring System, and technical assistance in the areas of spectrum policy, numbering, and HR development at CRC. Technical assistance to support the development and implementation of a new licensing policy was also completed in early 2013. Overall, as a result of the regulatory strengthening component, Mongolia’s telecommunications regulator has developed the capacity to regulate a largely private sector driven telecommunications sector, providing the needed certainty for operators to make large investments to provide telecommunications infrastructure and services. CRC also has the capacity and tools to undertake important regulatory functions such as licensing, tariff regulation, spectrum management, quality of service monitoring and regulatory compliance. 3.5.3 In the area of the delivery of e-Government services, the project helped to improve the enabling environment for PPP in general. This is reflected in the fact that by end of project, it was found that at least 10 government ministries and agencies were willing to embrace the PPP model for development of e-Government services and four such e-Gov services are being developed through the PPP model. Further, in e-Government, the project helped propel development of policies which led to the drafting of legislation that became a basis for a law approved later by the Parliament (in modified form). The project also helped developed an information security framework, and financed a related capacity building and awareness rising program. 3.5.4 The extent to which the subsidy model would produce results was unexpected and the demand for services was underestimated. Prior to the project, only about 40 soums were covered with mobile network as mobile operators pointed to the non-profitability of providing services in rural areas. Using the subsidy model, a pilot was conducted through which the operators realized the potential in these areas. As a result, the mobile operators ventured into more soums on their own, eventually covering all 330 soums in the country, many without using any subsidy (the World Bank subsidized provision of services in 90 soums, GoM subsidized up to 60, while the remaining were covered by the operators on their own). 3.5.5 Perhaps the most noteworthy contribution to the institutional strengthening was familiarizing the private sector on providing telecom services to over 150 far-flunked rural herder communities at the bagh-level. (c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.6 None identified. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME Rating: Moderate 4.1 The risks to maintaining the development outcome as well as the risks to the overall institutional framework for sustaining the achievements are moderate. The risk of weak - 17 - institutional capacity risk was identified during preparation, and addressed by training in and out of country. Given the capacity limitations of the Government, the Bank team worked to maximize reliance on the private sector in future operation of the telecommunications and IT service provision, while continuing to build capacity in relevant government agencies through consultancy support and training. Training provided to regulatory staff resulted in reduced costs and increased efficiency and effectiveness of government processes. 4.2 With the establishment and operationalization of the USOF, the major telecommunications and ICT infrastructure needs of rural Mongolia are satisfied to a large extent and GoM should be able to finance future projects using the USOF, barring unforeseen developments or needs. Furthermore, as sector revenues increase, the amount of revenues collected by the regulator from the operators has increased. Today, the sector regulator CRC is fully self-financed. CRC has its own income from sector fees, which it uses for its ongoing operations, capacity building, and investments in regulatory tools. 4.3 The rural access and PPPs in e-government components of this project do not impose additional financing requirements on the Government beyond those described in the PAD as they are based on an OBA approach and a one-time government investment in equity or otherwise towards the PPPs, respectively. The OBA projects supported by this project were also designed to be sustainable based on the pre-feasibility studies carried out by GoM which estimated an IRR of 25 percent. The operators that won the OBA tenders knew up front that their operations must be commercially sustainable. 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 5.1.1 As summarized in Sections 1 and 2, the Bank’s support for the project was well aligned with the prevailing CAS, GoM’s own ICT Vision for 2010, its e-Government Master Plan of 2005, and complemented several ongoing operations. Saliently, the project built upon previous initiatives by PPIAF on PPPs in Mongolia as well as GPOBA pilot projects implemented in the country. Objectives and components of the three partners (IDA, PHRD, and GoM) were dovetailed and well integrated into the design of the project. Working relationships with the Borrower, stakeholders and co-financiers were productive and supportive. During preparation, the Bank team assisted GoM in designing the project to meet the Bank’s technical, financial, economic, fiduciary, social and environment standards. It also helped shape the training programs to strengthen the institutional capacity of the relevant government and ICT sector authorities, the PIU, and other stakeholder organizations. Project design was kept simple and adapted to the limited government capacity with overall risk rightly rated as moderate. The Bank budget preparation costs were relatively modest at some $120,000. 5.1.2 The project introduced innovations such as using incentives to the private-sector for expanding telecom services across the country, international standards and good practices, and establishment of an enabling environment for applying PPP frameworks in Mongolia’s - 18 - ICT sector. The outcome indicators focused on quantifiable results for which the project could reasonably be held accountable. 5.1.3 The fact that the project design didn't foresee the curtailment of herder network by the rapid expansion of mobile network is not unusual, as the development of mobile networks historically outperformed even the most optimistic forecasts in most of the world. In 2006, the unprecedented expansion of mobile networks, which brought mobile coverage to all soums before the end of 2008, in a country with the world's lowest population density, was not anticipated, given that only about 100 out of the 330 soums had mobile coverage in 2006. This made the herder network obsolete in many areas and made it suitable only as for deep rural areas. Ultimately, however the herders received a better service, so, this should be seen a success not as a failure. The project design acknowledged risks associated with the implementation of e-Government applications as PPPs but, given the significant potential benefits, it was decided to follow this approach and try to mitigate the associated risks. 5.1.4 Overall, quality of Bank performance at entry had several strengths, which have been decisive in overcoming key challenges and ensuring good progress towards the key indicators. First, the team focused closely on the development effectiveness during project design. Second, the Bank team was proactive by asking for the sector to be moved to the Office of the Prime Minister, where it received more high level attention than hitherto. Third, the risk of weak institutional capacity was identified during preparation, and addressed by training programs, within and outside of the country. (b) Quality of Supervision Rating: Satisfactory 5.1.5 Supervision of the project was regular and vigorous. The Bank task team conducted twice yearly supervision missions, with several visits to outlying towns and rural areas to verify and experience mobile and satellite services that have already been rolled out, service quality, and the reception of users. The supervision team had all the needed skills, with technical support from the Bank’s leading specialist in e-government and e-applications generally, besides staff with skills and experience in other areas. The team made good use of the supervision budget, given the relatively modest allocation of $70,000 per year, which remained flat during the implementation period. Even then the team has managed to bring in the Lead Specialist to help supervise the e-government component, when that component faced delays. Similarly, the team also managed by piggy-backing several countries to arrange for a relatively good part of the team to join each supervision mission. 5.1.6 The procurement and financial management aspects were carefully supervised by the Bank team and no F&C issues have been uncovered or reported during implementation. In terms of procurement oversight, considering the overall weakness of the implementing agencies, the risk mitigation measures put in place from inception worked very well, particularly the comprehensive capacity building program to strengthen CRC and ICTA’s institutional capabilities that was organized. Procurement difficulties in the first year were identified, responded to, and resolved by the team with an intensive supervision effort. The Bank not only supervised but supported the project timely and diligently, working with other donors to provide international consultants who were employed with funding from the Japanese PHRD grant. A two stage procurement process for the e-Government services was proposed by the team to address the limited experience with PPP in Mongolia. In addition, - 19 - various toolkits and new regulatory policies were introduced, while ensuring that bid documents were properly designed and contained appropriate specifications and evaluation criteria. All supervision aide-memoires report on procurement issues with no major findings. Ex-post reviews demonstrate most contracts have been properly awarded and only minor procedural issues have been found. No supplemental audits or independent procurement reviews appear to have taken place. Project sites were visited regularly and this has been properly documented in the aide-memoires and supervision reports, with no high value/high risk contracts identified. Supervision missions have been quite detailed and thorough with only minor or procedural issues found. These have been highlighted and corrective actions taken in a timely and efficient manner. 5.1.7 Fiduciary aspects were properly reflected in aide-memoires with adequate integration of procurement and FM supervision into overall project supervision. The numerous financial management inputs to supervision activities somewhat exceeded the requirements of the assessed moderate level of risk. Partly, this was aimed at preemptive identification of issues and helping the client to resolve them. 5.1.8 The only shortcoming identified in supervision was the lack of oversight for some of the social aspects. As noted in Social Aspects (para 2.4.5) more could have been achieved by better capturing benefits disaggregated by social groups and gender. This represents, possibly, a missed opportunity to capture significant positive social development outcomes from the project. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 5.1.9 Bank performance is rated satisfactory. The Bank engaged with the Government initially through Bank executed technical assistance in the sector. Significant expert resources were invested for knowledge transfer activities and to familiarize the client with market based approaches to providing services in rural areas. The project team kept the project design simple and ensured implementation readiness that allowed on-time completion of the rural connectivity component. Adequate resources were allocated for implementation support and supervision, with ongoing transfer of knowledge to the client on policy, regulatory, and project implementation issues. 5.1.10 The project was assessed by QAG as part of the Second Quality Assessment of the Lending Portfolio (QALP-2) in 2010. Quality of Design was assessed as Satisfactory and Quality of Bank Supervision as Highly Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 5.2.1 The project received overall strong government commitment. GoM’s ownership of the project was partly through the ICT Vision 2010 paper, the E-Government Master Plan and the participation in the design of the OBA approach for rural access. The GoM exhibited its commitment by co-financing an amount of $1.85 million, representing over 15 percent of - 20 - total project costs. This demonstrates the importance of this project for the Government as it was very rare for GoM to contribute to project costs at that time. There was also clear assignment of responsibilities by Government among its various agencies, and overall, the government agencies concerned had been able to keep implementation proceeding on track. GoM also used the project’s PPP framework to enact a law embodying this PPP concept for all sectors, which is another indication of the government’s responsiveness and interest in the project. In addition, no F&C issues had been uncovered or reported during implementation. 5.2.2 Overall, GoM helped facilitate implementation to progress on track with disbursements only slightly behind the original schedule. By March 21, 2012, $7.14 million (SDR 4.64 million) has been disbursed from the IDA grant, representing over 83 percent of the IDA grant proceeds, and $1.68 million had been disbursed from the PHRD grant, representing over 83 percent of the PHRD grant proceeds. 5.2.3 While the project has contributed to the increase in private sector participation in the delivery of e-Government services through improving the enabling environment and awareness-raising for the implementation of e-Government services under PPP arrangements, the two pilot e-Government applications targeted by the project could not be implemented as the PPP approach was new for the Government and it took some time to finalize the PPP model to apply as well as to develop the bidding documents (see Table 3). Later, following the elections in mid-2012, the new GoM decided to cancel the implementation of these applications despite the fact that the associated contracts had been awarded through a competitive process. Therefore, the earlier considered second extension of the closing date to allow for the implementation of these e-Government applications to be completed could not be processed. Consequently, the IDA funds allocated to the implementation of these two e- Government applications ($1 million) were canceled. 5.2.4 Another shortcoming noted was that staff of this new client (for client executed projects) was neither familiar nor trained in Bank procedures at the beginning of implementation, which delayed start up. However, the initial slow and sluggish processes were corrected and set on the right track. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 5.2.5 An interagency/ministry steering committee and an adequately staffed PIU were established before project launch. The PIU was responsible for project management, monitoring and evaluation, reporting to the Project Steering Committee, GoM, and the World Bank. During the course of implementation, it was found that the PIU was committed to achieving the PDO as the unit made major strides in efficiency, management and oversight of project execution, M&E, financial management procurement, and promoted the project’s longer-term sustainability. Focus on capacity building was apparent as staff often participated in training activities to enhance their skills in carrying out their management and procurement functions. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory - 21 - 5.2.6 Given the government’s satisfactory performance (relentless corrective actions, commitment and support to the project) combined with satisfactory role of the PIU (exemplary efficiency, management, oversight, and coordination skills), overall performance of GoM is considered satisfactory. 6. LESSONS LEARNED 6.1 It is important to ensure that adequate upfront preparation is done prior to project approval. The World Bank engaged for the first time in the ICT sector in Mongolia, initially through Bank executed TA. The CRC implementation of the GPOBA pilots helped the client learn from market approaches to universal access, and also familiarized them with World Bank procurement and financial management procedures. This allowed for the successful and timely completion of the rural access component. 6.2 In projects of such nature, where private sector participation is a critical success factor, it is important to ensure credible stakeholder consultation. The client ensured that it consulted widely with private sector in order to gauge both their interest and design a practical approach to providing access in rural areas. 6.3 The results of the implementation of the universal access program demonstrates that an OBA subsidy auction can work to reach “beyond the market” for both rural telephony and Internet services by attracting private sector investments. The project proved that there is significant untapped demand for mobile services in rural areas. However, the Internet service provision developed more slowly, focusing on those soums which have the best chance of sustaining commercial Internet services after the award of the one-time subsidies and have “Internet-ready” schools that can most benefit from the service. 6.4 Key factors which contributed to the success of the universal access component of the project are summarized as follows: • CRC’s consistent and committed leadership in the management of USOF • Very positive and supportive response of the operators to support USOF and compete for the subsidies in challenging markets • The World Bank’s involvement with $5 million seed money to the USOF, guaranteeing the operators a benefit ahead of their contributions into the fund. 6.5 On Component 3, it was realized that a more conservative approach is required for accessing the capacity and speed at which e-government solutions can be implemented, especially when adopting a PPP approach. Preparation of enabling environment must precede efforts to implement e-Government services—if enabling environment preparation is delayed all downstream activities should be delayed as well. Beneficiary agencies and target services should be identified prior to project inception. 6.6 The World Bank’s lack of standard bidding document for PPPs in e-Government contributed to the delays. If e-Government PPPs are to be prioritized, either (a) adapted standard bidding documents should be produced, or (b) the bidding approach should be identified during project design and work on the contractual aspects of the bidding documents should begin at project inception. - 22 - 6.7 In addition, the project’s failure of delivering on the two pilot e-Government applications is linked to electoral changes—in a country like Mongolia where political and business interests overlap—political changes due to democratic elections sometimes lead to different business decisions which affect project implementation. Until the political environment in Mongolia matures, it would be preferable to avoid scheduling system procurements which span more than one election cycle. 6.8 Continuity in the Project Implementation Unit is key for satisfactory project implementation. Over the project period, the PIU director was changed only once. There was continuity in fiduciary staff working on the project. Despite change in government and leadership at the implementation agencies, the PIU was a constant that helped keep the project on track. 7. COMMENTS ON ISSUES RAISED BY BORROWER/IMPLEMENTING AGENCIES/PARTNERS (a) Borrower/implementing agencies (b) Co-financiers (c) Other partners and stakeholders - 23 - ANNEX 1. PROJECT COSTS AND FINANCING (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) 1: Information and Communication Infrastructure 7.00 6.33 90.43 Development in Rural Areas 2. Communications Regulatory 1.91 2.72 142.41 Development 3. Public Private Partnerships for 2.46 1.36 55.28 e-Government 4. Project Management .32 .55 171.88 Total Baseline Cost 11.68 10.96 93.84 Physical Contingencies 0.03 0.00 0.00 Price Contingencies 0.14 0.00 0.00 Total Project Costs 11.85 10.96 92.49 Front-end fee PPF 0.00 0.00 - Front-end fee IBRD 0.00 0.00 - Total Financing Required 11.85 10.96 92.49 (b) Financing Appraisal Actual/Latest Type of Co- Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Borrower 1.85 1.80 97.30 IDA Grant 8.00 7.16 89.50 JAPAN: Ministry of Finance - PHRD 2.00 2.00 100.00 Grants - 24 - ANNEX 2. OUTPUTS BY COMPONENT Actual Accomplishments at the Time of ICR and Appraisal Plans Reasons for Deviation Component 1: Information and communications infrastructure development in rural areas Capital subsidy support was provided to telephony services for herder communities of 152 baghs of 76 soums. But annual telephone minutes originating from the herder network are estimated to be about 530,000 annual minutes by end-2012, below 1,200,000 annual Capital subsidy support to minutes projected at appraisal. This shortfall is attributed to fast provide telephony services mobile networks expansion in the hinterlands, resulting in herders A at the bagh level (for herder using mobile phones instead of herder network. The herder network communities) was thereby scaled back, with 192 communities plugged into the network by end-2012 as opposed to initial projection of 350 during appraisal. The herder public access network, combined with the mobile services, reduced the average travel distance required to make a call for herders to 15 km from 39 km. The project provided capital subsidy support to mobile communications services in 90 soums and to Internet services for 34 soum centers. This contributed to a leap in annual telephony minutes originating in soum centers from 1.2 million in 2006 to 56.5 million by end -2012, and an increase in internet users from 300 in 2006 to Capital subsidy support to 12,000 by end-2012. Today, all (330) soums have access to mobile provide telephony or data and modern telephony services while 34 soums have access to B internet services for soum high speed internet. While the overall increase in availability/usage centers of cellular and mobile data services at the soum level played a key role in achievement of targets, the extension of digital telephony services to all soum centers partially funded by the project also helped. This is being stated, as in almost all cases, mobile internet services become available wherever modern digital telephony services are available. Through a consultancy, CRC was provided with transaction support at Technical assistance support each stage of the implementation of Mongolia’s Universal Access for consultant services to strategy to accelerate the development of rural telecommunications design the bidding infrastructure and services in locations that were deemed unlikely to C documents for the OBA attract sufficient private investment. The TA led to existing service schemes and to manage the providers being awarded with least-cost subsidies to provide implementation of these telecommunications access at the bagh level; and voice and internet programs access at the aimag and soum levels. Actual Accomplishments at the Time of ICR and Appraisal Plans Reasons for Deviation Component 2: Communications Regulatory Development - 25 - Actual Accomplishments at the Time of ICR and Appraisal Plans Reasons for Deviation SMMS was implemented in two phases providing CRC a convenient and practical tool to monitor the activities of radio frequency users, detect illegal users, as well as monitor the activities of legal radio Financing a radio spectrum frequency users. This led to a favorable environment for fair A management and monitoring competition and deployment of new technologies and services in system Mongolia. Clear client ownership was demonstrated by the fact that CRC co-financed 25 percent of the cost of this system from their own resources. Technical assistance support to finance the preparation of Through a consultancy, CRC was provided with transaction support at B bidding documents for the each stage of the bidding process and implementation for the SMMS. radio spectrum management and monitoring system (1) Two-stage trainings from highly qualified trainers were offered to CRC staff, thereby strengthening their capacity for radio spectrum management. (2) With the help of consultancy service on Spectrum policy, CRC was able to assign the entire 3G spectrum to 4 mobile Technical assistance to CRC operators, the Spectrum Master Plan was developed and is being in the areas of licensing, regularly updated, and the Spectrum management handbook was spectrum policy, monitoring published. (3) The Telecommunications Network Numbering Plan of C of the implementation of the Mongolia under the consultancy service on numbering was approved. universal access program, This led to introduction of fees for telecommunications network and specialized regulatory numbering starting from January 1, 2008; establishment of number advice utilization contracts; and organization of a tender for the provision of 800 and 900 services in Mongolia. (4) The unified licensing consultancy for converged services in the sector led to an updated regulatory regime. (5) The HR policy and development plan of CRC is being implemented as a result of HRD consultancy. Training for staff in areas such as regulation of competition, public The capacity of CRC was built through foreign trainings, in-country consultative processes, organized trainings and seminars, and study tours. 48 CRC staff have interconnection, price benefitted from foreign trainings and seminars, 1 local HRD training D regulation, resolution of involving 40 CRC staff and UA final workshop was organized disputes, consumer rights involving representatives from all stakeholders of UA/US programs. and complaints, accounting for costs and new trends in regulation. Actual Accomplishments at the Time of ICR and Appraisal Plans Reasons for Deviation Component 3: Public-Private Partnerships for e-Government The draft legislation and other deliverables developed under the Technical assistance for consultancy on enabling environment for PPPs in e-Government have developing the policy, legal been used by the Parliamentary ad-hoc committee on PPPs and the A and regulatory frameworks newly established National Development and Innovation Committee for PPPs in e-Government during the development of the newly approved Concession Law of Mongolia. - 26 - Actual Accomplishments at the Time of ICR and Appraisal Plans Reasons for Deviation Consultancy services for conducting a detailed Consultancy services were financed by the project for conducting the feasibility study to identify PPP feasibility study and providing transaction support for developing PPP project opportunities the RFPs. The consultant delivered the feasibility study, identified e- B for e-Government, rank gov applications for implementation, developed bidding documents order them and develop for two of them and supported the implementing agency during the RfPs to roll-out those bidding process. But as explained in section 2.2,.the bidding process applications with the most was cancelled. potential The e-Government applications in the areas supported by the project were not implemented. This is because the PPP approach was new for the Government and it took some time to finalize the PPP model to apply as well as to develop the bidding documents. Following the elections in mid-2012, the new Government decided to implement Technical assistance for these applications following a different approach (although contracts C developing a project in the for the implementation of both applications had been awarded) and area of e-Government through different sources of funding. Thereafter, the earlier considered extension of the closing date to allow for the implementation of these e-Government applications could not be processed. Consequently, the IDA funds allocated to the implementation of these two e-Government applications ($1 million) were canceled. 22 proposed projects of e-Mongolia program had been studied for Investment support for targets to apply PPP approaches where only two projects were chosen government’s participation which could serve as a single window for government, citizen and in implementing priority e- businesses, and public key infrastructure. During pre-proposal D Government project(s) in meetings, 9 companies expressed interest in supporting the PKI partnership with the private application, but only 3 of them submitted bid proposals. 6 companies sector were interested in Portals project whereas only 2 of them submitted bid proposals. The draft Information Security law and e-Signature law have been Technical assistance for developed under the consultancy on information security, and e- E developing an information Signature law has been approved by the Parliament in 2012. The security framework draft Information security law is in the process of submittal to the Parliament for approval. Training of ICTA staff and Chief Information Officers The capacity of ITPTA in the direction of using PPPs to leverage (CIOs) of government private-sector funds were enhanced by training 34 ITPTA staff ministries, agencies and abroad. These staff in-turn provided short reports on the training aimags/soums to deal with taken and on the training’s benefit for their day to day work and F e-Government issues and to performance. About 360 government officials from at least 50 help them use PPPs to entities have participated in training organized within Mongolia and leverage funds from the partially financed by the project. In addition, 25 ITPTA and 2 PIU private sector for the staff have received English language training financed by the project. provision of e -Government services. - 27 - ANNEX 3. ECONOMIC AND FINANCIAL ANALYSIS The economic and financial analyses were conducted for the rural access component (Component 1) at the time of appraisal. The rural access component provided competitively awarded capital subsidies to operators responsible for installing, marketing, operating, and maintaining ICT systems and services at soum- and bagh-level on a commercial basis. The nature of the rest of the project was not amenable to a quantitative economic and financial analysis. We replicated the analyses using the same methodology and actual post-project data where available. The total financial benefits of each subcomponent include the potential sales for the operators derived from the expanded access to telecommunications in rural areas. The total costs for the project consist of investment costs; operation and maintenance costs; administrative, marketing and sales costs; and interconnection costs (including the cost of satellite space segment if VSAT technology is used). Our financial model, using a discounted cash flow methodology, calculates that the IRR over 10 years (assuming no terminal value) enjoyed by the operator of the herder network (telephony services at the bagh-level) is 22 percent which is somewhat lower than the 28 percent estimated at appraisal. The average revenue per site turned out to be significantly lower than expected due to the faster-than-expected extension of mobile network coverage used by the herders as a substitute for the herder network. However, this was somewhat compensated with the lower-than-expected capital costs while the competitively awarded average subsidy per site was about 6 percent higher than expected. Our discounted cash flow model calculates that the average IRR over 10 years (assuming no terminal value) enjoyed by the operators of the soum level mobile networks is 37 percent which is significantly higher than the 25 percent estimated at appraisal. The average revenue per site turned out to be higher than expected due to both significantly higher number of users and somewhat higher average revenue per user. This was somewhat compensated with the higher-than-expected capital costs due to the operators opting for more conventional GSM and CDMA technologies instead of lower cost (but untested) alternatives. The competitively awarded average subsidy per site was about 10 percent higher than expected. It is estimated that the establishment of the bagh-level herder network and the expansion of the mobile networks have reduced the distance (time and cost) to communication from an average of 25 to 39 km to access the nearest telephone of any kind to 14 to 16 km. The resulting economic benefits include reduced costs, improved and strengthened family ties, improved security and coordination associated with migration and response to harsh weather, increased economic activity, and more efficient governance and social infrastructure activities. In addition, the soum-level mobile services provide economic benefits from improved access to telephony and Internet services demonstrated by strong demand for the new services. The services better integrate the soum with the aimag center and national capital and allowed social infrastructure and administration (e.g., education, health, governance) and market-based services to become more efficient at the soum-level. The services leverage Mongolia’s high literacy rate and its traditional high priority on education by offering improved opportunities for children, as well as adults, to access the benefits of the information society. - 28 - ANNEX 4. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESSES (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Kashmira Daruwalla Senior Procurement Specialist ECSO2 David I Sr Financial Management Specia LCSFM Tenzin Dolma Norbhu Senior ICT Policy Specialist TWICT Andrea Ruiz-Esparza Senior Program Assistant TWICT Peter Silarszky Senior Economist TWICT Supervision/ICR Dulguun Byambatsoo E T Consultant EASFM Kashmira Daruwalla Senior Procurement Specialist ECSO2 LEGLA- Carlos Ricardo Escudero Consultant HIS David I Sr Financial Management Specia LCSFM Xiaoping Li Senior Procurement Specialist AFTPW Haixia Li Sr Financial Management Specia EASFM Tenzin Dolma Norbhu Senior ICT Policy Specialist TWICT Lhagvasuren Ochir Operations Officer EACMF Andrea Ruiz-Esparza Senior Program Assistant TWICT Peter Silarszky Senior Economist TWICT Gurcharan Singh Senior Procurement Specialist TWICT Randeep Sudan Sector Manager TWICT Gerelgua Tserendagva Procurement Specialist EASR2 Tumentsogt Tsevegmid Senior Infrastructure Speciali EASCS Haiyan Wang Senior Finance Officer CTRLN (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY05 67.03 FY06 125.99 FY07 2.12 FY08 0.00 Total: 195.14 Supervision/ICR FY05 0.00 FY06 0.00 FY07 67.13 FY08 71.70 Total: 138.83 - 29 - ANNEX 5. BENEFICIARY SURVEY RESULTS Not applicable. - 30 - ANNEX 6. STAKEHOLDER WORKSHOP REPORT AND RESULTS Not applicable. - 31 - ANNEX 7. SUMMARY OF BORROWER'S ICR AND/OR COMMENTS ON DRAFT ICR The following report summarizes the views of representatives of the ICTPA and CRC on the implementation and completion of the Information and Communications Infrastructure Development project, co-financed by the International Development Association (IDA), the Government of Japan Policy and Human Resource Development (PHRD) facility, with funding from the Government of Mongolia. 1. Project Objective The objective of the Information and Communications Infrastructure Development project was to support the Government of Mongolia in the implementation of the universal service strategy and e-Mongolia national program to significantly increase the coverage and use of relevant ICT services among the rural population through an incentive program designed to encourage the participation of private operators in the rural segment of the ICT market; and increase private sector participation in the delivery of e-Government services, thereby improving public sector utilization of ICT. 2. Project Components The project had three main interrelated components aimed at achieving the project development objective. The first component, and primary focus of the project, entailed private sector led development of the information and communications infrastructure in rural Mongolia, through OBA schemes. Investment support provided for the deployment of infrastructure and public access to ICT services. As the regulatory agenda in Mongolia continued to evolve, the project supported the regulator in establishing a regime that promoted fair competition and market based incentives for the provision of access in rural Mongolia. The third component entailed building ICT policy leadership in Mongolia and included investment support for the implementation of key aspects of the national ICT policy focusing on increased and improved use of ICTs in the public sector. The following components were outlined in the Project Appraisal Document in 2006. 2.1 Information and communications infrastructure development in rural areas Component on Information and communications infrastructure development in rural areas was to include the following impacts: a. Establish and institute a competitive and transparent process for private sector participation in providing infrastructure access and services in rural and economically unviable areas of Mongolia through the implementation of the Government’s universal access strategy in order to accelerate the provision of telephony and Internet services in rural Mongolia. b. Provide the Communications Regulatory Commission (CRC) with transaction support at every stage of the implementation of Mongolia’s Universal Access strategy to accelerate the development of rural telecommunications infrastructure and services in locations that were deemed unlikely to attract sufficient private investment. In particular, under the project, existing service providers were awarded least-cost - 32 - subsidies to provide telecommunications access at the bagh level; and voice and internet access at the aimag and soum levels. This component financed: (i) Technical assistance to Transaction Support for the Implementation of the Universal Access Strategy of Mongolia though the OBA schemes for the implementation of the following UA/US programs; (ii) Capital subsidy support to provide telephony services for herder communities of 152 bags of 76 soums; (iii) Capital subsidy support to provide mobile communications services in 90 soums; Capital subsidy support to provide Internet services for 34 soum centers. 2.2 Communications Regulatory Development Component on Communications Regulatory Development was to assist CRC, the sector regulator to be strengthened through investment support and technical assistances. This was to include the following outputs: - Design, installation and operation of a radio spectrum management and monitoring system; - Technical assistance to CRC in the areas of licensing, spectrum policy, monitoring of the implementation of the universal access program and specialized regulatory advice; - Training for staff in areas such as regulation of competition, public consultative processes, interconnection, price regulation, resolution of disputes, consumer rights and complaints, accounting for costs and new trends in regulation This component financed: (i) The radio spectrum management and monitoring system; (ii) The Automated Spectrum Management System (iii) Technical assistance support to finance the preparation of bidding documents for the radio spectrum management and monitoring system plus the automated spectrum management system; (iv) Technical assistances in unified licensing, spectrum policy, monitoring of the implementation of the universal access program, numbering and HRD of CRC (v) Training for staff in above-mentioned areas. 2.3 Public Private Partnerships for e-Government Public Private Partnerships for e-Government component is to assist ITPTA 6 in creating an enabling environment for use of Public Private Partnerships (PPPs) towards provision of e- 6 During the course of the project, the client agency’s name and acronym changed repeatedly, from ICTA to ICTPA to ITPTA. Here, the current acronym is used. - 33 - Government services and roll out e-Government applications using PPPs. This component was to have the following outputs: - Developed the policy, legal and regulatory frameworks for PPPs in e-Government; - Consultancy services for conducting a detailed feasibility study to identify PPP project opportunities for e-Government, rank order them and develop Requests for Proposals to roll out those applications with most potential; - Identification and implementation of projects in the area of e-Government to provide investment support for government’s participation in implementing priority e-Government project(s) in partnership with the private sector; - Development of an information security framework; - Training of ITPTA staff and Chief Information Officers (CIOs) of government ministries, agencies and aimags/soums to deal with e-Government issues and to help them use PPPs to leverage funds from the private sector for the provision of e-Government services. This component financed technical assistances in the following areas: - Developing the policy, legal and regulatory frameworks for PPPs in e-Government - Conducting a detailed feasibility study to identify PPP project opportunities for e- Government, and develop Requests for Proposals to roll out those applications with most potential; - Developing an information security framework. - Capacity building of ITPTA 2.4 Project Management This component provided support for the operating costs of the Project Implementation Unit (PIU) that managed the project. Specifically, the project financed the services of a Project Director, Procurement Officer, and Financial Officer. The project also financed limited office equipment, furniture, a vehicle, audit, and limited incremental operating costs including office supplies, in-country travel costs and other operational expenses. 3. Benefits and Target population The project’s successful implementation yielded the following benefits to citizens, policy and regulatory institutions and industries: The universal access program • Although only 90 soums benefited directly from IDA financed subsidies, a large number of additional soums benefited indirectly as the UA/US program demonstrated that demand for services in rural areas is significantly higher than originally expected. 100% of the 335 soums now have at least one mobile operator and more than 250 soums have two or three operators present, in many cases also enabling a medium-speed Internet service. - 34 - • The herder public access network, combined with the mobile services, reduced the average travel distance required to make a call for herders to 15 km from 39 km. • 34 prime soum centers have broadband Internet access for public and private users at the same tariffs as in the capital, Ulanbaatar; schools are connected at discounted rates, and in all of these 34 communities people are benefitting from access to public Internet cafés. • With the infrastructure and service in place across the country, many more rural residents are able to communicate more easily, and benefit from educational and commercial access to the Internet. • The significant subsidies offered under both the pilots and ICIDP project effectively seeded and strengthened the USOF’s mandate by offering capital contributions which matched and were ahead of operators’ contributions to the USOF. This demonstrated the benefits of the USOF OBA style competitions to operators and thus increased its acceptance. Moreover, the tenders were prepared in consultation with operators and stakeholders, which increased their acceptance by the industry; • The Project resulted increased accessibility also opened a range of opportunities for the government, private sector firms, civil society organizations and development partners to enhance service delivery by using new technologies. • The views of stakeholders who have observed the progress of the ICIDP and earlier related programs implemented through the World Bank partnership stated that Mongolia’s UA program is internationally a “best case” example of universal service development and funding in challenging environments. Regulatory strengthening • CRC was supplied with installation of SMMS on May 15, 2009. The facility is successfully used by CRC staff. Thanks to the successful implementation, convenient and practical circumstances have been created to monitor the activities of radio frequency users, detecting illegal users, as well as monitor the activities of legal radio frequency users. This contributes to a favorable environment for fair competition and deployment of new technologies and services in Mongolia. • As a result of the project, CRC is now supported by the Automated Spectrum Monitoring System with automated and efficient management of the radio spectrum resource. The system which is composed from administrative and technical software tools is facilitating planning, allocation, registration of radio spectrum and estimating radio wave propagation. 2 stage trainings from highly qualified trainers with firsthand experience offered to CRC staff strengthening their capacity for radio spectrum management. • With the help of consultancy service on Spectrum policy, CRC had assigned the entire 3G spectrum to 4 mobile operators, the Spectrum Master Plan has been developed and is being regularly updated, and the Spectrum management handbook was published. • The developed Telecommunications Network Numbering Plan of Mongolia under the consultancy service on numbering has been approved. According to the recommendations of the consultant, CRC also introduced fees for telecommunications network numbering starting from January 1, 2008; established number utilization contracts; and organized a tender for the provision of 800 and 900 services in Mongolia. • Updated regulatory regime, thanks to the unified licensing consultancy applied in the regulatory activities for converged services in the sector. - 35 - • The HR policy and development plan of CRC is being implemented as a result of HRD consultancy. The capacity of CRC was built through foreign trainings, in-country organized trainings and seminars, and study tours. 48 CRC staff have benefitted from foreign trainings and seminars, 1 local HRD training involving 40 CRC staff and UA final workshop was organized involving representatives from all stakeholders of UA/US programs. PPP utilization in e-Government • The draft legislation and other deliverables developed under the consultancy on enabling environment for PPPs in e-Government have been used by the Parliamentary ad-hoc committee on PPPs and the newly established National Development and Innovation Committee during the development of the newly approved Concession Law of Mongolia. • The draft Information Security law and e-Signature law have been developed under the consultancy on information security, and e-Signature law has been approved by the Parliament in 2012. The draft Information security law is in the process of submittal to the Parliament for approval. • The capacity of ITPTA was enhanced and 34 ITPTA staff who have taken training abroad financed by project funds provided short reports on the training taken and on the training’s benefit for their day to day work and performance. About 360 government officials from at least 50 entities have participated in training organized within Mongolia and partially financed by the project. In addition, 25 ITPTA and 2 PIU staff have received English language training financed by the project. 3. Description of the Main risks to project success • Weak institutional capacity of policy and regulatory entities hampers sector development and project implementation To address this risk, the project provided a comprehensive program of capacity building to strengthen CRC and ITPTA’s institutional capabilities. The consultants assisted both agencies in the development of various policy and regulatory toolkits and draft documents, and to design and prepare bid specifications for the rural access tenders, radio spectrum monitoring system, automated spectrum management system and e-applications to provide Government services through PPP models. In-house capacity through training programs to address policy and regulatory issues and administer the USOF had a very positive impact on CRC for implementing US/UA program in Mongolia though the capacity building effort for ITPTA could not reach the project’s respective objective due to discontinuity of the staff at both managers and executing officers levels. The risk on possible project management was addressed during the project implementation and with efforts and supports of the contracted consultants and World Bank task team members, overall project implementation, rural access related procurement and contractual issues have been properly executed. • The government’s strategy on improving and consolidating ICT leadership, policy and regulation is not sustained The UA strategy and USOF were successfully implemented and maintained under ICIDP as CRC was strongly committed with consolidated, and also maintained its corporate commitment and ensured staff continuity, despite changes in the organization’s senior - 36 - management; there was a good level of skills assimilation and national program ownership within CRC at the critical times, which continued throughout the ICIDP implementation. On the other side, as the implementation of the project passed through 2 elections and changes in the government as well as in the ITPTA have caused challenges to keep sustainability of policy and regulation, thus failed to achieve the project objective. • Limited participation in the OBA tenders This risk was addressed during the implementation of UA/US program and as CRC’s capacity was strengthened. Thanks to the consultancy service by the best qualified consultants on the assignment, CRC adopted open consultation in the industry for the program and the tenders were prepared in consultation with operators and stakeholders, which increased their acceptance and facilitated participation in the program. All operators (4 mobile operators, 1 telecom operator and 2 ISPs) participated in the tenders and were awarded with the contracts for their respective successful bids and effectively contributed to the project success on US/UA program. • Risks of subsidizing commercially viable ventures or otherwise unsustainable businesses. Cost-benefit analysis of the projects were carried out by the consultants based on all of the strategic lessons learned from the current pilots, including actual revenues earned, costs incurred and the competitive behavior observed in the tendering process, and demand studies done during the existing project. For selecting locations for all three programs, priority areas have been advised by CRC taking account of the consultants’ studies for demands which involved the population distributions, geographic area, the number of baghs, rural household income, the potential affordability, revenue capacity of each soum centre and herder community. CRC noticed that the least subsidy design of the project was to avoid of risks subsidizing unsustainable businesses. • Lack of private sector interest in PPPs in e-Government This risk was a key concern during the two consultancy services to enable PPP environment and identification of possible e- government applications through PPP. 22 proposed projects of e-Mongolia program have been studied for targets to apply PPP approaches where only two projects such as Portals system which would serve as a single window for government, citizen and businesses, and Public key infrastructure- Certification authority have been chosen as suitable for the existing situation according to the international best practices as well as the needs of ITPTA, the client,. Bidding of two applications had 2 stages and during pre-proposal meetings, interests of private sectors showed comparatively high for instance 9 companies were interested in PKI/CA only 3 of them submitted bid proposals, 6 companies were interested in Portals project whereas only 2 of them submitted bid proposals. Private sector concerned the legal environment for PPP which was just introduced in e-government. 4. Key Sectoral development issues for the Period The key milestones and developments of the information and communications sector that have been taken place during the project period are as follows: - 37 - • Competition and market development Strong competition in the sector has positively impacted the ICT market development in addition to implementation of US/UA programs under ICIDP and USOF projects. The Mobile telephony penetration in 2012 is increased two times from 2007 reaching 116 though the fixed telephony density is decreased from 19 to 16 in respective years. Tariff of mobile communications services have been decreased 4-7 times compare with the first mobile service launch in 1996. • Infrastructure To date, 178 soums or 51% of the total 347 soums have been connected to the broadband network (161 by ICNC and 17 soums by private), 34% or 120 are expected to be connected through programs and projects, 49 soums have been left with no broadband coverage. The length of broadband dedicated FOC has been reached to 18,343.3 km and 43% is by ICNC, 28% is by Mobicom, 8% is by Railcom, 7% is by Gemnet and 15% is by Skynetwork. If it’s compared to the project start date, the number of soums covered with broadband network is extended by 4-5 times. The backbone network is mostly under the control of the Government although the private operators invested in the last 2-3 years. Separation policy which was adopted since 2005 has been applied only for the state owned operator. Though the separation is done, the network efficiency has not yet improved. • Policy and regulatory framework Over the last 5-10 years the telecommunication sector in Mongolia has been guided by a series of master plans, policies and programs. The project implementation has gone through two governments but main policies of e-Mongolia have been maintained. Most recently approved policies focused on broadband national program, e-Government national program and digital switchover of broadcasting system. USOF management has been shifted from CRC to ITPTA as a result of amendments to the Communications law, though USOF strategies have been kept as they are. There has been approved a new law on e-Signature. The new government is strongly relying on ICT to achieve its commitments and ITPTA has defined 7 priorities for ICT development based on the national ICT Policy document - a 2021 vision which is not yet approved by the parliament. 7 priorities include national satellite launching, delivery of seamless government services, extension of affordable rural broadband internet, digital switchover of radio and television broadcasting, HRD, e-products and postal service renovation through IT advancements. These priorities have a number of activities planned to be achieved until 2016. During the project period, regulations mostly focused on promoting more open competition, introducing new technologies and improving service quality. The licensing regime has been updated to meet converged environment in the sector by renewing license classifications as infrastructure- only for setting up networks and lease the networks, services and content. Terms and conditions of licenses -have been revised according to the respective classifications. As IPTV, 3G, WiMax, WiFi and triple services have been introduced, content development is required thus content regulations have been developed and applied. Non- geographical numbering regulation is applied and tariff was set up independent from distance differences in order to eliminate the digital divide. To develop rural internet, prices for transmission network for internet services have been decreased. Due to convergence of - 38 - services, regulatory activities are also converged and in this regard, broadcasting regulation is dealt by the ICT sector’s regulator. Regulatory activities for broadcasting content, competition amongst broadcasters, development of content, and migration to digital content are starting causing new efforts for the regulator. • USOF The USOF of Mongolia was legally created in 2000 and became operational in 2006. In 2007 USOF became managed by CRC when it started collecting the levy imposed on the ICT licensees (2% of gross profit) and disbursing under the Special Funds Law of Mongolia. USOF took advantage of the practical experience gained working with the World Bank- funded projects inclusive pilots and ICIDP. The USOF management was transferred to ICTPA by the amendment of the Communications Law approved in December 2008 and a Government resolution issued in May 2009. For the period from 2007 to 2012, over 50 projects have been implemented by USOF funding in mobile telecommunications, IT-internet, radio and television, post and demand studies disbursing more than 16 billion tugrug in a total. 6. Project Management and Implementation The project was officially launched in 2006 during a one day workshop with key stakeholders consisted from MOF, the ITPTA and CRC. Components 1 and 2 were implemented by CRC and Components 3 and 4 were managed by ITPTA respectively. The project was initially to close on 31 March 2012. In late 2011, the MoF requested an extension of the closing date for a further 12 months to the 31 March 2013 to complete the ongoing activities related to implementing e-Government applications using Public-Private Partnership approach. Due to delays of the procurement process of the two applications, Public Key Infrastructure and Certification Authority and Portal System and VPN and the World Bank approved the extension of the Closing Date of the Project by one (1) year as of March 31, 2013. Meantime, reallocation between disbursement categories was needed, as several categories have been overdrawn due to reasons such as higher than expected and unexpected costs, thus the saved funds were reallocated to the overdrawn Categories upon approval of the World Bank. Activities of the first two components have been accomplished within the project period, but activities for implementing e-Government applications using Public-Private Partnership approach have been delayed. Though the PPP agreement for Public Key Infrastructure and Certification Authority was signed within the project period, the court dispute was raised by the lost bidder and the implementation has been cancelled with concerns of national security and state secrecy issues. The procurement process for the project on Portal System and VPN was near to complete with expectation to sign the agreement, but, a new management team of ITPTA who took the office after a new reform government in September 2013 considered the project as a beyond-line with its concept to implement e-government policies. As implementation of two application projects have been cancelled by the GoM, the allocated budgets have been cancelled and withdrawn by the WB. Throughout the project the Steering Committee which was chaired by the Chairman of ITPTA met quarterly to discuss the progress of all activities. The PSC was composed from the Chairman of ITPTA, Chief Executive Officer of CRC, two officials from the MOF each at Director level, an official from ITPTA at Director level, and a member of the Board of - 39 - CRC. Due to changes in appointees of line stakeholders, a composition of the PSC has been re-formed 3 times for the project period. The PSC was supported by PIU that was set up at ITPTA on March 28, 2006 and made up from a Project director, a Procurement specialist and a financial specialist who cooperated closely with technical experts from ITPTA and CRC. PIU provided PSC, World Bank, MoF and other respective agencies with a monthly progress report in order to update them regarding ongoing project activities. The Financial Monitoring Report has been provided quarterly. The project implementation and management was regularly supported by World Bank’s in- country supervision missions held twice a year. A mid-term review was carried out between June 9 and 24, 2009 by the WB task team. Separate financial missions by the WB Finance Specialist were carried out on annual basis. The project financial accounts were audited in line with the GoM financial periods (January 1 - December 31) by the selected auditors of the MoF in first 5 years, and last one and a half year financial accounts were audited by the Chief auditor due to restructuring of duties in the MoF. The procurement plan was updated on regular basis during World Bank in-country supervision mission. 5. Project Activities Summary Progress Report 5.1 Overview Information & Communication Infrastructure Development Project Components • 4 components: 19 packages, 2 trainings Committed • USD 10,352,575.00 Completed Status • 21 packages completed - 80% Component 1. Information & Communication Infrastructure Development in Rural Areas Included Packages • 4 packages Completed 100% Subtotal Amount • USD 5,320,535.00 Outputs • 3 Universal Access Programs inclusive 152 bags with VSAT supported telecom services, 90 soums with mobile communications and 34 soum center with high-speed broadband internet services Component 2: Communications Regulatory Development Included Packages • 8 packages plus 1 training package Completed 100% Subtotal Amount • USD 1,982,228.00 Outputs • SMMS installation and operation, TAs in regulatory strengthening and capacity building trainings for CRC. Component 3: Public Private Partnerships (PPPs) for e-Government Included Packages • 6 packages plus 1 training package Completed 70% Subtotal Amount • USD 1,355,911 Outputs • Studies and draft documents on enabling PPP environment for e-Government, demand study and identification of e- applications for Government services through PPP, draft policy papers for Information security and capacity building trainings for ITPTA. - 40 - 5.2 Project budget and disbursement overview /IDA/PHRD/GoM/ Budget Overview 25,000,000 20,000,000 15,000,000 USD 10,000,000 5,000,000 0 -5,000,000 IDA PHRD GOM Total Available 839,331 -497 658,591 1,497,425 Committed 7,160,669 2,000,497 1,191,410 10,352,575 Total 8,000,000 2,000,000 1,850,000 11,850,000 Disbursement overview 10,000,000 5,000,000 0 GOM 2006 2007 2008 2009 2010 2011 PHRD 2012 2013 Total 2006 2007 2008 2009 2010 2011 2012 2013 Total PHRD 86,942 500,228 518,607 258,447 51,825 60,439 126,790 397,219 2,000,496 IDA 79,909 378,384 801,352 3,565,726 1,101,390 878,200 281,923 73,785 7,160,669 GOM 18,148 13,148 497,176 232,695 403,789 13,148 13,148 158 1,191,410 Total 185,000 891,760 1,817,134 4,056,867 1,557,004 951,787 421,861 471,162 - 41 - 5.3 Detailed description of Packages Package 1.1: Transaction support for the competitive awards of least-cost capital subsidies: Support for the implementation of the Universal Access Strategy of Mongolia Consultants • Intelecon Research, Canada Procurement • Quality Cost Based Selection Contract Amount • USD 459,500.00 Outputs • Universal Access Demand study • Detailed Designs of Universal Access Tenders for 3 Programs • Bidding processes of 3 UA programs Package 1.2: Capital subsidy for telephone services at the bagh level Supplier • Incomnet Co.,Ltd, MNG Procurement • International Competitive Bidding Contract Amount • USD 963,648.00 Outputs • 152 bags have: - Public Access Voice Telephony /PAVT/ - Public Access Messaging - Operator and National Emergency Services Package 1.3: Capital subsidy for soum center networks Suppliers • 4 operators: • G-Mobile, Unitel, Mobicom and Skytel Procurement • International Competitive Bidding (ICB) Contract Amount • USD 3,018,717.00 Outputs • 90 soums have accesses to the following services: - Individual Voice Telephony; - Public Access Voice Telephony; - Public Access Messaging; - Operator and Available Emergency Services Package 1.4: Internet Services for soum center Suppliers • 2 operators: • Mobicom corporation and Sky C&C CoLtd, Procurement • International Competitive Bidding Contract Amount • USD 878,670.00 Outputs • 34 soums have accesses to the following services: - Public Internet Access Centre (“Cybercafé) - School Network access to LAN of computer lab - Private demand and Internet Technical Support - Four separate trainings - 42 - Package 2.1: Technical assistance for the acquisition and implementation of a Spectrum Management and Monitoring System Consultants • Mr. Edward Farell (USA) Procurement • Individual Consultant Contract Amount • USD 79,080.00 Outputs • Detailed design of Spectrum Management and Monitoring System/SMMS/and Automated Spectrum Management System /ASMS/ • Bidding document of procuring SMMS and ASMS , bidding process. • Implementation of SMMS installation and commissioning Package 2.2: Spectrum Management and Monitoring System /SMMS/ Consultants • Tadiran Electronic Systems Ltd, Israel Procurement • International Competitive Bidding Contract Amount • USD 949, 269.75 Outputs • Fixed, Mobile and Portable SMMS supply, installation and commissioning Package 2.2b: Automated Spectrum Management System /ASMS/ Consultants • Tadiran Electronic Systems Ltd, Israel Procurement • International Competitive Bidding Contract Amount • USD 260,711.25 Outputs • ASMS supply , installation and commissioning Package 2.4: Technical assistance for Spectrum policy Consultants • Spectrum Strategy Consultants (Hong Kong, SAR, China) Procurement • Quality Cost Based Selection Contract Amount • USD 150,000.00 Outputs • Situation assessment • Radio Spectrum policy • Spectrum Management • Interference and Radio type approval Package 2.5a: Technical assistance - Methodology and Plan for Technical Audit for Universal Access Consultants • Mr. Arnold Ostevik, Canada Procurement • Individual Consultant Contract Amount • USD 50,000.00 Outputs • Methodologies and Plans for Technical Audit for 3 - 43 - Universal Access Programs • Supported the national technical auditing company to audit 3 Universal Access Programs Package 2.5b: Technical assistance - Execution of Technical Audit for Unversal Access Consultants • “GTTO” LLC , Mongolia Procurement • Consultant Qualifications Contract Amount • USD 59,680.00 Outputs • Supported by the international consultant, conducted technical auditing of 3 Universal Access Programs Package 2.7: Technical assistance – Numbering Consultants • Ms. Claire Milne (UK) Procurement • Individual Consultant Contract Amount • USD 38,500.00 Outputs • Numbering for competition in Mongolia Package 2.8: Technical assistance – Human Resource Development of CRC Consultants • Coachchanger Training Center, Mongolia Procurement • Consultant Qualifications Contract Amount • USD 19,987.00 Outputs • Analysis of HR management of CRC • HR policy and planning • Draft human resource development plan • Training of CRC staff Package 3.1: Technical assistance for enabling environment for PPPs in e-Government Consultants • Atos Consulting Ltd, UK Procurement • Quality Cost Based Selection Contract Amount • USD 359,766.00 Outputs - Background study on PPP in E-government - Developed Organizational design; PPP models recommended for e-Government Master Plan projects; PPP policy and manuals; and Draft law on PPP - Training for government official on PPP in e-government - Communication Plan - Next phase and performance indicators. Package 3.2: Technical assistance for identification, design and implementation support for PPPs in e-Government Consultants • Leading Associates, USA Procurement • Quality Cost Based Selection - 44 - Contract Amount • USD 596,329.25 Outputs - Feasibility study for each potential e-government/PPP system - Bidding documents and bidding processes for 2 e- government applications such as PKI/CA and Portals Package 3.5: Technical assistance for information security framework Consultants • ASTEC Global, Ireland Procurement • Quality Cost Based Selection Contract Amount • USD 139,595.50 Outputs • Draft National Strategy on Information Security • National Policy paper on Government Information Security • Draft legislation of Information security • Implementation plan for the recommended strategy • Trainings for government CIOs and Private sector participants Package 3.7: Technical assistance – Human Resource Development of ITPTA Consultants • Enkhbat Amarjargal, Mongolia Procurement • Individual Consultant Contract Amount • USD 10,221.00 Outputs • Analysis of HR management of ITPTA • HR policy and planning • Draft human resource development plan • Training of ITPTA staff 6. Lessons learned The key lessons learnt from this project’s experience are the importance of ensuring continuity of client leadership and strong commitments taken at both- decision making and executive levels. The Universal Access and Service programs have successfully achieved their goals thanks to a 10 year partnership between the Government of Mongolia and World Bank. The program has started in developing Universal Access Strategy and to prepare a Manual of Operating Procedures for the USOF, with support from the World Bank and Public-Private Infrastructure Advisory Facility (PPIAF) in 2004. Subsequently, the Global Partnership on Output-Based Aid (GPOBA) supported the piloting of the Universal Access leading to UA/US programs under ICIDP. All these activities were managed and executed through continuous leaders and executors of financing, consulting and implementing parties where the lessons learnt taken account and successes achieved were advanced in latter programs. - 45 - In contrast, projects for PPP utilization in e-government services suffered due to lack of continuity at both managerial and executive level, and so, unfortunately these failed to reach the initially envisioned objectives. Nevertheless, the identified applications are still high on the agenda of e-government service deliveries. - 46 - ANNEX 8. COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS The IDA credit was cofinanced by the client executed and World Bank administered Japan PHRD grant and Government funds. The Embassy of Japan was regularly informed about project implementation progress but did not take an active part in the project. There was no other co-financing involved. - 47 - ANNEX 9. LIST OF SUPPORTING DOCUMENTS Found in WBDocs, and the Portal - 48 - 85°E 90°E 95°E 100°E 110°E 115°E 120°E 105°E RUSSIAN FEDERATION Lake Baikal To Biysk To 50°N Ulan-Ude To To Chita Chita Hövsgöl Uvs Nuur 50°N Ulaangom Nuur Achit Hatgal Tavan Bogd Uul Nuur (4,374 m ) UVS Tesiyn HÖVSGÖL Sühbaatar Ereentsav Ho Olgii Mörön SELENGE SEL ENG E To vd BA B AYAN N- - Hyargas Sele nge Hulun Hailar Nuur Darhan DAR H A N -UUL D A R HAN -UUL LGI Ö LG I II r OR Erdenet O R HON HON Bayan Uul Nur Ide on On DORNOD Jargalant HA Z A V K HAN Bulgan (Khovd) Har Har Nuur BUL BULGAN G AN KHE NTI I KHENTII Kherlen Us Nuur (Choibalsan) Uliastai Ha ARHANGAI ARH ANG AI ULAANBAATAR ule n A ng Ker Tamsagbulag KHOVD TÖV an ay Erdenebulgan Zuunmod l n Öndörhaan h (Tsetserleg) av 120°E M Dz Tsagaan-Olom t ts Baruun-Urt . a Esonbulag GOVISÜMBER GOVISÜMBER GAI 45°N (Altai) Bayankhongor Choir SÜ KBA ATA R KBAATAR SÜKBA y GO AL GOVI-A T AI L TA I Arvaiheer Ö V Ö RK H A N Mandalgovi Hongor 45°N M DUNDGOV' DUNDG OV ' Sainshand o CHINA u BAYANKHONGO AYANKHONGOR BAYANKHONGOR n t DOR NOGOV' DORNOGOV' a To i Jining 115°E n Dalandzadgad r t s ÖMNÖGOV' e s e D o b i M ONG OL IA CHINA G SELECTED CITIES AND TOWNS MONGOLIA PROVINCE (AIMAG) CAPITALS 40°N To NATIONAL CAPITAL Hami RIVERS 0 100 200 300 Kilometers MAIN ROADS 40°N IBRD 33449R1 RAILROADS JANUARY 2009 0 50 100 150 200 Miles PROVINCE (AIMAG) BOUNDARIES 90°E 95°E 100°E 105°E 110°E INTERNATIONAL BOUNDARIES