Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004945 IMPLEMENTATION COMPLETION AND RESULTS REPORT (5025-BD, 5911-BD) ON A CREDIT IN THE AMOUNT OF SDR 44.2 MILLION (US$70.00 MILLION EQUIVALENT) AND ADDITIONAL FINANCING CREDIT IN THE AMOUNT OF SDR 28 MILLION (US$39 MILLION EQUIVALENT) TO THE PEOPLE'S REPUBLIC OF BANGLADESH FOR THE LEVERAGING ICT FOR GROWTH, EMPLOYMENT AND GOVERNANCE PROJECT December 24, 2019 Digital Development Global Practice South Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2019) Currency Unit = Bangladesh Taka (Tk) Tk 82.99.= US$1 US$ 1 = SDR 0.72 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS AF Additional Financing B2B Business to Business BACCO Bangladesh Association of Call Centers and Outsourcing BCC Bangladesh Computer Council BNDA Bangladesh National Digital Architecture CAS Country Assistance Strategy CBA Cost-Benefits Analysis CEO Chief Executive Officer CIRT Computer Incident Response Team CXO Chief Executive Officer EFA Economic and Financial Analysis EGDI e-Government Development Index EMP Environment Management Plan FM Financial Management FTFL Fast Track Future Leadership (program) FYP Five-Year Plan GIF Government Interoperability Framework GoB Government of Bangladesh IBA Institute of Business Administration ICT Information and Communication Technology ICR Implementation Completion and Results Report ISR Implementation Status and Results Report IUFR Interim Unaudited Financial Report IT Information Technology ITES IT-Enabled Services JET Jobs and Economic Transformation LICT Leveraging ICT Growth, Employment and Governance M&E Monitoring and Evaluation MOPTIT Ministry of Post, Telecommunications and IT NDC National Data Center NDRC National Disaster Recovery Center NEA National Enterprise Architecture NPV Net Present Value NRI Network Readiness Index PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective ROI Return on Investment STEP Systematic Tracking of Exchanges in Procurement TTL Task Team Leader Regional Vice President: Hartwig Schafer Country Director: Mercy Miyang Tembon Senior Global Practice Director: Boutheina Guermazi Practice Manager: Nicole Klingen Task Team Leaders: Siou Chew Kuek, T. M. Asaduzzaman ICR Main Contributors: Audrey Ariss TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .............................................................................................. 6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................... 10 II. OUTCOME .................................................................................................................... 13 A. RELEVANCE OF PDOs .................................................................................................. 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ............................................................................ 14 C. EFFICIENCY ................................................................................................................. 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING ......................................................... 21 E. OTHER OUTCOMES AND IMPACTS .............................................................................. 21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION......................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION .................................................................. 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................................ 25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ........................................ 27 C. BANK PERFORMANCE ............................................................................................... 28 D. RISK TO DEVELOPMENT OUTCOME ........................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 31 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS. .......................................................... 34 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 39 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 41 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 42 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 48 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 49 ANNEX 6. MAP OF BANGLADESH ......................................................................................... 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Click here to enter text. DATA SHEET BASIC INFORMATION Product Information Project ID Project Name BD: Leveraging ICT Growth, Employment and Governance P122201 Project Country Financing Instrument Bangladesh Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency PEOPLE'S REPUBLIC OF BANGLADESH Bangladesh Computer Council Project Development Objective (PDO) Original PDO The PDO is to (i) catalyze the growth of Bangladesh’s IT/ITES industry for employment creation and export diversification; and(ii)establish basic e-Government foundations to support public sector modernization. Page 1 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 70,000,000 63,518,274 54,829,137 IDA-50250 39,000,000 39,000,000 37,017,754 IDA-59110 Total 109,000,000 102,518,274 91,846,891 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 109,000,000 102,518,274 91,846,891 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 20-Sep-2012 24-Jan-2013 30-Aug-2015 31-Dec-2017 30-Jun-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 22-May-2014 6.16 Change in Components and Cost Cancellation of Financing 29-Sep-2016 28.85 Additional Financing Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financial Management Change in Procurement Change in Implementation Schedule 13-Jun-2018 56.31 Change in Results Framework Reallocation between Disbursement Categories Page 2 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 19-Mar-2013 Moderately Satisfactory Moderately Satisfactory 0 02 16-Oct-2013 Moderately Satisfactory Moderately Satisfactory 6.16 03 01-Jun-2014 Moderately Satisfactory Moderately Unsatisfactory 6.16 04 01-Dec-2014 Moderately Satisfactory Moderately Unsatisfactory 9.75 05 17-Jun-2015 Moderately Satisfactory Moderately Satisfactory 13.61 06 23-Dec-2015 Moderately Satisfactory Moderately Satisfactory 22.28 07 30-Jun-2016 Moderately Satisfactory Satisfactory 28.85 08 09-Sep-2016 Moderately Satisfactory Satisfactory 28.85 09 14-Mar-2017 Moderately Satisfactory Satisfactory 36.55 10 06-Nov-2017 Moderately Satisfactory Satisfactory 49.54 11 06-Jun-2018 Moderately Satisfactory Satisfactory 56.31 12 10-Dec-2018 Moderately Satisfactory Satisfactory 68.01 13 25-Jun-2019 Satisfactory Satisfactory 91.85 Page 3 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) SECTORS AND THEMES Sectors Major Sector/Sector (%) Information and Communications Technologies 76 Public Administration - Information and 65 Communications Technologies ICT Services 7 Other Information and Communications Technologies 4 Education 14 Workforce Development and Vocational Education 14 Industry, Trade and Services 10 Trade 10 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 11 Trade 11 Trade Facilitation 11 Private Sector Development 115 Business Enabling Environment 15 Investment and Business Climate 15 Jobs 100 Public Sector Management 44 Public Administration 44 E-Government, incl. e-services 44 Page 4 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Human Development and Gender 30 Education 12 Access to Education 3 Science and Technology 3 Teachers 3 Standards, Curriculum and Textbooks 3 Labor Market Policy and Programs 18 Labor Market Institutions 9 Active Labor Market Programs 9 ADM STAFF Role At Approval At ICR Regional Vice President: Isabel M. Guerrero Hartwig Schafer Country Director: Ellen A. Goldstein Mercy Miyang Tembon Director: Jose Luis Irigoyen Boutheina Guermazi Practice Manager: Randeep Sudan Nicole Klingen Siou Chew Kuek, T. M. Task Team Leader(s): Tenzin Dolma Norbhu Asaduzzaman ICR Contributing Author: Audrey Anne Alexandra Ariss Page 5 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Country Context 1. Bangladesh had a good track record of growth and development at appraisal. The economy grew by nearly 6 percent per year during 2002–2012 and was resilient to the effects of the 2007–2008 global financial crisis as well as frequent natural disasters. The country made progress on many aspects of human development and was on track to meet the Millennium Development Goals, including infant and child mortality and gender equality in education. Nonetheless, development needs remained large and pressing, with around a third of the population still living in poverty.1 2. The Government of Bangladesh (GoB) had developed its Perspective Plan of Bangladesh 2010– 2021, or so-called ‘Vision 2021’, and Sixth Five Year Plan (FYP) FY2011–2015 with ambitious goals for governance, economic growth, and employment. The plan aimed to accelerate the growth to 8 percent per year by 2015 and ensure social inclusion, good governance, and efficient delivery of public services. The GoB recognized the need to leverage information and communication technologies (ICT) to achieve these ambitious targets and underscored capacity, e-services, and outsourcing as priority areas. The GoB undertook significant actions under its Digital Bangladesh program to create a ‘knowledge economy’, which included strategic pillars that covered Digital Government, ICT in Business, Connecting Citizens, and Human Resource Development. The program fundamentally called for leveraging ICT for the country’s next stage of growth and development and implied broad and effective use of technology in education, health, job placement, and poverty reduction. Sectoral Context 3. Bangladesh’s ICT industry had developed considerably over the decade before appraisal. Telecommunications had generated total industry investments of around US$4.5 billion from 2002 to 2012, providing about 8 percent of the Government's total revenue per year. Mobile penetration rates had increased from less than 1 percent in 2002 to 56 percent at the end of 2011, while 98 percent of the country was covered by a wireless network. However, Bangladesh had significant room for improvement; only 3 percent of individuals used the internet in 2010.2 Its IT industry remained relatively nascent as compared to telecommunications, with total industry output estimated at US$300 million per year and IT/ITES3 claiming 39 percent of that value.4 Its e-readiness remained behind global averages especially in terms of the general population’s ICT skills, perception of the country’s suitability for ICT businesses, supporting infrastructure, and technology foundations for the public and private sector.5 1 World Bank, Global Poverty Working Group. 2010. 2 International Telecommunication Union (ITU). 2010. 3 ITES = Information technology-enabled services. 4 World Bank. 2009. Leveraging ICT For Growth and Competitiveness in Bangladesh: IT/ITES Industry Development. 5 Bangladesh ranked 107 out of 139 in the Global Competitiveness Index , 115 out of 138 in the Networked Readiness Index (2011); and 134 out of 183 in the United Nations (UN) e-Government Survey 2010. Page 6 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 4. The GoB recognized the need to be holistic in its approach to leveraging ICT, as seen in its Sixth FYP. This was reflected especially in the Digital Bangladesh strategy, and a number of key policy and legal instruments were approved in support of Digital Bangladesh, including ICT Policy 2009, ICT Act 2009, and Right to Information Act 2009. From an institutional perspective, an ICT Chapter was also included in the Government’s Secretarial Instructions. Public agencies’ systems and services were largely siloed, duplicative, and insufficiently secured and did not leverage electronic services sufficiently to support public sector reform efforts. Lack of shared IT hosting infrastructure and IT governance policies had exacerbated this and presented a major barrier to providing seamless e-services. Hence, there was a critical need for the technology foundations to improve the reliability and security of the Government services and public information. 5. Digital Bangladesh and ICT development comprised one of the six priority reform areas discussed at the 2010 Bangladesh Development Forum. The World Bank’s Country Assistance Strategy (CAS)6 mentions the role of ICT in achieving Bangladesh’s objectives of accelerated and inclusive growth through increased exports, economic diversification, and employment of women and youth. It also highlights the role of ICTs in achieving ‘Strategic Objective 4: Enhance Accountability and Promote Inclusion’ through enhanced transparency and accessibility of public services through IT. The CAS includes provisions for promoting e-governance applications and furthering the Digital Bangladesh agenda, including through interoperability of information systems and standards to secure data. Theory of Change 6. The project was designed to help grow Bangladesh’s IT/ITES industry and establish the foundations of e-government in the country. The Theory of Change was not a requirement of the Project Appraisal Document (PAD). It has been created for this Implementation Completion and Results Report (ICR) and includes the following assumptions: (a) Political commitment to public sector modernization (b) Continued growth of demand for IT/ITES services (c) Timely provision of in-kind counterpart funding Table 1. Theory of Change Component Activities Outputs Objectives Outcomes 1. IT/ITES • Top-up IT training of • Training programs and Catalyzed Employment Industry science graduates for IT course materials growth of creation Development services segment • Industry Statistics Bangladesh’s Export • Foundational skills for Study IT/ITES diversification ITES segment • Global engagements ( industry • Middle management industry meetings, training conferences and • Institutional capacity seminars) building for Bangladesh 6 Report No. 54615-BD Page 7 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Component Activities Outputs Objectives Outcomes Computer Council (BCC) and local industry • Industry promotion 2. e- • Shared infrastructure • National Data Center E-Government Public sector Government for IT hosting and • National Disaster foundations modernization remote conferencing Recovery Center established • Shared IT governance • National Enterprise policies, standards and Architecture structures • Computer Incident • Capacity building on e- Response Team (CIRT) government skills • Digital policies and standards • Training program and course materials on e- government skills Project Development Objectives (PDOs) 7. The PDOs, as described in the Financial Agreement and PAD, were to “(i) catalyze the growth of Bangladesh’s IT and IT Enabled Services industry for employment creation and export diversification; and (ii) establish basic e-Government foundations to support public sector modernization.” Key Expected Outcomes and Outcome Indicators 8. The PDO contains two separate development objectives, each measured by two indicators: Table 2: PDOs and indicators Outcome Indicators PDO 1: Catalyze the growth of Bangladesh's IT and IT Enabled Services industry for employment creation and export diversification i. Increase in IT/ITES Revenue ii. IT/ITES Employment PDO 2: Establish basic e-Government foundations to support public sector modernization iii. Percentage of public sector (Ministries/Divisions and Departments) officials that have undertaken one or more e-Government skills course at Bangladesh Computer Council (BCC) iv. World Economic Forum’s Networked Readiness Index (NRI) Ranking Components Page 8 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 9. The project was originally composed of three components to achieve the PDOs with a financing of US$70 million from the World Bank, including US$2 million for contingencies. Table 3. Original Component Funding IDA (US$ and % Original Component (2012, total US$70 million) of total) 1. IT/ITES Industry Development (a) Top-up skills training. Provision of training to approximately 10,000 non-computer science graduates from universities and colleges in IT services skills, and training and certification of faculty (b) Foundational skills training. Provision of training for approximately 20,000 trainees in language, customer service, cultural sensitization, and PC/keyboarding and provision of grants to eligible companies (c) Middle management training. Support to a local academic institution to provide IT/ITES focused supervisory/middle-management training programs in partnership with overseas institutions US$35 million (d) Institutional capacity building for BCC as an anchor institution and local industry. (50%) Setup of an IT/ITES unit in BCC and assistance in developing an industry development strategy and roadmap and an international advisory body to guide BCC’s efforts; setting up a forum among the various industry associations; and provision of support to local companies in adopting globally accepted certifications for IT/ITES (e) Industry promotion. To develop an industry promotion plan and provide business development assistance in the form of contact databases, outreach programs to Chief Executive Officer (CXO)-level clients, and support high-level industry champions within the Government for external client engagements and development of promotional tools and materials 2. e-Government (a) Shared infrastructure for IT hosting and remote conferencing. To expand and strengthen BCC’s capacity to provide shared IT hosting facilities to other government agencies and set up remote conferencing facilities for face-to-face collaboration between agencies (b) Shared IT governance policies, standards, and structures. (i) To establish enterprise architecture for coordinating investments across government agencies and (ii) to US$29 million establish the GoB’s information security program, set program goals and priorities (41%) that support Government’s mission, and support setup of a national Computer Incident Response Team (CIRT) (c) Capacity building on e-Government skills. (i) To build capacity for public sector IT- related staff on training on the use of the e-Government foundations and (ii) train higher-level staff in IT planning, strategizing, and project and change management skills 3. Project Management (a) Support for the establishment of a Project Coordination Unit (PCU); recruitment, US$4 million coordination and management of suitable technical, financial management (FM), (6%) procurement, monitoring and evaluation (M&E), and communications specialists Page 9 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 10. The project underwent three restructurings and an Additional Financing (AF), as detailed below. (a) #1 - Level 1 restructuring (May 2014)7. The project was moved into problem status in 2014 as it had used only US$527,316 of its $6.1 million in project advance funds after 10 months of implementation in February 2014; with projected spending of only US$1 million by September 2014. The Level 1 restructuring that ensued included the following: • Changes in components. Two activities were cancelled as they no longer needed to be financed under the project: (i) activities related to IT/ITES company certification under Component 1, taken up by then ongoing World Bank BD Private Sector Development project (P120843) and (ii) activities related to setting up of telepresence facility under Component 2, covered by the Info-Sarker project (financed by the GoB and Government of China). • Cancellation of funds. In line with the above cancelled activities, US$2.5 million (for IT/ITES certification activities) and US$2 million (for telepresence facility) of allocated funds were cancelled. Additionally, the unallocated US$2.08 million was cancelled as the contingency funding was no longer considered necessary. (b) #2 - AF [Credit No. 5911-BD] and Level 2 restructuring (September 2016).8 This included (i) US$39 million AF, (ii) changes in the Results Framework, and (iii) extension of project closing date. • Additional Financing. AF in the amount of US$39 million equivalent was requested. The details are as follows: o Under Component 1. US$12 million AF was required to address financing gap in the IT/ITES activities due to SDR/U.S. dollar exchange rate loss of US$8.4 million and the cancelled funds from restructuring #1. The AF financed expansion of the CEO outreach activity for investment promotion and international business development and support for a new training and employment program in the online freelancing sub-industry. o Under Component 2. US$21.5 million AF was required to scale up e-Government activities to meet immediate needs and critical demand by other public agencies for initial e-Government foundations. This included expansion of the national data center (NDC) and cybersecurity activities and mainstreaming of the national enterprise architecture (NEA). o Under Component 3. US$6.5 million AF was required, including counterpart financing of US$1 million, to ensure continuity of the PCU until the new project 7 Report No.: RES14921. 8 Report No.: PAD1990. Page 10 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) closing date, provide additional resources for immediate demands on the data center and NEA, and strengthen the IT/ITES unit in BCC. • Changes in Results Framework. The changes included the addition of two new monitoring indicators: (i) gender monitoring - ‘number of women who were trained and had gained marketable skills’ and (ii) citizen engagement (trainees satisfied with the training provided). The target dates for several indicators were also revised to reflect the new closing date. • Extension of the project closing date. Considering the above changes, including the scaling-up of the e-Government activities, an 18-month extension was provided. The new closing date was set as June 30, 2019. (c) #3: Level 2 Restructuring (June 2018)9 included one principal amendment: • Changes in the Results Framework. Two changes were introduced: o PDO indicator 4 was replaced with the United Nations (UN) EGDI. The World Economic Forum’s NRI had evolved, and the majority of the sub-indicators used to calculate the ranking, such as the quality of education system and quality of mathematics and science education, were deemed unrelated to this project. In light of this, a more e-Government-centric index was adopted to reflect the project component’s achievements. o The target for the intermediate result indicator on ‘Number of Manpower Trained’ was adjusted back to the original target of 30,000. This target had been increased from 30,000 to 40,000 during the previous restructuring, as the AF supported a new online outsourcing activity to train an additional 10,000 young people and women for the online freelancing industry. The PCU had to cancel this activity after 2 procurement attempts due to significant procurement risks Revised PDOs and Outcome Targets 11. The PDOs were not revised. Adjustments to PDO indicator targets are summarized below. Table 4. PDO Indicators and Targets Original Target at PDO Indicator Unit Baseline Target Closure 2013 2016 2019 1. Increase in IT/ITES Revenue US$ n.a. 200 million 200 million 2. IT/ITES Employment Number 12,000 42,000 42,000 9 Report No.: RES31411. Page 11 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Original Target at PDO Indicator Unit Baseline Target Closure 3. Percentage of public sector (Ministries/Divisions and Departments) % 0 25 40 officials that have undertaken 1 or more e-Government skills course at BCC 4. World Economic Forum’s Networked Ranking 150 n.a. 145 Readiness Index (NRI) 12. The outcome target for one of PDO indicator 3 was adjusted in restructuring #2 to reflect the additional time added through the 18-month extension of the closing date. PDO indicator 4 already considered this extra time as it was added during the same restructuring. PDO Indicators 1 and 2 were not adjusted as the extension considered the delayed impact between activities and industry growth outcomes. Revised PDO Indicators 13. As described, PDO indicator 4 was changed from the ‘World Economic Forum’s Networked Readiness Index (NRI)’ to the ‘United Nations E-Government Development Index (EGDI) during restructuring #2. Revised Components 14. The two main components and funding allocations were revised during the project through the restructurings and AF. These are summarized in table 4. The changes removed overlaps with other government activities that came up during implementation, addressed a financing gap due to exchange rate losses and previous cancelations, and scaled up activities due to increased demand. Table 5. Component Changes at Closure - Original and Closure Credit Original Credit Closure Component Changes at Closure (as Compared to Appraisal) (US$, millions (US$, millions and % of total)a and % of total) 1. IT/ITES Industry Development (a) Activities related to IT/ITES company certification were cancelled in restructuring #1. 35 44.5 (b) Activities on CEO outreach program were also strengthened, (51%) (43%) and an online freelancing activity added under the project’s AF. (c) The online freelancing activity was subsequently dropped in restructuring #2 due to significant procurement risks. 2. e-Government (a) Activities related to setting up of telepresence facility were cancelled. 29 48.5 (b) The AF provided scale-up of activities for the NDC, (43%) (47%) cybersecurity center, and NEA to meet increased demand and needs. Page 12 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Original Credit Closure Component Changes at Closure (as Compared to Appraisal) (US$, millions (US$, millions and % of total)a and % of total) 3. Project Management 4 10.5 No significant changes were made to activities under this component. (6%) (10%) Total 68 103.5 a Percentage is calculated excluding the contingency funding of US$2 million, ie over a total of US$68 million. Rationale for Changes and Their Implication on the Original Theory of Change 15. The changes did not have any major implications on the Theory of Change. The revised indicator provided more accuracy in measuring the PDO. The activities added in the AF and additional time contributed to more effective links in the results chains and increased the project’s achievements. 16. In summary, the restructurings and AF aimed to • Compensate for the shortfall due to SDR/U.S. dollar exchange rate variation; • Improve the M&E framework; • Allow sufficient time to meet employment targets and the delayed impacts of activities under Component 1; • Expand the NDC to meet the high demand from public agencies; and • Increase cyber-resilience further. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 17. Relevance of PDOs is rated High. The primary objectives of the project remain a critical and highly relevant component of Government and World Bank development strategies in Bangladesh. 18. The project was fully aligned with the Bangladesh’s current national priorities, especially given the country’s goals for job creation, export diversification, building of a knowledge-based society, and the adoption of the e-Government foundations —as outlined in the Government’s Seventh FYP for FY2016– 2020. The Seventh FYP promotes the development of ICT infrastructure, capabilities, and services to build a Resilient Digital Bangladesh (Chapter 12). It includes goals for rapid expansion of the IT/ITES industry, Page 13 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) strengthening of the country’s ICT enabling environment, and the development of ICT skills in Bangladesh.10 19. The PDOs are highly relevant to the goals the Bangladesh’s ‘Vision 2021’11 which highlights the role of e-governance to improve interagency coordination and increase efficiency within and across all agencies, and the development a knowledge economy. One of the main goals of the vision announced by the Prime Minister as part of this Vision was that of building two million IT professionals by 2021. 20. The project was aligned also with the World Bank’s Jobs and Economic Transformation (JET) agenda12, as well as current Country Partnership Framework FY16–2013 for Bangladesh. Though digital development is not set as an independent objective, the PDOs are directly relevant to Focus Area 1: Growth and Competitiveness and Objective 1.4 Enhanced business environment and trade facilitation, including to increase diversification of exports and improved competitiveness, as well as providing the e- government foundations to support digital public sector reforms such as e-procurement and automated administration of value added tax (VAT). B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Objectives / Outcomes 21. As outlined earlier, the PDO contains two separate development objectives: • PDO 1: Catalyze the growth of Bangladesh's IT and IT Enabled Services industry for employment creation and export diversification • PDO 2: Establish basic e-Government foundations to support public sector modernization 22. The efficacy of the project has been assessed in terms of (a) compliance with the commitments contained in the financing agreements, (b) level of implementation of the activities of the various components, and (c) level of achievement of indicator targets. The split rating was considered; since the objectives and ratings did not change, it was not included in this ICR report. 23. The ICR assesses the achievement of the PDOs as Substantial. The LICT Project achieved or exceeded all its targets at its closing in June 2019. 10 Government of Bangladesh. 2016. Seventh Five Year Plan FY2016–FY2020: Accelerating Growth, Empowering Citizens. Available at: https://erd.portal.gov.bd/sites/default/files/files/erd.portal.gov.bd/page/2f97258a_6f25_479a_8545_a498c65783f2/Developm ent%20Planning%207FYP%20&%20SDG.pdf. 11 Government of Bangladesh. Vision 21. Available at: https://bangladesh.gov.bd/sites/default/files/files/bangladesh.gov.bd/page/6dca6a2a_9857_4656_bce6_139584b7f160/Persp ective-Plan-of-Bangladesh.pdf. 12 World Bank. 2019. IDA 19 Special Theme: Jobs and Economic Transformation. Available at: http://documents.worldbank.org/curated/en/381791564075012778/pdf/Special-Theme-Jobs-and-Economic- Transformation.pdf 13 Report No. 103723-BD. Page 14 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 24. The project laid critical digital foundations for e-government and public sector modernization. The infrastructure built and operationalized under the project are utilized across government agencies and resulted in significant cost savings and increased efficiencies in public sector use of ICT and improved public services. Both this component’s NDC and cybersecurity had to be scaled up significantly and repeatedly under this project to meet increased and/or overwhelming demand. The project supported the establishment of foundational national infrastructure assets and services using a whole-of- government approach, making it effective and efficient. 25. The project led to significant increases in industry employment and revenue, the effects of which are expected to continue. The project’s IT-based services industry development component developed supply- and demand-side outputs on skills training for youth, targeted training for middle management, industry development strategy, and international promotion to multinational IT companies and international business development support local IT firms. 26. This rating is consistent with the rating of the projects final Implementation Status and Results Report (ISR). Ratings prior to this were Moderately Satisfactory as the project team was waiting for the validation of the M&E results, which took place in the last six months, and the completion of final activities under the Project. PDO 1: Catalyze growth of Bangladesh’s IT and IT Enabled Services industry for employment creation and export diversification 27. Achievement of PDO 1 is rated as Substantial. 28. Industry revenue increased at an average annual rate of 34 percent in the domestic market, and an average annual rate of 30 percent in the export market.14 Project-financed trainings, institutional capacity building, and industry promotion directly contributed to catalyzing the growth of the country’s IT/ITES industry and helped increase domestic and export market revenue and employment. The IT/ITES component resulted in a total of 47,000 people employed, representing 112 percent achievement of the target. Local IT/ITES companies benefited from improved business opportunities and increased the industry revenue by US$280 million—140 percent achievement against the US$200 million envisaged.15 Full-time employment in the IT/ITES industry grew at an average annual rate of 35 percent. Today, the industry employs over 125,000 people in software, business process outsourcing, and e-commerce subsectors16. Export diversification is seen through increase on IT/ITES industry revenue, as the global IT/ITES industry is export oriented in terms of providing cross-border outsourced services. While exports rose, there are no data on diversification, and it may be too early to evaluate the impacts of activities on this. 29. Data and information collected through interviews and surveys reveal overwhelmingly positive feedback on the trainings, which were described as ‘transformative’ and ‘life-changing’ by students, middle managers and executives alike. 35 trainees across the various programs participated in a 14 BCC. 2019. IT/ITES Industry Statistics Study. 15 All figures obtained from the projects M&E results. 16 BCC. 2019. IT/ITES Industry Statistics Study. Page 15 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) roundtable held during the ICR mission in July 2019.17 The majority cited the catalytic impact of the trainings, that allowed them to earn higher incomes than expected, provided new career paths they had not imagined, and enabled some to become entrepreneurs and freelancers. A number have themselves become trainers; one has even opened a training center, at the same time as freelancing. Several mentioned the revenue they earn as a result of the trainings allows them now to support their families and continue pursuing their education in parallel. One interviewee, who took the ‘Android App Development’ course now works as a software engineer working on android app development and earns a monthly salary of 25,000 Tk (approximately US$290). Another, who took the same training, works as a freelancer on Fiverr, an online marketplace based in Israel, and earns US$500 a month. Yet another cited said “it is like a dream to earn 35,000 Tk” thanks to the graphic design course financed by the LICT project.18 These training programs, that were a first in Bangladesh, had a clear positive impact on both firms and employment. Table 5. Indicators and Targets Target at Actual at Baseline Target at Appraisal Indicator Unit Closing Closing (January ‘13) (November ‘16) (June ‘19) (June ‘19) Increase in IT/ITES revenue US$ 0 200 million 200 million 280 million IT/ITES employment Number 12,000 42,000 42,000 47,000 Manpower trained Number 0 30,000 30,000 33,930 International consulting Number 0 5 5 5 firms providing statistical data for their ranking indices Women trained with Number 0 n.a. 9,000 10,430 marketable ICT skillsa Trainees satisfied with Number 0 n.a. 50 96 training provided by BCCa a Added during restructuring #2. 30. The PDO indicator targets were not adjusted during restructuring. Rather, the extended closing date reflected the additional time needed to implement activities and the lag time between project activities and their reflection in industry employment and revenue statistics. 31. The main project outputs relevant to this outcome include the following: • Foundational training and top-up IT training of science graduates for IT services. The project helped train over 35,000 people in IT/ITES skills and directly placed more than 11,100 of its beneficiaries into jobs. Through the Fast Track Future Leadership (FTFL) program, over 2,500 were trained in emerging technologies such as artificial intelligence, Internet of Things, augmented and virtual reality, and so on. The trainings were provided in all 64 districts of Bangladesh and approximately a third of the trainees were women. According to BCC, over 17 The roundtable included 3 CXOs, 5 middle-managers, and 27 FTFL, foundation and top-up trainees. Trainees were selected from Dhaka as it was not convenient for others outside the main city to travel to the city for the purpose of this roundtable only. Surveys administered to over 20,000 trainees and individual interviews with CXOs and middle-managers confirmed the feedback obtained during the roundtable. 18 BCC. 2019. Tales of Transformation. Page 16 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 90 percent of trainees were hired after the trainings. A database of trainees was created and shared with prospective employers through a portal: www.bdskills.gov.bd. The project has been successful in creating a pool of expertise to meet the new demand in these technologies in both local and global markets. In addition, six job fairs, six ‘techfests’ and over 70 ICT career camps were organized across Bangladesh to promote employment in the ICT sector. • Middle management and CXO training. The project also provided training to over 600 middle management professionals working in different IT/ITES organizations in Bangladesh through a partnership with Institute of Business Administration (IBA) Dhaka University, IIM Ahmedabad, IIT Delhi, Ernst and Young, and George Washington University. Additionally, over 40 CXOs were trained through a leadership program organized in partnership with the National University of Singapore during the project period. Feedback on these two programs was overwhelmingly positive. Beneficiaries interviewed spoke about the networking and collaboration opportunities that the program provided. In several cases, encounters facilitated by the program led to business partnerships and bridges between entrepreneurs and established businesses. • Business-to-business (B2B) matchmaking and international promotion of Bangladesh’s IT/ITES industry. The project organized seminars, workshops, and B2B with members of the IT/ITES industry of Bangladesh and helped link them with international organizations to gain market access of their products and services. The project conducted 31 sessions with local short-listed IT companies to prepare them for B2B matchmaking. Bangladeshi firms were also trained on global platforms such as Accelerance and Wonderlabs that act as an intermediary between customers and software companies. The project organized international meetings in India, Japan, the United States, Malaysia, and Singapore, and Bangladesh, to raise global awareness of Bangladesh’s IT/ITES industry. These have helped facilitate over 100 meetings between Bangladesh IT company CEOs and over 200 global CXOs (refers to chief-level officers) for business development. The project supported participation of Bangladeshi firms in Japan IT week in 2016, 2017, and 2018 through a CEO outreach program. The Honorable Minister of the ICT Division attended several meetings, and several contracts were signed with Bangladesh companies. Over 15 meetings and two B2B conferences took place in India to encourage investment in Bangladesh. High-level meetings were organized between the ICT Division and global investors in New York and other cities. These activities all served to increase global awareness of Bangladesh’s IT/ITES industry, support Bangladeshi firms reach international markets, and increase exports for the country. • IT/ITES Industry Statistics Study. The project also financed the development of an industry study and inputs used for industry reports. The study was led by the Center for Project Management and Information Systems (PMIS) at the University of Dhaka. Three global consulting and market intelligence firms have published reports on Bangladesh: white Page 17 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) papers were published by the Everest Group19 and HFS20 and a ‘Market Note’ was written by IDC,21 focusing on the potential of Bangladesh’s IT/ITES industry, hardware manufacturing, and emerging technologies respectively. Six other reports on the country’s IT and digital ecosystem are in development. Inclusion of Bangladesh’s in-industry reports helps improve the country’s rankings and raise awareness about the country’s industry. • Policy reforms. Policy reforms pursued through the project included the project’s redrafting of the country’s ICT Policy for 2019, 10 percent cash incentives for IT/ITES exports, and tax waivers for raw materials required for domestic hardware assembly and manufacturing. These helped boost the sector and facilitate the establishment of several smartphone and laptop assembly plants in Bangladesh over the last few years, resulting in both additional employment and foreign direct investment. PDO 2: Establish basic e-Government foundations to support public sector modernization 32. Achievement of PDO 2 is rated as High. 33. The project financed foundational digital government infrastructure that are critical for Bangladesh’s public sector modernization. These digital foundations have become national assets for the country and include the NDC, NDRC, Cybersecurity Center, and its CIRT, NEA, and e-Government Interoperability Framework (e-GIF). All these assets were constructed and operationalized through the project. Capacity-building programs helped shape IT project managers across government and contributed to the strengthening of BCC to manage ongoing IT assets, projects and ongoing trainings for other ministries and agencies. 34. The country’s ranking in the UN’s global EGDI improved from 150th position when the project started in 2013 to 115th position in 2018. This is a significant improvement of 35 places and Bangladesh has the second highest improvement in e-Government ranking among low- and middle-income countries and in the region from 2012. At project closing, 48 percent of public sector officials had also undertaken at least one e-government skills course provided by BCC, representing 120 percent achievement of targets. Table 6. Indicators and Targets at Closing and Appraisal Target at Target at Baseline Closing Indicator Unit Appraisal Closing (January ‘13) (June ‘19) (November ‘16) (June ‘19) Percentage of public sector % 0 25 40 48 (Ministries/Divisions and Departments) officials that have 19Everest Group Research. 2017. Betting on the future: The Bangladesh IT-ITES industry is poised for growth. Available at: http://lict.gov.bd/uploads/file/publication_5a3f7c94c9458.pdf 20 HFS Research. 2019. Bangladesh Emerges as a Distinctive Digital Hub for Emerging Technologies. Available at: http://lict.gov.bd/uploads/file/strategic/strategic_5ca49063b6714.pdf 21 IDC. 2018. Driving a Digital Bangladesh Through High-tech Manufacturing. Available at: http://lict.gov.bd/uploads/file/strategic/strategic_5ba775b3861b7.pdf Page 18 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) undertaken 1 or more e- government skills course at BCC E-Government Development Ranking 150 n.a. 145 115 Index (EGDI) Ministries using two or more e- % 0 50 50 66 government technology foundation established under the project 35. The main project outputs relevant to this outcome include the following: • Shared infrastructure for IT hosting (a) NDC. Under the project, the first NDC, and only Tier 3-certified government data center, was established in the country. It was expanded under the AF and hosts websites, e-mails, and applications. It is used by over 200 government departments as well as the Dhaka City Development Authority and hosts over 2,600 websites. The NDC provides a more cost-efficient and flexible technical infrastructure, reducing operating costs and reliability and security of digital applications and services for the GoB. A national disaster recovery center (NDRC) was also built in Jessore, in southwestern Bangladesh, and is operational. The Government’s major IT systems financed by the World Bank also uses the LICT Project’s national digital foundations. The national ID system (BD Identification System for Enhancing Access to Services (IDEA) Project [P121528]), value-add tax system (Revenue Mobilization Program for Results: VAT Improvement Program [P129770]), and the Government’s e-procurement system (Digitizing Implementation Monitoring and Public Procurement Project [P160758]) all utilize the NDC. (b) NEA. The NEA was implemented through services selected based on their impact and approved through the ICT Division and Prime Minister’s Office. These included e- recruitment for government officials, e-pensions for primary school teachers, e- procurement for grains, and Geo-dash (a geospatial data repository system). Digital signatures also implemented under this project are being used by the Police and Foreign Ministry for visa and passport applications, as well as to authenticate email attachments. The NEA received international awards from the World Summit on Information Society Forum, co-organized by the United Nations Educational, Scientific, and Cultural Organization; United Nations Development Programme, and United Nations Conference on Trade and Development. BCC also provided common applications and solutions such as a document management system, enterprise resource planning system, and other shared applications. (c) Cybersecurity capabilities. The project established the CIRT, which has achieved its full operating capability and collaborates with other CIRTs around the world. A cyber defense training center, cyber sensor network, digital forensic lab, and cyber security risk assessment framework and tool were also deployed under the project. These Page 19 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) represent major initiatives to strengthening the cyber-security ecosystem and critical information infrastructure of Bangladesh. • Shared IT governance policies, standards, and structures. The Bangladesh National Digital Architecture (BNDA) and e-GIF were established with the vision of delivering “connected, integrated, service-oriented, citizen-centric government” resulting in “better, faster and more secure digital services to the people of Bangladesh.”22 They provide a framework, principles, standards, guidelines, tools, and infrastructure for e-Government infrastructure and e-services. The BNDA coordinates investments across government agencies and helps avoid duplication and waste. The e-GIF facilitates cross-agency exchange of information. • Capacity building. The project has trained 48 percent of 13,000 public sector officials working for the GoB.23 The officials were provided useful digital government training in e- governance transformation (analytics, digitalization, automation, and process automation), enterprise architecture, cybersecurity, and IT project management. According to BCC, approximately 75 percent of those trained in project management are now managing large ICT initiatives around the country. The training program also provided training on basic systems, networks, and computer support and management skills for relevant IT staff across government agencies. Justification of Overall Efficacy Rating 36. The project has achieved or exceeded all the PDO-level targets and objectives; efficacy is consequently rated as being Substantial. The project successfully laid down e-Government foundations and built public sector capacity and catalyzed significant growth of Bangladesh’s IT/ITES industry. The e- Government foundations have become critical and national digital assets for the country. Bangladesh has significantly improved its comparative advantage in the global IT/ITES sector through significantly increased industry capacity, exports, and revenue and increased global visibility through the various demand-oriented activities. The industry is expected to continue to have positive development impact for Bangladesh as the Government aims to scale up its digital jobs and industry revenue targets significantly under their Digital Bangladesh program. C. EFFICIENCY Assessment of Efficiency and Rating 37. The efficiency is rated Substantial. See annex 4 for the full analysis. Component 1: IT/ITES industry development 38. The net present value (NPV) of this component is calculated at US$36.88 million and the return on investment (ROI) at 83 percent over 6.5 years. Economic analysis was conducted using a cost-benefit analysis (CBA), as was also done at appraisal and for the AF. The CBA calculates the benefits as the increase in total value added to the national economy due to the increased IT/ITES employment income, itself 22 BCC. 2019. LICTs Project Completion Report. 23 LICTs project M&E results. Page 20 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) calculated as a percentage of the total increase in IT/ITES industry revenue. The cost is calculated as the total investment cost of the IT/ITES component in this project (US$44.5 million). The break-even level of direct employment occurs at 8,391. This occurs in year 6 of the project and reflects the initial slow start of the project and the expected delayed impacts of such industry development activities relative to the e- government component. Component 2: e-Government foundation 39. The economic analysis for Component 2 was also reassessed and updated for this ICR, considering the cross-agency and cross-sectoral benefits of national digital foundations. The discounted present value of the project’s ROI is calculated to be 311 percent. The original approach that used the NDC as the proxy for calculating the component’s returns appears to remain valid. Studies have estimated that the ROI of consolidated data centers can be as high as 81 percent, much higher than the 15 percent assumption used in this analysis. A desktop research conducted did not find availability of ROI examples for other e- Government subcomponents on national-level cybersecurity, enterprise architecture, and digital signature. The input cost of the component represents only 1.9 percent of the five-year budget of Ministry of Post, Telecommunications and IT (MOPTIT). This is significantly below the conservative 15 percent used as the benchmark for savings in such investments, and continues to be indicative of high cost- effectiveness. D. JUSTIFICATION OF OVERALL OUTCOME RATING 40. The overall outcome rating is considered Satisfactory. The project successfully achieved its objectives of catalyzing public sector modernization and industry growth, met or exceeded all targets, demonstrated a significant ROI, and received overwhelmingly positive feedback from the government, private sector, and trainees alike. It remains highly relevant to the country’s national strategy and is aligned with the goals of the World Bank’s current CPF. Table 7. Outcome Ratings Relevance of Objectives Efficacy Efficiency Overall Outcome High Substantial Substantial Satisfactory E. OTHER OUTCOMES AND IMPACTS Gender 41. The project contributed to the Government’s goals of reducing gender gap. The project helped improve employment opportunities for women in the IT sector, provided opportunities for higher-value jobs and remuneration, and equipped women with digital skills that are highly fungible across other sectors. The project contributed significantly to increasing the percentage of women working in the IT/ITES industry up from 5 percent to 20 percent. Out of over 35,000 Bangladeshi youth who benefitted from the IT/ITES program, an estimated 10,450 women benefited by gaining marketable and globally recognized IT/ITES skills. Over 2600 participated in the ‘Development of Women Entrepreneurs’ course, which was held 178 times. According to the records of Bangladesh Association of Call Centers and Outsourcing (BACCO), the number of their member organizations has almost doubled in 2019 from what Page 21 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) it was in 2016, with the percentage of female employees in BACCO member organizations at around 40 percent. Of those that undertook top-up training, 33 percent were women. For middle-level managers, about 10 percent of trainees were women and in CXO training, out of 47 participants, 5 were women (about 10 percent). 42. Additionally, one of the e-services provided—e-pensions for primary school teachers—benefitted women in particular. Of the approximately 650,000 teachers in Bangladesh, approximately 56 percent are women.24 Whilst it was not possible to find national and gender-disaggregated statistics on use of the e- pension system, an a2i programme study describes how this services reduces travel costs and removes challenges women can face by going to government offices to collect their pensions in person. 25 Institutional Strengthening 43. The project has significantly contributed to the strengthening of government institutions, especially BCC and ICT Division of MOPTIT, and helped build their capacity to implement and maintain critical and large-scale foundational infrastructure. The project conducted numerous training programs in the areas of enterprise architecture, e-governance, IT service management, software development, cyber security, IT governance, data governance and protection, project management, and so on. The project contributed to building local training capacity as well: while the percentage of local trainers at the start of the project was 20, it had progressed to 85 percent by the end of the project. 44. The project made a significant contribution in the development of private sector companies and organizations working in the IT/ITES sector in Bangladesh as well by providing training to their middle managers and to the CXO-level company personnel, as described earlier. Mobilizing Private Sector Financing 45. All the training programs under Component 1 were co-designed with industry. The success of the trainings was in part due to the demonstrated willingness of the private sector to participate. Firms subsidized the costs of the middle-management (25 percent) and CXO trainings (50 percent). Poverty Reduction and Shared Prosperity 46. As described in the previous sections, the project contributed to increasing IT/ITES industry revenue, export diversification, and budget efficiency of the GoB. Activities under this project thus helped Bangladesh better compete in the global economy by strengthening markets and national capacity. It especially helped over 35,000 youth from across the country find employment and increase their salaries, support their families, and pursue further education. 47. Industry-related studies have also shown that one direct job can create spillover effect of creating four indirect jobs. These direct and indirect jobs have contributed in enhancing their income levels. 24 BANBEIS Educational Database. 2018. Available at: http://data.banbeis.gov.bd/index.php?option=com_content&view=article&id=2049:table-21-number-of-primary-education- institution-teacher-and-student-2018&catid=265:primary-education-2017&Itemid=283 25 Bangladesh Access to Information (a2i) Programme. Simplification of Primary School Teachers’ Pension Service. Available at: https://a2i.gov.bd/publication/teachers-pension-service/ Page 22 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Though this was not measured for this project, interviews with beneficiaries provide anecdotal evidence of this taking place. For example, some of those trained have themselves become trainers; one trainee has started giving training to about 200 people in graphic design. Another who received training in digital marketing resigned from his job in an IT company and is now running his own company employing 10 people. Although the complete ripple effect of training individuals is hard to measure, there is palpable evidence of its effects. Other Unintended Outcomes and Impacts 48. Based on trainee testimonies during a round table session in July 2019, the CXO training program strengthened management skills and widened the overall business perspectives of private sector leaders. Notably, they mentioned a cultural shift in terms of investment in human capital and skills development, with several CXOs (as well as middle managers) saying that they were more likely now to invest in human resources in their firms/departments. There was also a broader change in perceptions about IT skills being useful and relevant across levels of the business hierarchy and functions; previously most thought this was solely the purview of students in science, technology, engineering, and technology. 49. Several participants consider the program to have been key to creating a culture of networking among mid-level managers and CXOs within the ICT industry. The community created has allowed them to keep abreast of the developments in the industry, to share experiences as well as develop B2B partnerships. The project also facilitated the enhanced collaboration between new and established entrepreneurs. The project has also helped close the gap between skills need and academia. Following the success of the CXO training program, the Institute of Bangladesh Administration introduced new courses such as data analytics into the curriculum of Bachelor’s in Business Administration (BBA) and Master’s in Business Administration (MBA) programs in the country. 50. As part of the project, an online database of all people trained was developed. This was intended for follow up and promotion of future trainings, but has since been transformed into a job portal. BCC intends now to use it for all its training programs. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 51. High-level support. The project benefited from high levels of government championship and support and the GoB’s Digital Bangladesh agenda throughout the project’s implementation period. The project team was able to leverage high-level actors to facilitate implementation and manage internal and external stakeholders. The PCU was highly motivated to successfully implement the project to provide visible benefits for the government champions. 52. Multi-stakeholder consultations. The project’s IT/ITES component had been designed based on a detailed analytical World Bank study conducted for Bangladesh in 2009.26 The study already incorporated a multi-stakeholder approach in its IT/ITES design as extensive consultation meetings and workshops were conducted with the stakeholders during the study period, including key 26World Bank. 2009. “Leveraging ICT for Growth and Competitiveness in Bangladesh: IT/ITES Industry Development.” Working Paper No. 68689. Page 23 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ministries/divisions/departments and industry associations such as the Bangladesh Association of Software and Information Service, Bangladesh Computer Samity (an association of computer items vendor), and the Association of Mobile Telecom Operators of Bangladesh. 53. Lessons learned applied to project design. The project integrated a number of lessons learned from other projects and academic research, which were reflected in the project design. • The project design for the IT/ITES component was built on similar experiences from the World Bank projects in Ghana, Mexico, and Kenya. The activities were designed to leverage significantly on the global partnerships, networks, and expertise. • Similarly, the design of the e-Government component was based on extensive experience in the e-Government projects in several countries including Ghana, Moldova, Sri Lanka, and Vietnam. The e-Government component leveraged the findings and recommendations from the World Bank supported Bangladesh e-Government Strategy of 2009. For this component, extensive engagement was needed to demonstrate the benefits of the approach, including with other governments such as Australia, Singapore, and Estonia. • The project implementation arrangements were streamlined based on past operational lessons. The enabling technology foundations were an integral part of the e-Government strategy and were provided by one single agency with adequate skills and capacity to manage technology. The project recognized that having multiple implementing agencies in an investment project could limit implementation performance in Bangladesh. The implementation arrangements were therefore anchored only in BCC and focus was placed on enhancing the capacity of BCC to support government agencies, in turn helping enable other agencies to roll out their e-services more effectively and efficiently and train IT personnel from the agencies on e-Government services. 54. Risk assessment. During project preparation, key risks were identified and mitigation measures adopted with the overall risk considered as Moderate. Major risk was limited to staff capacity and staff turnover. To address these risks, the PCU was adequately staffed with technical specialists and fiduciary staff who were provided additional training to strengthen capacity. Procurement risks were considered manageable as there were few high-value contracts. Governance risks were addressed through broad based participation in project governance arrangements and transparency in project monitoring. B. KEY FACTORS DURING IMPLEMENTATION 55. Problem project status of the project. The project had to be classified as a problem project after its first year as it had used less than 9 percent of its budget after its first year of implementation. Significant efforts were undertaken the restructure and cancel funds (as described earlier, through a level 1 restructuring), reorient the Government’s project team to focus on implementation instead of pre- implementation consultations and refinements, and speed up procurement and delivery of priority activities. The project was moved out of problem status a year later in 2015 as implementation picked up significantly. It met the disbursement targets set up in the 2014 restructuring; disbursements had increased to $10.3 million with 72 percent commitment of project funds. Page 24 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 56. High level of commitment, transparency, and stability in BCC. The team had very high level of commitment to the project’s success, as evidenced by their significant stepping up of efforts to speed up implementation and bring the project out of problem status within a 12-month period. It also had a very high level of technical competencies, both within the core PCU team and with the consultants hired for the project, as evidenced by the draft quality of numerous procurement documents and specifications. Additionally, activities were designed to be within the control of one implementing institution, BCC, which resulted in smooth implementation of both components. The PCU was able to supervise and engage technically with vendors for implementation issues. The team’s leadership also remained stable during implementation with no change of project director and the deputy project director. The team also maintained the highest level of integrity. It had to manage external pressures on transparent procurement and made the difficult decision to drop the online outsourcing activity when its fiduciary risk proved too risky during its procurement process. 57. Continued consultation with the IT/ITES industry and academia. Continued engagement with firms and universities ensured that trainings remained relevant and useful to students and the private sector alike, and informed activities financed under the AF. Industry associations were also included in steering and implementation committees and could represent the needs of the sector. For example, entrepreneurs requested ‘train-the-trainer’ programs as well as trainings on Internet of Things that were later included. Collaborations with the National University of Singapore, Indian Institute of Technology (Delhi), Indian Institute of Management (Ahmedabad), and Boston Consulting Group brought international expertise to project implementation and ensured the ongoing relevance of trainings to international markets and industry demands. 58. Highly technical and specialized equipment/services. Almost all procurement in this project was done through International Competitive Bidding due to both the highly technical nature of the procurement packages and limited local suppliers and consultants. Some of the procurement packages had to undergo special national security and U.S. government vetting and high-level approvals which could take several months. In other cases, equipment had to be shipped from Europe (for example, precision air conditioning for the NDC). 59. Exchange rate loss. Due to the fluctuations in U.S. dollar/SDR exchange rate (which reduced the purchasing power of the credit by almost US$8 million), an AF was needed to fill the financing gap. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 60. Choice of indicators for M&E of the PDOs was suitable for the measurement of the PDOs. 61. The indicators of number of employees and IT/ITES income are suitable metrics for industry and employment growth. Additional indicators of industry growth could have also included measures of industry financing or investments, as well as tracking the number of firms operating within the industry. The PDO indicator on ‘Increase on IT/ITES industry revenue’ is used as an indicator for export diversification as the global IT/ITES industry is export oriented in terms of providing cross-border Page 25 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) outsourced services. As mentioned in the PAD, IT/ITES industry revenue had been used because data are more readily available compared to export revenue. The use of total industry revenue is also expected to provide a more holistic perspective of the project’s total impact on the IT/ITES industry, as export revenue is unlikely to reflect the project’s positive impact for the informal IT/ITES segment consisting of freelancers. The project included the financing of an industry statistics report under Component 1 to address the challenge of lack of industry data and measure progress made. Though BCC incorporated estimates of market revenue from both domestic and export markets in the study, additional indicators could have been used for measuring export diversification such as calculating export market size and product diversification. It is nonetheless unlikely that the impacts on export diversification would be captured after only a few years. Indeed, indicators for PDO 1 at appraisal did not entirely account for the delayed impacts of activities on industry growth. 62. Indicators for measuring the establishment of e-Government foundations for public sector modernization was appropriate, considering both the provision and use of the core foundations built. 63. The project’s Results Framework was prepared according to the World Bank’s guidelines. BCC’s core primary responsibility is project M&E. The monitoring system was based on key performance indicators and intermediate outcome indicators as included in annex 1 of the PAD. M&E Implementation 64. Project M&E was conducted regularly by the PCU, which included an M&E specialist. Reporting templates that sought to capture the required information from components were developed for monthly, quarterly, and annual reporting and data collection was consistent. M&E implementation was rated Moderately Satisfactory until the final ISR as fully verified data were not available until the last year. The World Bank financial and procurement teams conducted M&E periodically, which helped gauge project performance against expected outcomes. The Project Steering Committee meetings, chaired by the secretary of the ICT Division, were conducted half-yearly to monitor and review the project’s status and financial and other project-related matters. The World Bank’s review reports were received after completion of review missions by the World Bank team. 65. A midterm review was conducted in August–September 2015 to assess implementation progress which provided in-depth assessment of progress toward project outcomes. Various measures including restructuring were recommended to reorient the project to ensure it achieved its objectives. The task team regularly collected data, updated progress against the baseline, and highlighted issues for World Bank management’s attention. In addition, quarterly and annual progress reports and monthly fiscal reports were used to monitor project’s outcome and results indicators. M&E Utilization 66. Data collected through M&E informed shifts in implementation of the project. The data collected from the progress reports on indicators were evaluated and used to inform decision making on activities. For example, based on the data and information collected from the mission reports including midterm review and the Results Framework, the Government and the World Bank restructured the project, provided AF, introduced changes in the outcome indicators, and extended the project closing date. Page 26 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 67. The project team was proactive in using data, surveys, interviews and other analyses to improve the project. For instance, a skill gap analysis of technical skills was conducted every six months with industry, and high-demand top-up trainings, such as Java, were created or scaled to accommodate needs. Detailed feedback data – including over 20,000 survey responses – were collected on trainees as well to ensure trainings continued to meet industry needs and were adapted to the situations of students. For example, feedback on top-up trainings revealed that their initial three-month duration was not convenient to students who needed these to be less intensive. Consequently, trainings were extended to nine months to be aligned with the academic calendar. The urgency of implementing the project’s middle management and CXO-level training also became apparent to the project teams during implementation, as feedback from the lower-level trainees highlighted the supervision and managerial-level challenges they encountered in applying their learning and these challenges had to be addressed to successfully increase the overall capacity of these local firms. Feedback was also used by academic institutions such as Dhaka University’s IBA to revise curricula and management courses. 68. The team also adopted a new indicator to measure the PDO. As indicated, the project team changed one of the indicators for measuring e-governance performance from the NRI published by the World Economic Forum to the EGDI published by the UN. 69. The IT/ITES Industry Study, conducted by Dhaka University and validated by the Bangladesh Bureau of Statistics now provides the most up-to-date data and statistics on the sector. Data collected have been used for industry promotion activities, including providing statistics for industry white papers and reports. 70. BCC and the ICT Division continue to track all the project indicators. Justification of Overall Rating of Quality of M&E 71. The overall quality of M&E is considered Substantial. M&E was relevant and flexible, and adequate adjustments in the indicators and target values were introduced during implementation. The project has also established a methodology for IT/ITES statistics gathering that continues to be used by BCC and its parent ICT Division. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 72. Environmental and social safeguards compliance. The environmental category of the LICT Project was classified as C in accordance with the World Bank’s safeguard policy: Environmental Assessment (OP 4.01). Following the second restructuring, the project installed an electrical substation for the expansion of the NDC. Therefore, an environmental screening/assessment was carried out and an environment management plan (EMP) was prepared accordingly. The PCU took necessary measures for ensuring EMP compliances along with the health and safety measures for the workers during the construction of the substation. 73. Fiduciary compliance. The project complied with all the fiduciary covenants during implementation. Internal control arrangements were put in place and adequate FM, procurement, and disbursement systems were maintained. The project was compliant in submitting audit reports within the Page 27 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) agreed deadline and with unqualified audit opinions on the financial statements. The audit reports concluded that the utilization of funds as presented gave a true and fair view. • FM. The project was compliant in submitting the interim unaudited financial reports (IUFRs) and audit reports throughout the project implementation period, for both the original and AF credits. A computerized accounting software was installed which allowed adequate monitoring of expenditures. A Chart of Accounts was developed following the Government’s code of accounts and project components. Internal and external audits were conducted annually. Other than a few exceptions, due to unreconciled balances between the Designated Account and Client Connection, the IUFRs were generally acceptable to the World Bank. An extensive FM review was conducted in 2017 following a large number of mis-categorizations of expenditures and turnover in FM specialists in the PCU. These were all subsequently resolved. The FM Manual incorporated the necessary internal control guidelines using the country finance system. All the project’s internal controls worked as intended. Key project staff attended training workshops to fully understand the approved manual. Audits were carried out regularly as part of the World Bank’s implementation support activities with minor observations that were addressed adequately by the Government’s project team. • Procurement. In the initial phase of the project, BCC experienced some delays in implementation and procuring complex large ICT packages. The procurement performance was therefore deemed initially moderately satisfactory. However, as the project gained momentum, the technical and procurement personnel of the PCU became well acquainted with the IT procurement system (single and multi-stage bidding processes). The project successfully completed all the targeted procurement packages including the transformational procurements related to the NDC, cybersecurity, and cloud computing. All prerequisites were adequately put in place to help ensure successful procurement and implementation. The World Bank’s post-review missions found areas to be improved in contract management and these were adhered to during project implementation. All procurement contracts were awarded in line with the World Bank’s Procurement Guidelines and Regulations. While there were occasional delays in processing procurement packages, this was due to occasional weak coordination to get all technical inputs on time to start the procurement process and because of the nature of the complex IT system, as well as the occasional additional security clearances needed for highly sensitive packages. There were no instances of mis-procurement. Despite a high procurement risk rating, the overall procurement performance remained Satisfactory throughout the project. C. BANK PERFORMANCE Quality at Entry 74. The project was grounded in the realities of Bangladesh and its challenges in the IT sector. The identification team focused on intervention gaps that existed in the country to meaningfully leverage on ICT experience to achieve targets of similar transforming countries in the region. The project consulted with the key stakeholders in the sector (BCC, ICT Division of MOPTIT, the IT/ITES industry associations and companies, and numerous universities across the country for training). Stakeholders interviewed Page 28 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) appreciated the level of engagement that took place from the preparation phases (through completion), especially the multi-stakeholder engagements and consultations. Activities during preparation helped identify the supply- and demand-side activities that could, together, contribute to achieving project outcomes, as well as the multiple types of trainings for technical employees, middle managers, and CXOs. 75. During project appraisal, the World Bank considered adequacy of project design including all relevant aspects such as technical, financial, economic, institutional, and procurement. Major risk factors and lessons learned from earlier projects were considered and incorporated into the design. Experienced and committed task teams, both in Dhaka as well as at the Headquarters in Washington, were constituted to provide technical support to the project. Quality of Supervision 76. The World Bank team was able to provide strong and responsive implementation support to the project. The World Bank team conducted regular supervision missions, on average every two to three months, in addition to monthly and fortnightly teleconferences to take stock of progress. The team’s close monitoring enabled rapidly responsive adjustments, including project restructuring, AF, and closing date extension. The Aide Memoires and ISRs were candid, detailed, and well targeted to outline important events and formulate a clear picture of implementation progress and issues. An additional co-task team leader was assigned in 2017 as needed to improve the quality of on-the-ground support and supervision. While turnover in procurement specialists caused some project delays, the impacts of this were mitigated with the advent of the Systematic Tracking of Exchanges in Procurement (STEP) platform. 77. The progress and guidance ware recorded in 13 ISRs and 12 Aide Memoires. The project had been classified as problem status after a year of implementation. The World Bank team had been deeply involved in helping the project to turn around its performance after this downgrade in performance. The new TTL and the task team initiated immediate strategic approach and high involvement actions to cancel funds and simplify the project and helped the Government’s project team to reorient the project toward actual implementation instead of pre-implementation refinements and conducted close implementation support to speed up procurement and delivery of priority activities. The team also instituted various management mechanisms and close project support to address critical implementation bottlenecks, such as the disbursement spreadsheet to target high disbursement items and track progress, procurement response tracking spreadsheet to manage World Bank clearances for timely responses, and master schedule to track and monitor every activity and its procurement and implementation milestones. Justification of Overall Rating of Bank Performance 78. The overall World Bank Performance rating is considered to be Satisfactory. The strong technical and fiduciary competence of the World Bank team and its strategic approach and high involvement in project management were critical in improving the project’s overall implementation during its initial challenging periods and its subsequent achievements. D. RISK TO DEVELOPMENT OUTCOME 79. The risk to development outcome is considered Low. The LICT Project’s e -government foundations have been recognized as national digital assets by the GoB and their continued operation is Page 29 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) critical for the IT systems of their public sector. In this context, the GoB had requested for a LICT II Project to bring their e-government achievements to a transformative, whole-of-government digital approach and further develop their IT industry development for a digital economy and the Fourth Industrial Revolution. The LICT II Project (P161086) is targeted for Board in March 2020. The Government has also initiated two bridging projects on e-government and IT/ITES industry development to maintain the infrastructure created under this project and sustain the industry growth while it also builds its internal staff capacity and pending the start of the proposed LICT II Project to be supported by the World Bank (P161086). The IT/ITES component was designed to be catalytic by providing a sufficient resource pool of supply and institutional capacity to reach a sustainable of industry competitiveness. 80. Specific risks are detailed here: • Technical. Technical risk is considered Low. BCC has acquired the necessary skills and competence to carry out highly complex technologies including NDC, NEA, certificate authority, forensic laboratory, cyber range, risk assessment framework, e-GIF, and computer training laboratory. The BNDA was developed by tailoring the Open Group Architecture Framework which is a technologically neutral framework for developing enterprise architecture, thereby avoiding vendor lock-in. • Sectoral. This risk is considered to be Low. The project’s IT-based services component has created significant demand by local beneficiaries for trainings and all trainings were co- designed with industry. A number of courses will continue beyond the end of the project. Dhaka University’s IBA, for instance, has taken the mid-level managers training on as their regular course named Advanced Certificate for Management Professionals (ACMP). Nonetheless, due to the changing nature of technology, there is the need for the employers to follow up on IT/ITES skills training. Curricula and content for trainings have been handed over to BCC, and a database of all trainees was created and shared with potential employers. It will be important to retain the existing technical people so that their experience and technical skills would continue to benefit BCC to sustain project achievements. • Human resources. Most of the 52 consultants who worked under the project have been transferred to the two bridging projects mentioned earlier and the proposed LICT II Project will also support the hiring of additional consultants. BCC has also created 168 permanent staff positions that will be eventually trained to take over the maintenance and sustainability of the project’s deliverables. Trainings were all transferred to BCC’s seven regional training centers, and BCC trainers were trained to continue these. This risk is considered to be Low. • Financial. This risk is also assessed as Low. Maintenance and operational cost of assets of BCC have been mainstreamed by including as a line item in the government budget. Under the project, business plans for the shared data center, cloud computing services, and the CIRT were established. The high demand for these shared services by public agencies has been demonstrated, for example, by the full booking of the data centers before completion and the scaling up of basic infrastructure through the AF. The proposed LICT II Project should also ensure the continued operation and mainstreaming of outputs delivered under this first project. Page 30 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) • Political and institutional. While there is strong championship and support from the Government for Digital Bangladesh, there is a risk that the next government may not give priority to the sector and that those with high technical skills will leave. However, the risk is considered Low given the strategic importance of the sector and active efforts of the project’s leadership to transfer knowledge and skills. The recognition awarded for NEA applications have also demonstrated the utility of the assets built under the project as well as visibility within the Government, such that support for its maintenance will continue. V. LESSONS AND RECOMMENDATIONS 81. Stability and commitment of PCU. The team composition of the PCU remained largely unchanged during the entire implementation period. This brought significant stability and enabled the team to build up their capacity gradually, providing sufficient time for the component leads and team members to work increasingly seamlessly with each other. The PCU’s technical competency and steady commitment to the project’s success was also instrumental in achieving the project’s outcomes; as described earlier. 82. Quick project turnaround in 12-months' period. Immediate actions to restructure and cancel project funds help to streamline the project’s design by removing duplicative activities with other GoB projects and sent the necessary signal to government stakeholders on the seriousness of the situation. Significant efforts also had to be made to reorient the PCU to focus on results, disbursements, implementation priorities and deliverables; instead of extensive pre-implementation consultations and technical refinements. Rapid and coordinated implementation actions and use of simple but useful project management tools developed by the PCU and Bank task team can help improve prioritization and monitoring of implementation of projects. It was these instrumental actions that significantly improved the project's performance and enabled it to be moved out of problem project classification within a year. 83. Simplicity and flexibility in project design. The project benefitted from the fact that the design was not highly complicated from the outset and the project team did not significantly alter the scope or level of ambition in light of the success that was progressively achieved. Flexibility in the design allowed the project to be more adaptive to sector needs, especially in the design and scaling of the e-government component’s national-level infrastructure and services. The project showed how activities could start on a small scale and scale rapidly, with additional financing in areas that are successful. 84. Managing large, technical, and high-risk procurement activities. The success of the project was achieved through close cooperation of technical and fiduciary teams in the World Bank, along with the PCU. High technical expertise within the PCU and the World Bank teams enabled rapid review and sophisticated technical discussions and resulting procurement specifications. These enabled the project to successfully implement numerous large ICT activities that are typically considered high-risk activities as they have been shown to be challenging to implement in the country’s context. Deep involvement of the World Bank team in providing detailed and substantive technical inputs and close procurement support for bidding documents and processes proved to be an effective in managing high-risk procurement packages. In terms of efficiency of procurement processes, more outreach and the formation of qualification criteria could nonetheless boost competition further, especially within international markets. 85. Use of international expertise. The project team reached out to some of the leading international experts from countries that lead the e-Government agenda. Recommendations received helped the Page 31 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) technical design and provided overall direction. During implementation, engagement with global institutions, participation in international forums, and industry dialogue across countries and regions help ensure sustained relevance of Bangladesh’s IT/ITES sector and the project’s activities. 86. High-level leadership for shared infrastructure adoption. Endorsement of a digital transformation of the country at the highest levels of government, including a highly popular Digital Bangladesh agenda, formation of digital task force, and the comprehensive infusion of ICT into the Government’s country development plans helped provide the necessary political support for implementation. Basic shared digital infrastructure and services across government are typically in high demand, and an integrated whole-of-government approach will help further improve ICT usage and enable digital transformation across agencies. 87. IT/ITES skills development across functions and organizational hierarchies. The sector benefitted from capacity being built at all levels: trainings from entry level to CXOs to stimulate industry development. Skills development activities for industry development projects should not focus only on the lower-level technical staff but include work culture training and capacity building for higher-level supervisory and management personnel. This cross-functional approach is useful and relevant to the World Bank’s JET agenda. Digital business transformation training (CXO training) was especially successful with local users as it helped develop the digital leadership mindsets and networks to the local and international industry. 88. Impact on women’s employment. The project successfully trained over 10,000 women and helped double female employment in BACCO member organizations. A number of activities helped ensure female participation: students enrolled in the project were selected from various science/engineering subjects where the selection process maintained a minimum gender ratio of 30 percent to encourage women's participation and targeted communications were carried out by identifying female spokespersons, targeting media channels, encouraging women master trainers, and reaching out to women-oriented educational institutions. 89. Focus on demand-side industry activities for industry development. The project’s most effective IT/ITES industry development supply-side activities are complemented by significant demand-side activities, such as the top-up training for 10,000 youth that had a minimum 60 percent employment target for the appointed consulting firm, subsidized middle-management trainings, and FTFL program that incentivized local companies to hire and retain new staff by providing customized training according to their needs. It is relatively straightforward to provide trainings to beneficiaries but significantly more challenging to provide them sustainable jobs and income as there are much broader external factors for projects to consider and support. For instance, the industry suffered from negative international perceptions of outsourcing and significantly increased automation that developed during the project’s implementation period, which affected demand of certain IT/ITES services in Bangladesh and resulted in the need for the project to scale up its demand-side activities to meet the project’s employment and industry revenue targets. 90. Industry growth measurement. Relatedly, IT/ITES industry growth depends on a number of national and international macroeconomic factors. PDOs and indicators should account for delayed impacts of industry activities on growth, employment, and exports, as well as potential sectoral Page 32 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) disruptions. Since limited data were available, the project developed its own industry statistics study, relying on academic institutions to help ensure the integrity of the data. Page 33 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS. dfsdf A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Establish basic e-Government foundations to support public sector modernization Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Percentage of public sector Percentage 0.00 25.00 40.00 (Ministries/Divisions and Departments) officials 24-Jan-2013 23-Jan-2018 30-Jun-2019 undertaken 1 or more e- Government skills course at BCC Comments (achievements against targets): This target was exceeded at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target E-Government Development Number 150.00 145.00 Index (EGDI) 02-Jan-2012 30-Jun-2019 Comments (achievements against targets): This target, added during Restructuring #2 was exceeded at completion. Objective/Outcome: Catalyze growth of Bangladesh’s IT/ITES industry for employment creation and export diversification Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Increase in IT/ITES Revenue Amount(USD) 0.00 200000000.00 200000000.00 Page 34 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 24-Jan-2013 23-Jan-2018 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target IT/ITES Employment (number Number 12000.00 42000.00 42000.00 of people) 24-Jan-2013 23-Jan-2018 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. A.2 Intermediate Results Indicators Component: e-Government Technology Foundations Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Percentage of ministries Percentage 0.00 50.00 50.00 using 2 or more e- Government technology 24-Jan-2013 23-Jan-2018 30-Jun-2019 foundations established under the project Comments (achievements against targets): This target was exceeded at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Page 35 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Percentage savings in Percentage 0.00 15.00 20.00 government IT spending 24-Jan-2018 23-Jan-2018 30-Jun-2019 Comments (achievements against targets): This target was reached at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Number of e-Government Number 0.00 5.00 6.00 skill courses available in BCC 24-Jan-2013 23-Jan-2018 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. Component: IT/ITES Industry Development Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Number of Manpower Number 0.00 30000.00 30000.00 Trained under the Project (number of people) 24-Jan-2013 23-Jan-2018 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Number of international Number 0.00 5.00 5.00 Page 36 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) consulting firms provided 24-Jan-2013 23-Jan-2018 30-Jun-2019 statistical data for their ranking indices Comments (achievements against targets): This target was reached at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Number of women who have Number 0.00 9000.00 been trained and gained marketable ICT skills 30-Nov-2016 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. Formally Revised Indicator Name Unit of Measure Baseline Original Target Target Trainees satisfied with the Percentage 0.00 50.00 training provided by BCC 30-Nov-2016 30-Jun-2019 Comments (achievements against targets): This target was exceeded at completion. Page 37 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) B. KEY OUTPUTS BY COMPONENT PDO 1: Outcome Indicators 1. Increase in IT/ITES Revenue 2. IT/ITES Employment 1. Number of manpower trained under the project 2. Number of international consulting firms provided statistical data for their ranking Intermediate Results indices Indicators 3. Number of women who have been trained and gained marketable ICT skills 4. Trainees satisfied with the training provided by BCC Component 1: IT/ITES Industry Development 1. Top-up IT Training Program 2. Foundation Skills Training Program 3. Fast Track Future Leaders (FTFL)/Hire and Train Program 4. Middle management training services for IT services Key Outputs by 5. CXO training program Component 6. Strategic CEO Outreach Services on IT/ITES Business Development 7. Capacity-building for local industry 8. Capacity-building for BCC 9. Global engagements (B2B meetings, conferences, seminars) 10. ICT Policy PDO 2: 1. Percentage of public sector officials taken one or more e-Government skills course at Outcome Indicators BCC 2. E-Government Development Index (EGDI) 1. Percentage of ministries using two or more e-government technology foundations Intermediate Results 2. Percentage savings in E-Government IT spending Indicators 3. Number of E-Government skill courses available in BCC Component 2: e-Governance 1. Trainings on e-Government skills (IT project management, cybersecurity, digital transformation) Key Outputs by 2. NDC and NDRC Component 3. CIRT and cybersecurity sensors 4. NEA and e-Government Interoperability Framework (e-GIF) 5. Policies for CIRT and NDC Page 38 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Siou Chew Kuek, T. M. Asaduzzaman Task Team Leader(s) A.N.M. Mustafizur Rahman Procurement Specialist(s) Mohammed Atikuzzaman Financial Management Specialist Andrea Ruiz-Esparza Team Member Seda Pahlavooni Team Member Tapas Paul Team Member Sabah Moyeen Social Specialist Jorge Luis Alva-Luperdi Team Member Syed Khaled Ahsan Team Member Iqbal Ahmed Environmental Specialist Mohammad Reaz Uddin Chowdhury Team Member Shegufta Shahriar Team Member Nahid Sharmin Team Member Audrey Anne Alexandra Ariss ICR Author B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 29.892 249,858.58 FY12 33.745 163,601.76 FY13 12.400 31,668.75 Page 39 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Total 76.04 445,129.09 Supervision/ICR FY13 14.382 127,814.16 FY14 31.715 111,925.27 FY15 38.525 176,402.61 FY16 38.197 168,898.22 FY17 19.450 130,393.44 FY18 20.525 154,826.25 FY19 20.767 153,400.00 FY20 5.198 59,524.00 Total 188.76 1,083,183.95 Page 40 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Components (US$, millions) Closing (US$, millions) Approval (%) IT/ITES Industry Development 35.00 44.50 127 e-Government 29.00 48.50 167 Project Management (including counterpart financing of US$1 4.00 10.50 263 million) Total 68.00 103.50 152 Page 41 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 4. EFFICIENCY ANALYSIS EFA for Component 1 on IT/ITES Skills 1. An economic and financial analysis (EFA) was conducted at appraisal as well as for the AF and is based on a cost-benefits approach that has been replicated here. The CBA calculates the benefits as the increase in total value added to the local economy due to the increased IT/ITES employment income, which is calculated as a percentage of the total increase in IT/ITES industry revenue.27 The cost is based on the total investment cost of the IT/ITES component in this project (US$44.5 million). The approach is conservative in both its estimate of industry revenue and broader economic impact on employment. It does not consider indirect employment; even though studies have shown that each direct IT/ITES job is associated with up to 4 indirect jobs.28 The analysis takes into account the cancellation of funds during restructuring #1, the AF, and extension of closing date. The analysis also provides a CBA based on the breakeven scenario to estimate the minimum level of employment needed for the project to be cost- effective. 2. The assumptions used for the inputs are as follows and differ from those used at appraisal: • The industry growth rate is based on the Industry Statistics 2019 Report published by BCC. At appraisal, the rate was based on a normal curve distribution. • Estimated employment growth per year is derived from total IT/ITES employment data calculated by BCC as part of the Results Framework for this project. Previously, the calculation of total value added was based on a 20.19 percent proportion of the total increase in industry revenue, itself based on a 2011 study which estimated the value added by India’s IT/TES industry to its economy.29 • The CBA uses a cash discount rate of 10.8 percent per year as an average for the NPV calculations, and this is based on the five-year average of nominal lending rates for Bangladesh from 2014 to 2018.30 The rate used at appraisal was 14.8 percent, based also on the average of the five years before project approval.31 This rate is conservative since the project is financed by lower-cost IDA funds. 3. Table 4.1 provides detailed breakdown of the CBA based on these inputs and assumptions. Based on the project cost of US$44.5 million for the IT/ITES component, the NPV of this component is US$36.88 million and the ROI at 83 percent over 6.5 years. The decrease of cash discount rate had a significant effect on NPV, which would be 38 percent if the rate had not decreased. 27 IT/ITES industry revenue is used because data are more readily available compared to export revenue. The use of total industry revenue is also expected to provide a more holistic perspective of the project’s total impact on IT/ITES, as export revenue is unlikely to reflect the project’s positive impact for the informal IT/ITES segment consisting of freelancers. 28 Gokarn, Subir. 2007. The Rising Tide - Employment and Output Linkages of IT-ITES. NASSCOM&CRISIL Ltd. http://www.nasscom.in/upload/51269/NASSCOMCRISIL.pdf. 29 Joshi, Seema. 2011. Can IT and ITES Be an Engine of Growth for India: An Empirical Analysis. 30 This differs from the assumption at appraisal, where an average nominal rate of 14.8 percent was used for calculations, which was based on the average rates for Bangladesh from 2010 to 2014. 31 This is a conservative figure as the project was financed by low-cost IDA funds. Page 42 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Table 4.1. CBA Calendar Year Unit 2013 2014 2015 2016 2017 2018 2019 (1H) Totals Inputs and Assumptions Industry Growth Rate (Yearly) % 1 7 18 13 37 37 20 133 Industry Growth Rate % 1 8 26 39 76 113 133 n.a. (Cumulative) Disbursement Cycle % 1 4 5 15 20 35 20 100 Total IT/ITES Employment Number 12,000 17,000 18,500 20,000 29,000 37,000 47,000 n.a. Growth in IT/ITES Number Employment — 5,000 1,500 1,500 9,000 8,000 10,000 — Cumulative IT/ITES Number employment growth — 5,000 6,500 8,000 17,000 25,000 35,000 — Cumulative Increase in IT/ITES US$ Industry Revenue — 35,000,000 45,500,000 56,000,000 119,000,000 175,000,000 245,000,000 n.a. Benefits Contribution to Total Value US$ Add by Direct Employees — 7,066,500 9,186,450 11,306,400 24,026,100 35,332,500 49,465,500 136,383,450 Cost Estimated Project US$ 445,000 1,780,000 2,225,000 6,675,000 8,900,000 15,575,000 8,900,000 44,500,000 Disbursements for IT/ITES Component Net Benefits/Cost Gross US$ (445,000) 5,286,500 6,961,450 4,631,400 15,126,100 19,757,500 40,565,500 91,883,450 Cumulative Discount Rate % 10.8 21.7 32.5 43.4 54.2 65.0 70.46 n.a. Discounted Cash Flows and US$ NPV (Total) (396,762) 4,140,387 4,697,586 2,623,225 6,927,754 6,907,222 11,983,049 36,882,461 Rate of Return % 83 Page 43 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) 4. Using the same inputs and assumptions, table 4.2 calculates that the breakeven level of direct employment occurs at 8,391. This implies that the component breaks even at an estimated 25 percent of employment (based on total direct employment equalling the total number of trainees). This occurs in year 6 of the project, in line with what was anticipated at the time of the AF. As was noted in the project paper for the AF, this reflects the initial slow start of the project and the expected delayed impacts of such industry development activities relative to the e-government components. Page 44 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Table 4.2. Rate of Return Calculation Calendar Year Unit 2013 2014 2015 2016 2017 2018 2019 (1H) Totals Inputs and Assumptions Cumulative IT/ITES number 84 671 2,182 3,272 6,377 9,482 11,160 n.a. employment growth Cumulative Increase in US$ 587,370 4,698,960 15,271,620 22,907,430 44,640,120 66,372,810 78,120,210 n.a. IT/ITES Industry Revenue Benefits Contribution to Total Value US$ 118,590 948,720 3,083,340 4,625,010 9,012,840 13,400,670 15,772,470 46,961,641 Add by Direct Employees Cost Estimated Project US$ 445,000 1,780,000 2,225,000 6,675,000 8,900,000 15,575,000 8,900,000 44,500,000 Disbursements for IT/ITES Component Net Benefits/Cost Gross US$ (326,410) (831,280) 858,340 (2,049,990) 112,840 (2,174,330) 6,872,470 2,461,641 Cumulative Discount Rate % 10.8 21.7 32.5 43.4 54.2 65.0 70.46 n.a. (%) Discounted Cash Flows US$ (291,027) (651,058) 579,208 (1,161,114) 51,681 (760,146) 2,030,128 (202,329) and NPV (Total) Rate of Return % 0 (At breakeven level) Page 45 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) EFA for Component 2 on E-Government 5. The EFA for the e-Government component was also reassessed and updated for this ICR while taking into account the cross-agency and cross-sectoral benefits of these national digital foundations. The original approach that uses the NDC as the proxy for calculating the component’s returns appears to remain valid. A quick desktop research conducted did not find availability of ROI examples for the other e-Government subcomponents on national-level cybersecurity, enterprise architecture, and digital signature. The simplified approach for the analysis also appears to remain valid. There are various methodologies that have been proposed for analysis of returns, but they require extensive data on the direct and indirect effects to be useable. 6. Given these issues the project’s original approach is updated to analyze the likely returns for this component. It uses the same simplified approach and highly conservative assumptions and data to provide an overall returns perspective. A simplified ROI analysis has also been included in this ICR’s analysis to provide an additional estimate and perspective of this component’s returns. 7. For these purposes, the annual ICT budget of the Government has been updated from the GoB’s latest Annual Development Budget for FY2019 to 2020.32 This remains a highly conservative estimate as it includes only the budget for MOPTIT and excludes the ICT budget of all other GoB agencies. 8. This analysis also revalidated the original approach’s reference to 15 percent savings for use of consolidated datacenters in the public and private sectors. The desktop research conducted suggests that consolidated datacenters still have significant ROI. A Forrester research study commissioned by Dell estimated the average ROI for 5 customers to be 81 percent,33 which is still far higher than the 15 percent used for the original and this analysis. 9. The analysis also updates the interest rate used to calculate the discounted returns on investment for this component. It uses the commercial lending rate for Bangladesh, which is at an average of 10.84 percent per year over the past 5 years.34 This is also a highly prudent estimate compared to the concessional interest rate of this IDA-funded project. 10. Table 4.3 summarizes the results of the revalidated and updated analysis. The input cost of the component is only 1.9 percent of the five-year budget of MOPTIT (consisting of both the ICT Division and Post and Telecom Division). This is significantly below the conservative 15 percent savings used for comparison and continues to be indicative of this component’s high cost-effectiveness. Table 4.3. Costs Summary Time Period FY2019 to 2020 US$, millions US$, millions In Tk, crores In Tk per year (5 Years) ICT Division Budget 1,647 16,470,000,000 200,853,659 1,004 32 Ministry of Finance, Government of Bangladesh. 2019. Budget in Brief (by Ministries and Divisions). Accessed August 9, 2019. 33 Forrester. 2019. The Total Economic Impact™ Of Dell EMC Data Center Modernization and Migration Services. Available at: https://www.emc.com/collateral/analyst-reports/dellemc-economic-impact-datacenter-modernization-migration.pdf. Accessed August 9, 2019. 34 Economist Intelligence Unit. 2019. Country Report: Bangladesh. Available at: http://country.eiu.com/Bangladesh. Accessed August 9, 2019. Page 46 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) Time Period FY2019 to 2020 US$, millions US$, millions In Tk, crores In Tk per year (5 Years) Post and Telecom Division Budget 2,397 23,970,000,000 292,317,073 1,462 Total MOPTIT Budget 4,044 40,440,000,000 493,170,732 2,466 Total Cost of Component 2 (Original n.a. 47.5 — 47.5 and AF Budget) Percentage of Cost/Budget n.a. — — 1.9% 11. The additional ROI estimation included for this ICR is presented in table 4.4. It shows an estimated ROI of 311 percent based on highly conservative approach and estimates. Table 4.4. ROI Calculation Benchmarked Savings by Commercial Customers (as used in original analysis) 15% - Less: Component Cost as Percentage of MOPTIT Budget 1.9% Estimated Savings in Percentage 13.1% Estimated Savings in US$, millions - Undiscounted 322.38 - Discounted at Average Commercial Lending Interest Rate (5 years cumulative) 147.65 Discounted Returns on Investment 311% Page 47 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Comments received from LICT Project Director Md. Rezaul Karim on December 9, 2019: “This is to inform you that I, the Project Director of Leveraging ICT for Growth, Employment and Governance Project and Joint Secretary, ICT Division, have gone through the ICR. The facts and figures that have been mentioned in the ICR are found to be consistent to the Project Completion Report (PCR) of Bangladesh Computer Council. The achievements mentioned in the ICR are also in line with the findings of the PCR. So, you can go ahead with the ICR and if any further information is required then I will be happy to provide you upon your request.” Page 48 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 6. SUPPORTING DOCUMENTS • Financing Agreement, Credit No. 5025-BD • Additional Financing Agreement, Credit No. 5911-BD • Amendment to Financing Agreement Letter, June 21, 2018 • Project Appraisal Document, Report No. 65376-BD • Project Restructuring Paper, Report No. RES14921 • Project Restructuring Paper, Report No. RES31411 • Project Additional Financing Restructuring Paper, Report No. PAD1990 • Implementation Status and Results Reports: 1 through 13 (February 2013 through June 2018) • Mid Term Review Report (September 2015) • Aide-Memoires: (a) March 5-18, 2011 (b) June 12-18, 2013 (c) September 8-11, 2014 (d) April 19-23, 2015 (e) August 30- September 7, 2015 (f) May 22-28, 2016 (g) December 11-15, 2016 (h) May 21-25, 2017 (i) September 10-14, 2017 (j) September 3-6, 2018 (k) October 29-November 1, 2018 (l) June 11-13, 2019 • Bangladesh Country Assistance Strategy FY11-15, Report No. 54615-BD • Bangladesh Country Partnership Framework FY16-20, Report No. 103723-BD • Memos regarding amendments to the Credit Agreements and Restructuring Papers • Bangladesh IT/ITES Industry Survey, June 2019 • BCC, Tales of Transformation, June 2019 • Borrower’s Implementation Completion Report, October 2019 Page 49 of 50 The World Bank BD: Leveraging ICT Growth, Employment and Governance Project (P122201) ANNEX 6. MAP OF BANGLADESH Page 50 of 50