Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004954 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A CREDIT AND GRANT IN THE AMOUNT OF SDR123.6 MILLION AND CAD19.8 MILLION (US$195.7 MILLION EQUIVALENT) TO THE PEOPLE'S REPUBLIC OF BANGLADESH FOR THE SKILLS AND TRAINING ENHANCEMENT PROJECT December 27, 2019 Education Global Practice South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Aug 06, 2019}) Currency Unit = BDT BDT 84.495 = US$1 US$ 1.37623 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Hartwig Schafer Country Director: Mercy Miyang Tembon Regional Director: Lynne D. Sherburne-Benz Practice Manager: Mario Cristian Aedo Inostroza Task Team Leader(s): Mokhlesur Rahman, Shiro Nakata ICR Main Contributor: Suzana Nagele de Campos Abbott ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank ADM Accountability and Decision-Making ADP Annual Development Plan AF Additional Financing ASSET Accelerating and Strengthening Skills for Economic Transformation BGMEA Bangladesh Garment Manufacturers and Exporters Association BMET Bureau of Manpower, Employment and Training BTEB Bangladesh Technical Education Board CAS Country Assistance Strategy CFAA Country Financial Accountability Assessment CGA Controller General Accounts CPF Country Partnership Framework CPS Country Partnership Strategy DA Designated Account DO Development Objective DPP Development Project Proposal DTE Department of Technical Education EA Environmental Assessment EC European Commission EFA Education for All EIA Environmental Impact Assessment EMF Environmental Management Framework EMP Environmental Management Policy FM Financial Management FY Fiscal Year GDP Gross Domestic Product GOB Government of Bangladesh HRMIS Human Resource Management Information System HSC Higher Secondary Certificate HSC (Voc) Higher Secondary Certificate (Vocational) IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results ICT Information and Communication Technology IDA International Development Association IDG Institutional Development Grant IDP Institutional Development Plan IFC International Finance Corporation IFRs Interim Financial Reports ILO International Labor Organization IMC Institutional Management Committee IOI Intermediate Outcome Indicator IP Implementation Progress IPP Indigenous Peoples Plan ISC Industry Skills Councils ISR Implementation Status Report IUFR Interim Unaudited Financial Report KPI Key Performance Indicators MDTF Multi-Donor Trust Fund M&E Monitoring and Evaluation MOE Ministry of Education MOEWOE Ministry of Expatriate Welfare and Overseas Employment MOLE Ministry of Labor and Employment MOU Memorandum of Understanding MS Moderately Satisfactory MTR Mid-Term Review NER Net Enrollment Rate NYPi Nanyang Polytechnic International NSAPR National Strategy for Accelerated Poverty Reduction NSDP National Skills Development Policy NSDC National Skills Development Council NTVQF National Technical and Vocational Qualification Framework OI Outcome Indicator OP Operational Policy PAD Project Appraisal Document PD Project Director PDO Project Development Objective PDV Present Discounted Value PFM Project Financial Management PIC Project Implementation Committee PID Project Information Document PIM Project Implementation Manual PIU Project Implementation Unit PMT Proxy Means Test PR Project Restructuring PSC Project Steering Committee RADP Revised Annual Development Plan RF Results Framework RPL Recognition of Prior Learning SDR Special Drawing Rights SMF Social Management Framework SSC Secondary School Certificate SSC (Voc) Secondary School Certificate (Vocational) STEP Skills and Training Enhancement Project TF Trust Fund TOC Theory of Change TOR Terms of Reference TSCs Technical Schools and Colleges TTCs Technical Training Centers TVET Technical and Vocational Education and Training US$ United States Dollars WB World Bank TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 11 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 21 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 23 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 23 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 25 A. KEY FACTORS DURING PREPARATION ................................................................................... 25 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 25 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 26 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 28 C. BANK PERFORMANCE ........................................................................................................... 29 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 30 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 32 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 43 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 46 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 47 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 52 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 54 ANNEX 7. SUMMARY OF BORROWER ICR ............................................................................. 55 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P090807 Bangladesh - Skills and Training Enhancement Project Country Financing Instrument Bangladesh Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Supplement P145118-Additional 26-Sep-2013 Recipient Executed Activities Financing: Skills and Training Enhancement Project Additional Financing P155389-Additional 30-Oct-2015 IBRD/IDA Financing Skills and Training Enhancement Project Organizations Borrower Implementing Agency PEOPLE'S REPUBLIC OF BANGLADESH Directorate of Technical Education Page 1 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Project Development Objective (PDO) Original PDO The Project Development Objective is to strengthen Training Institutions to improve training quality and employability of trainees, including those from disadvantaged socioeconomic backgrounds. FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing P090807 IDA-47640 79,000,000 78,745,377 77,148,515 P090807 TF-15113 15,709,907 15,670,555 15,670,555 P090807 IDA-57360 100,000,000 100,000,000 97,263,909 Total 194,709,907 194,415,932 190,082,979 Non-World Bank Financing 0 0 0 Borrower/Recipient 9,000,000 29,800,000 25,239,000 Total 9,000,000 29,800,000 25,239,000 Total Project Cost 203,709,907 224,215,932 215,321,979 KEY DATES Project Approval Effectiveness MTR Review Original Closing Actual Closing P090807 10-Jun-2010 22-Aug-2010 30-Jun-2016 30-Jun-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 18-Jan-2014 28.63 21-Jan-2016 80.16 Additional Financing 29-Jun-2017 127.41 Change in Loan Closing Date(s) Page 2 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) KEY RATINGS Outcome Bank Performance M&E Quality Highly Satisfactory Satisfactory High RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 21-Feb-2011 Satisfactory Satisfactory 4.93 02 21-Sep-2011 Satisfactory Moderately Satisfactory 4.93 03 30-Apr-2012 Satisfactory Moderately Satisfactory 10.61 04 29-Oct-2012 Satisfactory Satisfactory 15.28 05 17-May-2013 Satisfactory Satisfactory 20.06 06 06-Dec-2013 Satisfactory Satisfactory 28.63 07 04-Jun-2014 Satisfactory Satisfactory 34.60 08 30-Jul-2014 Satisfactory Satisfactory 52.67 09 22-Jan-2015 Moderately Satisfactory Moderately Satisfactory 52.67 10 31-Mar-2015 Satisfactory Moderately Satisfactory 60.59 11 08-Oct-2015 Satisfactory Moderately Satisfactory 75.21 12 26-Apr-2016 Satisfactory Satisfactory 81.82 13 20-Sep-2016 Satisfactory Satisfactory 109.29 14 22-Mar-2017 Satisfactory Moderately Satisfactory 109.29 15 03-Oct-2017 Satisfactory Moderately Satisfactory 127.41 16 03-Apr-2018 Satisfactory Satisfactory 141.40 17 16-Nov-2018 Satisfactory Satisfactory 159.71 Page 3 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 5 Other Public Administration 5 Education 90 Tertiary Education 90 Industry, Trade and Services 5 Other Industry, Trade and Services 5 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 10 Trade 10 Trade Facilitation 10 Private Sector Development 110 Business Enabling Environment 10 Innovation and Technology Policy 10 Jobs 100 Public Sector Management 5 Public Administration 5 Municipal Institution Building 5 Page 4 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Human Development and Gender 76 Education 76 Access to Education 19 Science and Technology 19 Teachers 19 Standards, Curriculum and Textbooks 19 ADM STAFF Role At Approval At ICR Regional Vice President: Isabel M. Guerrero Hartwig Schafer Country Director: Ellen A. Goldstein Mercy Miyang Tembon Director: Michal J. Rutkowski Lynne D. Sherburne-Benz Practice Manager: Amit Dar Mario Cristian Aedo Inostroza Nalin Jena, Venkatesh Mokhlesur Rahman, Shiro Task Team Leader(s): Sundararaman Nakata Suzana Nagele de Campos ICR Contributing Author: Abbott Page 5 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At the time of appraisal of the Skills and Training Enhancement Project (STEP, the Project), Bangladesh had been enjoying continued economic growth, despite some slow-down due to the global economic environment, thanks to a vital contribution of the export sector and remittances from migrant workers. The growth of RMG and knitwear exports had been impressive, and Bangladesh ranked second only to China in terms of global market shares. Exports accounted for an estimated 20 percent of GDP, making Bangladesh one of the leading exporting Least Developed Countries in the world. Bangladeshi migrant workers sent an average annual remittance of about US$816 per annum that totaled an estimated US$9 billion. However, the vast majority of Bangladeshi migrant workers were unskilled. Skilling the workforce could increase their wages in international labor markets and protect the workers from unemployment in the event of economic downturns. Further, about 85 percent of Bangladesh’s exports were based on only seven or eight products, and continued weak demand from export markets was seen as have a potentially adverse impact on the country’s GDP. The economy slowed down around 2008 due to the decline in aggregate demand, which had been affected by a decrease in exports and a reduction in the inflow of remittances as job losses mounted in host countries. 2. The Government of Bangladesh (GOB) sought to boost economic growth by ensuring that its workforce was adequately skilled so that its key industries would remain competitive in global markets. The size of the labor force had grown considerably, fueled by greater participation of women and youth. As a result, the demands on institutions delivering education and skills had increased significantly. Hence, the GOB sought to improve access to skills and training programs, improve their quality and effectiveness, and strengthen the overall governance in the education and skills sectors. Bangladesh had witnessed tremendous gains in primary and secondary education outcomes. Net enrollment rates (NER) at the primary and secondary levels were 90 percent and 45 percent, respectively, with gender parity. However, overall literacy rates were below 50 percent, and school completion rates were meager. The labor force’s average years of schooling was 4.2 years, lower than that of the overall population’s 4.4 years. Only a tiny proportion of the labor forces had any vocational education or training. Additional efforts were needed to harness the strength of its population. 3. TVET provision in Bangladesh suffered from weak governance structures, low quality, and a lack of relevance of programs to the needs of the labor market, high rates of internal inefficiencies, and a lack of resources. Access to TVET was low, especially for women and the economically disadvantaged youth. As a result, only a tiny proportion of the labor force had any vocation education or training. For those that did, the quality of training provided by TVET institutions was poor, with an overemphasis on theoretical work and minimal emphasis on practical work. Overly centralized structures and procedures made it difficult for institutions to ensure demand orientation and left little room for industries and employers to play a role in identifying the skills needed by industry. The TVET sector’s governance, management, and financing made it challenging to improve the relevance and quality of training while ensuring access, with equity, to an ever- increasing number of skilled workers required by industry. The TVET sector is highly fragmented with more than 20 ministries offering programs with varied skills training programs. There was little or no coordination Page 6 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) or standards, poor standardization and equivalency among different programs and levels, weak accreditation to ensure that trainees have achieved the minimum level of expected competencies. Technical institutions had a large unutilized capacity, with low pass rates and employability of graduates. TVET financing was also low. Bangladesh spent less than 1.5 percent of its education budget on all technical education and training in the country. Page 7 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) PROJECT: PDO -- to strengthen Training Institutions to improve training quality and employability of trainees, Theory of Change (Results Chain) including those from disadvantaged socio-economic backgrounds Project Activities Outputs Immediate Project Outcomes Long-term Outcomes Components • Support to Institutional • Institutional [PDO Indicators] Development Plan (IDP) of Public Development Grants (IDG) • Pass rates of diploma programs increased and Private Institutions Offering awarded competitively to Diploma • Employment rates and completion rates for Diploma Programs Institutions trainees of short-term courses increased Improve • Support to IDP of Public • IDG awarded • Stipends increased and enrollment of the Quality and and Private Short-term training competitively to short-term training disadvantaged and female students increased • Enrollment in TVET Relevance Institutions institutions increased of Training • Provision of Stipends to • Stipends provided to [Other Outcomes] • Private sector disadvantaged student of diploma disadvantaged and female students • Enrollment in supported courses increased participates continuously as a and short-term training • Teachers contracted • Teacher vacancies reduced partner in designing and institutions • Teachers trained • Share of females in enrollment increased delivering of TVET • Quality of TVET improved significantly • Supporting Industry Skills • ISCs/NSDC have office • TVET contributes Pilots In Councils (ISCs) and the NSDC and TA and functioning • Links between TVET sector and private significantly to more and TVET • Strengthening the • SSC (Voc) pilot sector strengthened better employment of TVET Secondary School Certificate (Voc) apprenticeship program evaluated graduates • All students • Strengthening BTEB to interested in pursuing TVET support quality assurance and • Number of staff • Increased sectoral capacity for improving are able to do so strengthen curriculum devt. trained in core expertise quality, relevance and delivery of TVET programs, • TVET contributes • Improving BTEB areas in DTE, BTEB and BMET and coordinating delivery among sectoral significantly to meeting Institutional • RPL program institutions Capacity monitoring and certification demand for skilled workers • Strengthening capacity for become operational • Increased capacity for monitoring and both in Bangladesh and abroad Development planning, curricula & standard evaluation, inter alia, through better and more setting for BMET timely information and results • Implementation of • Workers skills assessed through RPL Recognition of Prior Learning (RPL) • Project Implementation • Awareness raising • More students become aware of TVET programs Project Unit campaigns of project-supported Mgt, • Results of monitoring and evaluation are used as Communic • Communications with initiatives stakeholders input to improve design and delivery of TVET ation and • Impact evaluation M&E • Monitoring and study programs evaluation of project activities Page 8 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Project Development Objectives (PDOs) 4. The Project Development Objective, as specified in the Financing Agreement, was to strengthen Training Institutions to improve training quality and employability of trainees, including those from disadvantaged socio-economic backgrounds. The wording of the PDO in the Project Appraisal Document (PAD) was slightly different and specified that the Project would strengthen “selected public and private training institutions”1. Key Expected Outcomes and Outcome Indicators 5. The key expected outcome was to make the TVET sector more responsive to current and emerging labor market needs for a skilled workforce. The PDO indicators described in the PAD were the following2: • Pass rates of students from supported Diploma programs; • Employment rates for trainees of short-term courses within 6 months of course completion; • Number of students receiving stipends in supported polytechnics and short-term training courses (student- years); and • Achievements in vocational competencies assessed on a sample basis. Components 6. Component 1: Improve the Quality and Relevance of Training (Total US$72.9 million; IDA US$65.4 million; Actual US$155.7 million). This component was envisaged to support public and private institutions chosen based on well-defined eligibility and selection criteria under two sub-components. 7. Sub-Component 1.1: Window I: Support to Public and Private Institutions Offering Diploma Programs (Total US$58.9 million; IDA US$51.4 million)3. Window I would focus on improving the quality and relevance of diploma level technical education in 33 public and private institutions based on eligibility and selection criteria. Once endorsed by the Project Implementation Committee (PIC), participating institutions would prepare a comprehensive Institutional Development Plan (IDP). Implementation grants would be subject to a ceiling of US$1 million and financed (a) infrastructure rehabilitation and refurbishment; (b) modernization of equipment and teaching and learning instruments; (c) capacity development in the form of course/training materials, training, study tours, and exposure visits; (d) remuneration for guest faculty from industry; (e) teacher training; (f) students’ training materials and cost towards industry visits; and (g) improvements in the administration of institutions, outreach, communications, and placement improvements. An Annual Performance Grant of up to US$200,000 would be awarded to the top five 1 World Bank, Project Appraisal Document, Skills and Training Enhancement Project, Report No. 53549-BD dated May 11, 2010. 2 The wording of the PDO Indicators in Annex 1 of the PAD differed slightly from that in the main text. The wording in Annex 1 is shown in this section as it was more detailed. 3 Bangladesh’s formal pre-employment Technical Vocational Education and Training (TVET) system consisted of three levels, with an intake capacity of about 500,000 students. The three levels include: (i) short-courses through a range of short-term, three to six-month training courses; (ii) two-year vocational programs for students in secondary and higher secondary education (SSC (Voc) and HSC (Voc)); and (iii) a four-year post-secondary Diploma program. Three agencies oversee the pre-employment TVET system: (i) the Directorate of Technical Education (DTE) under the MOE, responsible for setting the overall policy framework of vocational education and training; (ii) the Bangladesh Technical Education Board (BTEB), a statutory agency, also under the MOE, responsible for maintaining the training standards, conducting assessments and examinations, and providing student verification, and helping accredit institutions; and (iii) Bureau of Manpower, Employment and Training (BMET) under the MOEWOE focuses on estimating demand for labor in both domestic and overseas markets. Page 9 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) performing institutions, as measured at the end of the third year of implementation through performance indicators. 8. To increase access to post-secondary programs to students from socio-economically disadvantaged backgrounds, the Project provides stipends (US10/monthly)4 to students in eligible polytechnics who were estimated to be in the bottom two income quintiles. The screening was done using Proxy-Means Testing (PMT)5. Beneficiaries were expected to meet the following criteria to remain eligible: (a) maintaining 75 percent attendance, and (b) maintaining 45 percent pass marks in the annual examinations. 9. Sub-Component 1.2: Window II: Support to Public and Private Institutions Offering Short-term Programs (Total US$14.0 million; IDA US$14.0 million). Window II would support training programs of a minimum of 360 hours with a duration of six months or less for about 40-50 institutions. Institutions and courses offered would be selected based on development proposals submitted by the institutions, and those selected would be provided an implementation grant of up to US$250 per student enrolled in the course. Proceeds of the grants covered (a) tuition costs to the institution; and (b) stipends of US$10 per month to students in courses supported by the Project. The Project was expected to support over 70,500 trainees over its life. 10. Component 2: Pilots in TVET (Total US$5.5 million; IDA US$5.3 million; Actual US$6.2 million). This component was designed to support two types of pilot activities aimed at strengthening the TVET system, through (a) direct start-up and operational support to Industry Skills Councils and the National Skills Development Council, and (b) assistance to Secondary School Certification (Vocational) schools (SSC (Voc)) to determine the most effective means to strengthen the programs. 11. Sub-Component 2.1: Supporting Industry Skills Councils (ISCs) and the National Skills Development Corporation (NSDC) (Total US$0.5 million; IDA US$0.3 million). This sub-component was designed to provide one-time grant financing and operational cost support during the five years of the program (on a declining basis) for the establishment of specific offices of about two to four ISCs or NSDC, in addition to all the outreach activities expected of the ISCs6. The Better Business Forum would also benefit from the support for the NSDC. 12. Sub-Component 2.2: Strengthening the Secondary School Certificate (Vocational) (Total US$5.0 million; IDA US$5.0 million). This sub-component was designed to implement and evaluate several different interventions and strategies to improve the effectiveness of essential vocational training opportunities for vulnerable students that were considered likely to take employment after secondary education rather than continuing further education. The existing SSC (Voc), as implemented, was deemed to be ineffective, with few students graduating with employable skills, and the pilots were expected to determine a more effective way of program delivery and improved employment outcomes. 4 The amount was roughly the minimum amount needed to cover out-of-pocket daily expenses for educational activities. Bangladesh has a long history of providing stipend to boost school enrollment especially for girls. Those programs have been very popular among students and parents. 5 Using the same Proxy-Means Testing methodology as is used in Secondary Education Quality and Access Enhancement Project to determine the individual’s poverty status 6 A number of ISCs were being established under the framework of the TVET Reforms Program of the EC-ILO with the aim of providing critical information to the TVET sector about the current and projected human resources and training needs in selected industries, accurate data on skills bottlenecks in these industries with the aim of helping to develop industry-led standards, and training products and services that will be recognized and accredited by national bodies such as BTEB, and by other international accreditation agencies, or by industry representatives. The NSDC role was to guide the overall regulatory and legislative framework for TVET and skills development in Bangladesh. Page 10 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) 13. Component 3: Institutional Capacity Development (Total US$4.7 million; IDA US$4.4 million; Actual US$34.8 million). This component was designed to strengthen the Government’s capacity to manage the TVET sector by supporting institutional capacity development of the Directorate of Technical Education (DTE), the Bangladesh Technical Education Board (BTEB) under the Ministry of Education (MOE), and the Bureau of Manpower Employment and Training (BMET) under the Ministry of Expatriate Welfare and Overseas Employment (MOEWOE). 14. Component 4: Project Management, Communications, and Monitoring and Evaluation (Total US$4.9 million; IDA US$3.9 million; Actual US$13.0 million). Sub-Component 4.1: Project Management and Communications (Total US$4.3 million; IDA US$3.3 million). This sub-component was designed to help establish a separate STEP Project Implementation Unit (PIU), a project management and implementation structure, to support the improvement of system management and implementation of reforms through training of policy planners, managers, and administrators and enhance communications. 15. Sub-Component 4.2: Project Monitoring and Evaluation (Total US$0.6 million; IDA US$0.6 million). This sub- component was designed to support the monitoring and evaluation (M&E) of project progress and outcomes, and enhancing the M&E capacities in the training institutions receiving support under the Project. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) 16. The STEP underwent two rounds of Additional Financing (AF) with project restructuring to meet emerging needs. Each of the AFs and corresponding project restructurings incorporated lessons and findings from implementation and evaluations that ensured the Project’s continued relevance to the Government’s priorities. For instance, the teams identified issues with female enrollment that led to an adjustment to make stipends available to all female TVET students regardless of socio-economic standing7. Also, the Project was adjusted based on findings of the evaluation of the SSC (Voc) program that recommended downsizing the SSC (Voc) pilot towards an operational pilot to establish a standardized apprenticeship program that would strengthen the labor market linkages of SSC (Voc) institutions. The original Project financed the development of the Recognition of Prior Learning program, and based on progress, the subsequent AFs were adjusted to fund first, the roll-out of RPL to several assessment centers and implementation, and later, an impact evaluation of the program. Also, given the Government’s plans to promote efficiency, transparency, and accountability in public financing by moving to e-Government procurement, the second AF responded to its request to move to e-Government Procurement in central TVET agencies and selected polytechnics. 17. The first AF in the amount of US$16.71 million, was approved in September 2013 under a Multi-Donor Trust Fund with funding from Canada. At the time, US$2.75 million had been disbursed from the original credit, and the Project was rated Moderately Satisfactory (MS) for both Development Objectives (DO) and Implementation Performance (IP). The AF was to help finance the costs associated with the need to provide additional resources under the original components of the Project to (a) scale-up ongoing institutional grants to additional public and private institutions; (b) fill the funding gap for stipends which emerged as a result of higher demand for the poverty-targeted stipends than originally anticipated; and (c) scale-up 7 It was recognized that most of female students were screened out in the means-testing process for stipend. Female students in post-secondary TVET tend to be from less impoverished families compared male students as only families with some financial means would consider post- secondary education for girls. To address this reality and achive the goal of increasing female enrollment, stipends were made available to all female students in the supported polytechnics. Page 11 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) capacity building, studies, communications and monitoring, and evaluation activities. For sub-component 2.2: Strengthening the SSC (Voc), it was decided to finance a pilot for the apprenticeship program. The pilot would address the disconnect between SSC (Voc) programs with employers’ skill needs by establishing an institutionalized apprenticeship program. 18. A second AF in the amount of SDR 71.3 million (US$100 million equivalent) was approved in October 2015. At the time, US$19.5 million had been disbursed, and the Project was rated MS for both DO and IP. The AF was to help finance the costs to (a) scale-up ongoing institutional grants to additional public and private institutions; (b) fine-tune existing schemes and mechanisms, including (i) refinement of stipend program; (ii) greater flexibility in technology/trade support in polytechnics and short-courses; (iii) introduction of e- government procurement (e-GP); and (iv) enhancement of communication and mobilization activities; and (c) introduce further innovations: (i) digitizing teaching and learning contents; (ii) improving in-service pedagogical and subject training in Technical Teachers’ Training College (TTTC), Technical Teachers’ Training Institute (TTTI) and Vocational Teachers’ Training Institute (VTTI); and (iii) impact evaluation for RPL activities. The RF was revised to reflect the expanded scope; and the introduction of sub-indicators for disaggregated groups (gender and disadvantaged). The Closing Date of the AF was set on June 30, 2019. Revised PDOs and Outcome Targets 19. Revisions to the PDO Indicators in the AFs and PRs are presented in Table 1. Revisions to Intermediate Outcome Indicators are presented in Annex 1. 20. In addition, the Intermediate Results Indicators were adjusted to reflect the following: (a) new targets to cover the Project’s expanded scope; (b) a minor rewording of indicators; (c) deleting the intermediate outcome indicator related to “internal revenue generated as a share of recurrent expenditure” as it would require legislative changes; and (d) setting targets for two intermediate outcome indicators, “evaluation of SSC (Voc) pilot” and “impact evaluation”. Table 1. Revision of PDO Indicators EOP Targets No. PDO Indicators Changes made and reasons Original 1st AF 2nd AF Pass Rates of students from New targets were added due to the 60 60 65 supported Diploma Programs expanded project scope and duration. This sub-indicator was added to track the - - 68 Female (sub-indicator) outcome for disadvantaged groups 1. This sub-indicator was added to track the - - 60 Disadvantaged Students (sub- outcome for disadvantaged groups. indicator) Disadvantaged students refer to students receiving stipends. Employment Rates for trainees New targets were added due to the 44 44 46 2. of short-term courses within 6 expanded project scope and duration. months of course completion Modified to “No. of Disadvantaged students 32,000 97,000 175,000 Number of Students receiving supported through stipends selected based 3. stipends in supported on household information based on Page 12 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Polytechnics and short-term selection criteria” to reflect the total training courses (student-years) number of disadvantaged students (*1). Modified to intermediate results indicator TBD - - as the nature of the activity is Achievements in vocational piloting a new assessment 4. competencies assessed on a concept and is part of the larger sample basis system improvement rather than an outcome by itself. Note: (1) Students enrolled in the eligible short-term training courses were automatically qualified to receive stipends. Revised Components 21. The first AF adjusted Sub-component 2.2, Strengthening the SSC (Voc), to finance an operational pilot that would respond to one of the key findings of the assessment study that was discussed during the Mid-Term Review. The operational pilot would address the disconnect between SSC (Voc) programs with employers’ skill needs by establishing an enhanced apprenticeship program that would strengthen the labor market linages of SSC (Voc) institutions by mobilizing various stakeholders and preparing operational manuals. Other Changes 22. The process of awarding Annual Performance Grants to IDGs under Sub-component 1.1 was canceled. This cancellation was because institutions faced initial challenges in the utilization of grants due to limited absorptive capacity. This dampened the motivation of awarding additional grants to the institutions. 23. The first AF included a reallocation of resources from Component 2 to Component 1 to fill the funding gap under the stipend programs due to higher than estimated demand from project beneficiaries. It also supported the decision to extend stipends to all female students in supported polytechnics to increase their participation and decrease their drop-out rates. The second AF also adjusted project costs and the implementation schedule, procurement arrangements (to introduce e-GP and framework agreements). The Closing Dates for both the original IDA Credit and the MDTF were extended from June 30, 2016 to June 30, 2017 to align better with the AF closing date of June 30, 2019. Rationale for Changes and Their Implication on the Original Theory of Change 24. The changes described above were mostly to reflect new scope and targets resulting from AFs and the expansion of activities given implementation progress. The modifications caused no change in the Project’s Original Theory of Change, and only intensified efforts to achieve expected outcomes. The expansion of stipends to all female students strengthened the Project’s goal of increasing female enrollment in TVET as it gave a firm assurance and strong incentive of financial support to families of female students. In the country context, female students who were enrolling in TVET institutions tended to be from non-poor families that Page 13 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) could afford post-secondary education. As a result, the design of the stipend program provided solely based on means-testing results was largely excluding female students. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 25. The Relevance of the PDOs against the World Bank’s Country Partnership Framework for Bangladesh for the Period FY16-FY20 is High8. At Appraisal, the Project’s objectives and design were relevant to the development priorities of the Bangladesh government and consistent with the World Bank’s CAS to address the skills needs and gaps to enhance the competitiveness of Bangladesh in the global market and boost export-led growth. The government was keen on increasing the skills level of the entire population to increase the return to the economy and remittance from migrant workers. 26. Skills development continues to be the priority for the Bangladesh government and the World Bank’s engagement. At closing, the PDO remains well aligned with the strategies outlined in the World Bank Country Partnership Framework for Bangladesh for the Period FY16-FY20, specifically under its Focus Area 2: Social Inclusion. The CPF highlights that Bangladesh’s sustained growth will require continued attention to worker productivity and the protection of the poor. WBG activities will aim to consolidate the equity and access gains in health and education with constant due regard to gender and address the next generation of challenges related to quality improvements in health, access to clean drinking water and sanitation, and education and skills development. The Project is in full alignment with this country strategy of the WBG. In the labor market, most of the workforce remain undereducated, and the skills mismatch poses a significant recruitment problem for employers, putting worker productivity and the protection of the poor at risk. In the skills sector, the Project supported efforts to reform technical and vocational skills development to forge stronger links with industry and reduce skills mismatch. B. ACHIEVEMENT OF PDOs (EFFICACY) 27. The Project Development Objective, as specified in the Financing Agreement, was to strengthen Training Institutions to improve training quality and employability of trainees, including those from disadvantaged socio-economic backgrounds. The PDO had the following three objectives. (1) To strengthen training institutions to improve quality. (2) To strengthen training institutions to improve the employability of trainees. (3) To improve training quality and employability of trainees from disadvantaged socio-economic backgrounds. The Project’s PDO remained unchanged throughout implementation. Minor adjustments in its RF 8World Bank, Country Partnership Framework for Bangladesh for the Period FY16-FY20, Report No. 103723-BD dated March 8, 2016. Page 14 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) were to reflect new targets resulting from AFs and the expansion of original activities and the introduction of new ones based on the progress in TVET reform actions. Therefore, this analysis does not evaluate the Project efficacy weighted by disbursements in after each of AFs/PRs. The Project was never at risk, as reflected in the Implementation Performance and PDO ratings in its ISRs. Split ratings should not alter the overall efficacy rating. Details on project outputs are in Annex 1. 28. The Project was hugely successful in the achievement of its objectives. At completion, STEP had exceeded by far targets for all its Outcome Indicators and all but one of its Intermediate Outcome indicators. More importantly, the Project’s catalytic effect of its various activities has resulted in profound changes in Bangladesh’s TVET sector. There has been a marked transformation in the public’s perception of technical education. An ever-greater number of aspiring students see technical education as an attractive option9. Participating institutions have witnessed the benefits of establishing strong linkages with industry10. Partnerships forged with foreign technical training institutes have raised the bar in terms of expectations for training quality. Assessment of Achievement of Each Objective/Outcome 29. Objective 1 – Strengthen training institutions to improve quality: Progress towards achievement of this objective was High. This objective was measured by the PDO Indicator 1, which tracked the pass rates of students from supported diploma programs. There was notable achievement towards this objective, exceeding expectations as measured by the key indicators to measure progress. The pass rate11 increased from 50 percent at baseline to 86.5 percent, exceeding the revised target of 65 percent. It is also above the national average pass rate of 77.9 percent. In 2016, the government initiated a reform in examination methodologies, which resulted in fluctuation in the pass rates in the next two years12. However, the project supported polytechnics demonstrated continuous improvement trajectory in their pass rates and also marked significantly higher pass rates than the national average13. 9 At the baseline, supported polytechnics enrolled only 55 percent of the sanctioned intake capacity. By the project completion, 99 percent of the intake capacity was filled, while the number of intake capacity also increased significantly. 10 All of the project supported institutions – both polytechnics and short-course providers – have established Institution Management Committee (IMC) which includes representatives from local industries to involve industries and employers in management and teaching processes. 11 Pass rate refers to the rate of students passing the final national examination at the end of four year diploma course. This examination determines whether students are qualified to graduate from the course. Transition rate, which is used as an Intermediate Results Indicator to measue student retention, refer to the rate of students passing the year-end examination which determines whether students can move on to the next grade. 12 For instance, in 2016, overall pass rate shot up significantly. From 2017 onwards, pass rates stabilized as the transition was completed. As such, the use of pass rate as a proxy of quality continues to be valid. 13It should be noted that the Project’s supported institutions were selected based on their merit and are likely to have had better pass rates than others. Nevertheless, overall improving trend for supported institutions demonstrate the steady improvement over time, and the substantial gap from the national average attests to significant gains in quality. Page 15 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) 30. The performance of both females and disadvantaged students exceeded that of the average of all students: the overall pass rates of females and disadvantaged students were 89.5 percent and 92.5 percent, respectively. 31. The IDG activities, such as strengthening linkages to the industry, led to the improved learning experience and increased popularity of technical education at polytechnics and short-course institutions. The provision of stipends to disadvantaged students also had positive impacts on the learning experience. The shortage of teachers was a long-standing concern in polytechnics institutes. By contracting and training over 1,100 contractual teachers, the share of vacancies in sanctioned faculty posts of project-supported courses fell from 25 percent at baseline to 3 percent at completion (against a target of 5 percent). 32. Pedagogical and subject-based training of teachers received special attention under the Project. A total of 7,980 teachers of polytechnics and short-term courses received training against a target of 6,000. The Project established the Dhaka Technical Teachers Training Institute, which trained 1076 and 1126 teachers, respectively, for subjects and pedagogy. It formed a formal partnership with Nanyang Polytechnic International (NYPi) in Singapore to train 1,794 teachers and 158 managers of polytechnics. It also went into a formal partnership with Guangzhou Industry and Trade Technician College in China to train 553 teachers and 44 managers of short-course institutions. A teacher training survey conducted in November 2015 in nine project-supported polytechnics found that, overall, teachers were satisfied with the training. 33. STEP also supported a range of awareness-raising regarding technical education through posters, radio and television clips, the internet, and other means. The determination to improve the quality of training coupled with initial positive results have sparked a renewed interest in technical and vocational education. By completion, 36 awareness-raising campaigns had been carried out against an end-of-project target of 30 campaigns. Enrollment as a proportion of sanctioned intake capacity in supported diploma courses increased from 55 percent at baseline to 99 percent at completion (against a target of 95 percent). 34. The academic performance of students has improved and resulted in better student retention. The transition rate from the second to the third semester in supported diploma courses increased from 80 percent at baseline to 98 percent at completion (against a target of 95 percent). The completion rates in supported short-courses also increased significantly from 50 percent at baseline to 96 percent at completion (against a target of 95 percent). 35. The Satisfaction Survey (student, faculty, and employers) for polytechnic institutions provides highly encouraging results that support the above findings.14 Both students and teachers of beneficiary institutions rated significantly higher than those of the control group on most of the indicators of quality improvement, as summarized in Table 2. More importantly, the responses of the majority of the employers interviewed in the survey are encouraging. Twenty-one percent of employers maintain regular contact with polytechnics. Fifty-nine 14 The Satisfaction Survey (Student, Faculty and Employers) for IDG recipient institutions under the Skills and Training Enhancement Project (STEP) dated December 2, 2018 surveyed 4975 students and 482 teachers of 33 polytechnic institutes (of which 18 were public and 5 were private STEP beneficiaries and 8 were public and 2 were private control) and 66 public and private employers with a structured questionnaire and held 27 focus group discussions in 9 polytechnics with faculty members. Page 16 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) percent reported that there was a good demand for graduates from these polytechnics in the job market. Eighty-two percent found that these graduates were good workers with the desired skills. The only area they found weak was the effectiveness of the job placement cell. Overall, three-quarters of the employers were satisfied with the graduates of project- supported polytechnics whom they hired over the previous 12 months. Table 2: Average Rating of Students and Teachers on Selected Indicators of Quality Improvement Response of Students Response of Teachers Topic Indicator Beneficiary Control Beneficiary Control Polytechnic Polytechnic Polytechnic Polytechnic Curricula, Standard of textbooks 7.4 7.17 6.85 6.30 Textbooks Relevance of project work 8.0 7.58 8.63 8.07 Quality of Theoretical class 7.67 7.18 8.09 7.74 Teaching Practical 7.13 6.22 7.22 7.11 Teaching Laboratory equipment 6.61 4.91 7.37 6.41 Facilities Computer/software 6.39 4.83 - - Teachers’ Teaching complex subjects 7.64 7.26 8.56 8.44 Capacity Overall teaching capacity 8.00 7.67 8.63 8.22 Career Assistance by the polytechnic 7.39 6.73 8.12 8.01 Development Effectiveness of placement cell 6.05 5.07 7.04 6.59 Note: Ratings are on a scale of 10, where 1 is least satisfied, and 10 is the most satisfied. 36. Objective 2 – Strengthen training institutions to improve the employability of trainees: Progress towards achievement of this objective was High. The success of this objective was measured by the employment rate for trainees of short-term courses within six months of course completion15. The employment rates for trainees of short-courses within six months of course completion increased from 30 percent at baseline to 51 percent at completion, against a formally revised target of 46 percent. This improvement in employability is particularly remarkable in the job market context, where youth unemployment continued to be a thorny challenge. The Labour Force Survey (LSF) 2010 reported 5.4 percent unemployment rate for the age group of 25-29 years, while the LFS 2017 reported 5.7 percent unemployment rate for the age group of 25-34 years, which shows no improvement in job market environment for youths who are the majority of short-course trainees. There was also a notable achievement in strengthening training institutions to improve the employability of trainees, as measured by the key indicators selected to measure progress. 37. Both polytechnics and short-course institutions implemented practical actions to improve the employability of trainees. All the beneficiary institutions have established Institutional Management Committees (IMCs). The IMCs are composed of representatives of the training institution and local industries and ensure that the training meets the needs of local industries. Beneficiary polytechnics have established over 300 formal partnerships with industries to 15The Project focused on the employment rate of short-course graduates, rather than that of polytechnic graduates, as the post-graduation progression of polytechnic students is diverse. It is often the case that polytechnic graduates move on to higher education or other further education opportunities, depending on the availability of such opportunities. This situation makes the use of the employment rate of polytechnic graduates less reliable. Page 17 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) cement strong institution-industry linkages. These partnerships have promoted the refurbishment of labs and facilities, industry exposure for students through seminars by guest lecturers, and support for job placement. STEP also promoted the establishment of job placement cells in all project-supported polytechnics and short-course institutions. The mandate of these offices is to provide career counseling and job search skills training, facilitate the transition to employment, and help intermediate job placement between graduates and potential employers. The Project organized 33 job fairs where job seekers can contact interested employers. DTE has implemented a job information portal where graduates can find information on employment opportunities. The success of the first job fair, where around 1,200 students received job offers or commitments from employers, leads to the carrying out of this event yearly. The National Skills Competition was started in 2014 and held each year successfully to help showcase the skills, talent, and potential for innovation of polytechnic students and promote the value of technical training. 38. The pilot apprenticeship program provided workplace experience to 9,125 SSC (Voc) students to enhance their practical skills and employability. The apprenticeship program was offered in five trades (electrical, refrigeration and air conditioning, welding, dressmaking and tailoring, and food processing) in 20 institutes (10 public and 10 private). The apprenticeship program aimed to enhance the job relevance of the SSC (Voc) program through the partnership with industry. Upon completion, students were assessed under the NTVQF assessment framework. The ex-post assessment study of the SSC (Voc) Apprenticeship Program found that 62 percent and 25 percent of students in the pilot program were, respectively, highly satisfied and satisfied with the quality of the program16. Eighty-six percent of the beneficiary students reported that participation in the pilot program increased their skills. Eighty percent of them found that industry training increased their self-confidence. Because of their age, only 5.0 percent of pilot program students in the treatment group sought employment upon completion. Most of the other students are still in the education system. The average monthly income of these employed graduates was about 250 percent above that of employed graduates in the control group. 39. To improve job market opportunities for informal sector workers, STEP piloted and expanded Recognition of Prior Learning (RPL). The STEP rolled out the RPL program initially to 10 assessment centers affiliated to BTEB, and then to an additional 10 centers under the second AF. In all, 39,060 persons enrolled in the program to be assessed, exceeding the end-of-project target of 30,000 persons. The assessment of the RPL found that participants found the orientation program useful, relevant to the practical field of work, with suitable raw materials and tools for the RPL assessment.17 The assessment reports that there was a significant increase in full-time and part-time wage employment in public and private entities after the RPL assessment, while there was a substantial decrease in self-employment, 16 Evaluation of SSC (Voc) Apprenticeship Piloting Program dated July, 2018 surveyed 700 students in 7 SSC (Voc) institutes that participated in the pilot (treatment group) and 400 students in 7 SSC (Voc) institutes that did not participate (control group) through face-to-face interviews using a structured questionnaire. Focus group discussions were held with teachers of SSC (Voc) institutes participating in the pilot, and key informant interviews were held with directors/principals/heads of those institutes, with trainers and owners of industry partners involved with the industry training program and with employers of pilot program graduates. 17 Report on Recognition of Prior Learning (RPL) Assessment, dated August 2016. The assessment collected data from 600 Competent, 200 Partially Competent assesses, from 10 RPL assessment centers and 200 control respondents (who applied but were not selected for RPL assessment, through face-to-face interviews using a structured questionnaire. Focus group discussions were held with teachers of RPL assessment centers, and key informant interviews were held with directors/principals/heads of assessment centers, BTEB registered industry assessor, BTEB officials and employers of the RPL assesses. Page 18 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) unemployment, and student/trainees. Most of the Competent (77 percent) and Partially Competent (91 percent) assesses reported greater social recognition. More importantly, the monthly income of RPL participants was significantly higher for those that received Competent and Partially Competent assessments alike, as presented in Table 3. Table 3: Minimum, maximum and average monthly income of RPL assessees Minimum Salary Maximum Salary Average Salary BDT BDT BDT Competent Partially Competent Partially Competent Partially Competent Competent Competent Wage employed Before Assessment 1,500 1,500 18,000 17,000 6,226 6,163 After Assessment 3,000 2,500 30,000 28,000 13,228 11,967 Self-employed Before Assessment 9,000 9,000 25,000 25,000 14,183 12,170 After Assessment 15,000 15,000 45,000 40,000 18,983 15,990 40. The Project also addressed the system-level capacity to enhance the market relevance of TVET programs through its support to ISCs. There are 12 ISCs in Bangladesh as of December 2019. The ISCs have an institutional mandate to create linkages between skills program providers and private industries. Development of competency standards, participation in curriculum revisions, labor market analysis, and coordination of skills training are part of their vital missions. Different development partners support ISCs. The Project took up two ISCs – ICT ISC and Light Engineering – to provide support through NSDC. Although there were challenges in coordination and institutional capacity of NSDC and ISCs, the Project supported ICT ISC to develop market-aligned competency standards in several ICT occupations and integrate them into the national framework. 41. Objective 3 – Improve training quality and employability for trainees from disadvantaged socio-economic backgrounds. Achievement of the inclusiveness aspect of this objective was measured by the number of disadvantaged students supported through stipends selected based on household information-based selection criteria (PDO Indicator 3). The sub- indicators (female students and low-income students) of PDO Indicator 1 measured the quality aspect of this objective, while the gender breakdown of PDO Indicator 2 measured the employability aspect. The employability of low-income students was not adequately measured; therefore, progress towards achievement of this objective was Substantial. 42. The Project successfully contributed to enhancing the inclusiveness of TVET. Stipends supported a total of 285,422 disadvantaged students in supported polytechnics (53 public and 109 private), against the revised target of 175,000. Stipends also supported 139,477 students of short-courses. The Project also succeeded in increasing female enrollment in short-courses from 5 percent at baseline to 32 percent at completion, exceeding the revised target of 30 percent. 43. Disadvantaged students performed well in the academic outcome, indicating improved training quality for them. Stipend recipients outperformed the average pass rates: 89.5 Page 19 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) percent for females and 92.5 percent for low-income students against an average of 86.5 for all students. 44. A study was carried out to evaluate the effectiveness of the stipends18. It was found that the Project had been successful in increasing enrollment and increasing transition rates. Application to public polytechnics became more competitive; the acceptance rate dropped from 37 percent in 2010 to 26 percent in 2013. Academic ability of applicants to public polytechnics witnessed steady improvement from 2010 to 2013 with a gradual increase in the proportion of applicants with an A+ grade. This trend was more pronounced for female applicants. More importantly, the study found that students receiving stipends on average outperformed those that did not by 0.04 grade point average points. 45. The Project expanded enrollment and improved the employability of female short-course trainees. The PDO Indicator that measured the employability of female students is the gender breakdown of PDO Indicator 2. The Project’s six-monthly monitoring reports monitored this indicator by gender. They showed that the Project exceeded expectations as the female employment rate for short course graduates reached 58 percent against the target of 46 percent by completion. 46. Short-courses under the Project expanded access to skills training opportunities for low- income students and produced good employment outcomes. A tracer study of short-term training graduates found that the Project was successful in attracting poorer students to the supported short-courses through stipends and free tuition19. It also found that the employment rate among Project-supported graduates was higher than that of non-supported graduates from the same institutions. One of the critical factors was job placement support. Although weak in most institutions, it was better catered and utilized by Project-supported students. Table 4: Summary of Objectives and Achievement of PDO Indicator Unpacked Objectives PDO Indicators EOP Targets Achieved 1. To improve training quality for Pass rates of students from 65 86.5 trainees supported Diploma programs (%) 2. To improve the employability of Employment rates for trainees of 46 51 trainees short-term courses within 6 months of course completion 3. To improve training quality and Number of disadvantaged 175,000 240,118 employability for trainees from a students supported through disadvantaged socio-economic stipends selected based on background household information based selection criteria 18Ahmed, Shaikh S., Stipend Effectiveness of the Skills and Training Education Project (STEP), July, 2015. 19World Bank, South Asia Region Education Global Practice, How Does the Short-Term Training Program Contribute to Skills Development in Bangladesh? A Tracer Study of the Short-Term Training Graduates, September 2015. Page 20 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Pass rates of students from 68 89.5 supported Diploma programs (%) for Female Pass rates of students from 60 92.5 supported Diploma programs (%) for Disadvantaged Employment rates for trainees of 46 58 short-term courses within 6 months of course completion for Female*1 Note: (*1) the results framework did not have gender disaggregation for this indicator; gender- disaggregated employment records were provided in the project’s six-monthly monitoring reports. Justification of Overall Efficacy Rating 47. Overall achievement of objectives was high based on the achievement of the PDO as measured by the PDO Indicators, Intermediate Indicators, study findings, M&E reports, and other evidence. Considering that the outcome of its Objective 3 was not fully measured by PDO Indicators, Overall Efficacy is rated Substantial. C. EFFICIENCY 48. This section presents the Project’s efficiency in both economic terms and also in terms of the efficiency with which it was implemented. Based on both of these, the Project’s efficiency is rated High. The updated economic analysis is summarized below. It is described in detail, together with the input parameters used for the analysis, in Annex 4. 49. Following the analysis prepared at Appraisal of the second AF, the economic impact of STEP is estimated as the increased lifetime labor earnings of the Project’s beneficiaries. The Project’s three significant benefits are generated from the following beneficiaries: (a) polytechnic graduates who would not have graduated in the absence of the Project’s institutional grants for improving the quality of training (Sub-component 1.1); (b) graduates of polytechnics who would not have graduated in the absence of stipends provided by the Project (Sub-component 1.1); and (c) the employed graduates of short-term programs (Sub-component 1.2). The PDV of expected economic costs comprises both direct costs (the Project’s cost and students’ out- of-pocket costs) and indirect costs (students’ forgone labor earnings) of completing the TVET diploma. 50. The ex-post cost-benefit analysis is evaluated using the outcome indicators (real pass rates and the number of polytechnic graduates and the short-term program graduates supported Page 21 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) by STEP over the project period). By the end of 2019, approximately 113,270 students 20 have or will have graduated from polytechnics supported by the STEP over the project period of the AF. Among them, around 41,600 polytechnic graduates have benefited from the stipend program. The pass rate (graduation rate) has improved from 64 percent in 2015 (the end of the original STEP project) to 86 percent in 2018. In this analysis, the impact of the IDG and the impact of stipend are estimated separately since the pass rates of stipend students (91% on average) are distinguishably higher than the overall pass rate (87% on average). Besides, it is reasonable to assume that the program impact of the stipend along with IDG on enrollment would be higher than of the IDG only. 51. Several scenarios were considered, varying both the enrollment increase of stipend beneficiaries and the wage premium for graduates of short-term programs. The lowest impact scenario (30% enrollment increase for the stipend beneficiaries and 10% wage premium for short-term programs) yields a Net Present Value (NPV) of US$495 million and an Internal Rate of Return (IRR) of 18.7%, and the highest impact scenario (50% enrollment increase for the stipend beneficiaries and 30% wage premium for short-term programs) yields NPV of US$576 million and an IRR of 19.2%. As a comparison, the re-evaluated NPV of the STEP is higher compared to the one at the Appraisal, mainly due to inclusion of benefits from the short-term programs and because the actual pass-rate of STEP-supported graduates (87%) is higher than assumed at Appraisal (high case of 70%). In summary, the Project’s results justify its investment. 52. The Project was also implemented efficiently. Supported by a strong PIU, the Project that counted upon two additional financings was implemented over nine years. At completion, the PIU was staffed by about 50 staff members responsible for coordinating project activities among over 250 training institutions and four sector institutions. There was an initial delay in the utilization of IDGs by polytechnics as it was a new concept in TVET and required a substantial capacity development of polytechnics for fiduciary management. However, the delay was fully recovered afterward as the stakeholders gained the capacity to implement IDG activities. The PIU also intervened directly by centrally procuring some of the common items on the institutions’ behalf. The total cost for project management amounted to US$11 million over the nine years or approximately 5.2 percent of project costs. The Closing Date was extended only to allow additional time to complete activities incorporated under the second AF. Upon completion, 100% of the funds under the original credit, the first and second AF were disbursed (see Financing section of Data Sheet for disbursement ratios). Assessment of Efficiency and Rating 53. Efficiency is rated High based on its higher than appraised estimated return, the reasonable costs of project management, and the high efficiency with which the Project was implemented, with original activities scaled up and new ones introduced through successive additional financings. 202016 and 2017 graduate numbers are 37,319 and 19,316, respectively. The 2018 and 2019 graduate numbers are estimated based on past two years graduate numbers. Page 22 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) D. JUSTIFICATION OF OVERALL OUTCOME RATING 54. Relevance is rated High, Efficacy is rated Substantial, and Efficiency is rated High, as described above. The Project’s objectives have been achieved at a high level exceeding all the PDO Indicator targets, substantiated by evidence from Intermediate Indicators, other studies, and M&E report, with a minor caveat that one objective could not be fully measured by PDO Indicators. Hence, the Project’s Overall Outcome Rating is Satisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 55. The enrollment of female students in TVET was particularly low at the time of Appraisal (5 percent). The Government’s 7th Five Year Plan explicitly encourages female enrollment in TVET and aims at increasing the females’ ratio in the overall TVET from 27 percent in 2015 to 40 percent by 202021. By the time of the MTR, little progress had been made in terms of expanding female enrollment in polytechnics and ensuring their retention. Only 12 percent of the Project’s stipend beneficiaries were female. Also, the high opportunity cost of studying for four years due to the availability of jobs in the RMG sector and other industries appeared to impact the high rate of drop-out. To help increase enrollment and reduce drop-out, the Project: (a) expanded the provision of stipends to all female students in eligible project-supported institutions; (b) prioritized removing the supply-side constraints by improving institutional environments and offering more female- favored training courses such as garments, computer, civil engineering and electronics; and (c) increased awareness-raising. As a result, the enrollment of females in TVET institutions increased to over 20 percent by completion. Women’s participation in the program helped enhance employment opportunities and promote their empowerment. Institutional Strengthening 56. In addition to the strengthening of TVET institutions, the Project had an essential role in supporting the development and enhancement of the institutional capacity of key government stakeholders responsible for TVET sector management, operation, and coordination. The main focus was on support to DTE and the BTEB under the MOE, the BMET under the MOEWOE, and also on the ISCs and the NSDC. The Project helped strengthen the institutional framework for TVET by financing: (a) strengthening of ISCs22 and establishment of the Dhaka Technical Teachers Training Institute for training of teachers of short-course programs; (b) improvements in physical and information system capacity at DTE, BTEB, and BMET; (c) improvements to the efficiency of existing systems; and (d) improvements to administrative and managerial capacity. A total of 635 managers and staff of DTE, BMET, and BTEB received managerial and leadership training, both nationally and internationally (against a target of 15 managers). 21 A study on female participation in skills development diagnosed the main causes of low female participation in TVET as: (a) supply constraints-lack of institutions nearby, non-conducive facility and faculty environment, and non-availability of female friendly courses; (b) reputational issues as the TVET sector was associated with the image of male domination and low- performers; (c) weak demand for training among potential female trainees due to lack of knowledge; and (d) household financial constraints. 22The ISCs are an important part of the system-level capacity to promote industry collaboration. As of December 2019, 12 ISCs have been established. Different development parnters are supporting ISCs. The Project aimed to support two ISCs – ICT ISC and Light Engineering ISC – through NSDC. Due to complications with coordination with NSDC, the Project could provide assistance to ICT ISC only. Page 23 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Mobilizing Private Sector Financing 57. The Project promoted public-private sector partnerships not only for financing but for technical inputs as well. Incorporating the private sector in TVET programs was very much at the heart of its planned quality improvements. To ensure the relevance of TVET to the needs of industry, and to expand the provision of quality TVET, the Project’s design contemplated strengthening and sustaining continued linkages with the private sector in most areas. It also benefited from private sector expertise and training delivery, although the exact cost of these contributions has not been calculated. First, the IMC of each participating institution counted upon significant representation from the private sector. These IMCs provided guidance on how the training institutions could adequately respond to the needs of the labor market and helped facilitate institutions’ greater involvement with the industry by establishing 930 industry partners. Second, to level the playing field between public and private institutions, a deliberate decision was made at Appraisal that both public and private institutions would be allowed to compete for resources under the Project. Private institutions were eligible to apply for competitively funded IDGs to implement quality improvement activities, and students in these institutions were eligible to receive stipends. In all, 19 private polytechnics received funding for their IDGs under the Project. Also, 28 private training institutions (out of a total of 80 institutions) delivered short-term programs were supported by the Project. The Project also piloted public and private partnerships for enterprise-based training by private sector organizations. The Bangladesh Garment Manufacturers and Exports Association (BGMEA) trained 14,040 unskilled laborers in five of its training centers, providing food and accommodation support to disadvantaged trainees from rural areas with subsidized costs. Poverty Reduction and Shared Prosperity 58. The Project aimed to improve access of TVET programs to marginalized communities through better targeting and more inclusive processes. The proxy means-tested (PMT) targeted stipend program was designed to ensure that persons from disadvantaged socio-economic backgrounds could participate in the TVET program, thus ensuring a policy of social inclusion. The Project provided stipends to 285,422 students in 162 polytechnics and 139,477 students in 71 short-term training programs. Of these, approximately 18 percent were female, while the others were from disadvantaged groups selected by PMT scores, or other qualifying conditions.23 Other Unintended Outcomes and Impacts 59. The Project has helped improve the image of technical education in Bangladesh, with the result that enrollments are increasing rapidly, the educational standing of incoming students in terms of their grade point averages is increasing, and acceptance rates at selected institutes, especially the public polytechnics, are decreasing. Further, the Project helped host a show-case for the promotion of small-scale local innovation by TVET teachers and students. Teachers who were trained in Singapore benefitted from the experience and pursued small-scale research at their polytechnics. Students’ innovations in TVET were promoted through the National Skill Competitions. Examples of students’ innovations include Central Nebulizer & Suction Machine, Wheel Spray Pump, Digital Chicken Farming System, etc. 23 This includes orphaned students, students from households with a divorced or widow headed family, or indigenous people. Page 24 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 60. The STEP was responsive to the Government’s policy priorities. The GOB viewed the skilling of its human resources as critical for maintaining its competitive edge in global markets, and its focuses on education and training were expected to have a significant impact on the country’s economic development. The Government’s New Education Policy (NEP) 2010 set as a goal to increase the share of students in the TVET system to 25 percent by expanding access to quality vocation education. The NEP recognized the challenges that the TVET system was facing in linking the supply of skills to those in demand by the workplace and seek to promote greater private sector participation in the provision and determination of skill needs. At a broader level, the Government’s priority was reflected in its Second National Strategy for Accelerated Poverty Reduction (NSAPR-II), a transitional poverty reduction strategy in place until the launch of its Sixth Five-Year Plan for FY11-15. All objectives were supported by the Project as an investment in quality tertiary and vocational education would be needed to produce better trained and educated workers required for improved productivity and higher- skilled jobs. 61. The Project built upon assistance provided by other development partners, most notably the European Commission, International Labor Organization (EC-ILO), and the Asian Development Bank (ADB). The EC-ILO Technical and Vocational Education and Training Reform Project was designed to support policy reforms of TVET, systems, and legislation, a new National Skills Development Policy (2011), National Technical and Vocational Qualifications, a new TVET Teacher Training Program, a new TVET Manager Training Program, Apprenticeship Guidelines and Guidelines for Recognition of Prior Learning. The ADB funded Skills Development Project, with similar objectives as the STEP, aimed to improve the relevance of and access to TVET programs and improve the capacity for quality TVET delivery. While there were no formal partnership arrangements, the STEP preparation team coordinated closely with teams of both the EC-ILO and ADB to ensure coordination in which the Project would build upon the policy reform efforts promoted by the EC-ILO project without duplication of efforts. This close coordination allowed the Project to focus the investment in institutional capacity building and expansion of training programs and maximize the return on investment. B. KEY FACTORS DURING IMPLEMENTATION 62. The economic growth and industry development scenario was consistently positive, creating a favorable environment for linkages with the private sector and demand for skills training. Bangladesh maintained its robust growth performance, providing an environment conducive to the achievement of the Project’s objectives. Bangladesh’s economy grew, on average, 6.5 percent per annum, reaching 7.9 percent in 2017/2018. Exports and remittances have been buoyant. This rapid growth enabled Bangladesh to nurture a range of emerging industries and increase demand for various technical skills. Emerging industries such as ceramic, ICT, tourism, and construction have all become very active players in skills Page 25 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) development and essential partners for the government and institutions to upgrade skills training programs. 63. The Government provided sustained commitment to the PDOs throughout implementation. Developing a quality skilled workforce to accelerate economic growth remained very much at the center of the Government’s policy priorities over the project implementation period, and the Government took several important policy decisions that reflect this. Reforms envisaged in the NSDP include: (a) a shift to a new national skills qualification system, the National Technical and Vocational Qualification Framework (NTVQF), and (b) introduction of a series of new modes of skills development, apprenticeship and workplace training, Recognition of Prior Learning (RPL), and enhancing skills development for overseas employment. Then, to provide more focused priority to TVET within the MOE, in November 2016, the Government reorganized the ministry, creating a separate division responsible for Technical and Madrasah education. Finally, in 2018 legislation was passed transforming the NSDC 24 to the National Skills Development Authority, with full legal status as the sole authority for overseeing technical and vocational education and training in Bangladesh. 64. The performance of the Project Implementation Unit was strong with steady leadership. The DTE, with the support of a Project Implementation Unit (PIU), was responsible for project implementation. The unit was responsible for all project fiduciary and safeguard processes for coordinating monitoring and evaluation processes and for periodic reporting, including coordinating fiduciary, safeguard, and monitoring processes among a vast number of participating institutions and providing stipends to their eligible students. Despite some initial delays in staffing critical positions in the PIU, its performance was reliable throughout implementation. Turnover was infrequent, and over the Project’s nine years of implementation, the PIU was headed by only three Project Directors, each of which remained in the position for at least three years. The Project’s implementation was well recorded and monitored, and its documentation provides a strong institutional memory. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 65. The Project assigned high importance to M&E. The Project's M&E was designed to monitor project achievements, effective use of resources, promote learning from implementation experience, and measure project impact. M&E was to be coordinated by the M&E Unit, within the PIU, which would collect performance monitoring data from each of the participating institutions based on a format designed for the Project that included the Project’s key performance indicators. The RF was concise, with regularly available data fo cused on those results that would be impacted by the Project’s activities. However, the RF had a minor gap in measuring progress towards the employability of socio-economically disadvantaged students (Section III, 24The NSDC was the apex authority on skills development in Bangladesh, responsible for coordinating the provision of skills development, ensuring that public institutions are best utilized to maximize effective use of resources, and implementing arrangements so that private providers can access public facilities to increase the scope of skills training. Page 26 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Objective III). The Project addressed this by providing gender-segregated data through its periodic monitoring reports. The M&E design also contemplated carrying out regular quality assurance surveys, annual performance audits, and evaluation studies. Also, the Project incorporated several activities aimed at improving the monitoring of training institutions, students, graduates through the development and implementation of management information systems in DTE, BTEB, and BMET (Section 1 A). M&E Implementation 66. The implementation of M&E was slow to begin. At the start, the absence of an integrated system of data management and analysis, delays in systematic reporting, and weak utilization of the M&E data for timely decision-making constrained the use of evidence for improving training quality and delivery under the Project. The PIU addressed these issues systematically through staffing of the M&E Unit, and the design, development, and implementation of a functional management information system for the Project. Other TVET sector institutions counted upon the PIU’s support to enhance their M&E capabilities, including the BTEB, BMET, and DTE, as well as participating in training institutions. Given the Project’s increased scope and its corresponding requirements, staffing of the MIS Unit was reinforced after the second AF. Reporting was provided regularly in a timely fashion, and the RF was continuously updated with detailed information collected from beneficiary institutions. It was updated during each of the AFs and restructurings to reflect progress in implementation and additional activities that would be carried out with the additional funding. Further, a large number of assessments, surveys, tracer, and other studies (some carried out by the World Bank as technical assistance) were carried out to evaluate progress. These included: (i) two round of graduate tracer studies on short- course graduates; (ii) two rounds of graduate tracer studies on polytechnic graduates; (iii) effectiveness assessment on IDG support for polytechnics; (iv) evaluation of SSC (Voc) pilot program; (v) effectiveness study on stipend program; (vi) RCT-based RPL impact evaluation study; (vii) ex-post RPL effectiveness assessment; (viii) analysis on challenges for post-secondary TVET for female; (ix) stocktaking study of ISCs; (x) review on NTVQF; and (xi) teacher training effectiveness assessment. M&E Utilization 67. Semi-annual progress reports were prepared in a timely fashion and submitted before each joint review mission. These reports helped the Government and World Bank implementation support teams identify progress and critical challenges and made the periodic support mission more effective. Project monitoring information and findings and recommendations of the assessments and surveys were used very effectively for introducing modifications in the Project’s design in the AFs and restructuring (Section I B). In addition, the findings of the more recent assessments have been valuable for the design of a follow-on operation, currently under preparation (Section IV D). Justification of Overall Rating of Quality of M&E 68. The Overall Quality of M&E is rated High. After addressing minor issues at the start of implementation, the Project’s monitoring was timely, updated regularly, and provided thorough information, including gender- wise breakdowns. The Project’s monitoring data, including that summarized in its RF, was used to guide implementation support. Together with the findings of the many surveys and evaluations carried out, the monitoring data helped feed into routine adjustments to the Project’s design. Page 27 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 69. Environmental Safeguards. The Project triggered OP/BP 4.01 (Environmental Assessment) and was categorized under environmental Category B since it could finance rehabilitation/renovation/refurbishment of the existing buildings, classrooms, workshops, and library and enhancing workshops and laboratory facilities. While the environmental impacts of any possible works, the physical intervention and detailed design of the infrastructure were not known at Appraisal. Therefore, and Environmental Management Framework (EMF) was prepared to incorporate the guidelines necessary for the preparation of an Environmental Management Plan (EMP) for each relevant activity after conducting environmental screening. Throughout the project, the M&E Unit of the PIU conducted regular monitoring of environmental compliance and reported to the findings. No significant issues were encountered. 70. Social Safeguards. The Project was expected to generate positive social impacts through its efforts to increase access of TVET programs to marginalized communities through better targeting and more inclusive processes. Nevertheless, this was a nation-wide program, and the exact locations of where activities would be financed had not been determined at Appraisal. Therefore, OP/BP 4.10 (Indigenous Peoples) was triggered if project activities would be implemented areas that housed indigenous peoples and other vulnerable communities. A framework approach was adopted, and a Social Management Framework (SMF) was prepared. Throughout the project implementation, the M&E Unit of the PIU conducted regular monitoring of social safeguard compliance. No significant issues were encountered. 71. Fiduciary compliance. At Appraisal, risks for both procurement and financial management were considered High. This was mostly because of the large number of dispersed training institutions that would be procuring goods and services, and handling the related financial management under the framework of their IDGs. The Project financed the contracting of both procurement and financial management specialists for the PIU, and the PIU, in turn, worked with training institutions to provide hands-on support and training. Other than minor delays with staffing and then actual contracting and recommendations for documentation requirements, neither procurement or financial management presented significant problems. Procurement plans updated regularly, and project audit reports were received in a timely fashion. For financial management, the Project was compliant in submitting the Interim Unaudited Financial Reports (IUFRs) and audit reports throughout the project implementation period, for both the Original and AF Credits. Except for a few minor exceptions on unreconciled balances of the utilization of funds between the designated account and client connection, the IUFRs were acceptable to the Bank. Also, to improve transparency and efficiency in procurement, the Project financed the introduction of e-Government procurement in DTE (and in the PIU), BTEB and BMET, and in polytechnics on a pilot basis. No significant issues were encountered. Page 28 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) C. BANK PERFORMANCE Quality at Entry 72. The World Bank’s performance in ensuring quality at entry is rated Satisfactory. The Project’s preparation was grounded in the recommendations of a policy study and subsequent discussions with the Government. The preparation team helped put in place a project that responded directly to the Government’s priorities for its TVET sector, and that was coordinated to complement assistance that was being provided by other donors (EC-ILO and ADB). The results framework was concise, with intermediate outcome indicators leading to the achievement of its PDO indicators. The team was focused on results, as reflected in the Project’s planned implementation of several assessments and surveys throughout implementation. Institutional arrangements were complicated, in part reflecting the institutional complexity of Bangladesh’s TVET sector. Still, the establishment and later training of a strong PIU with well-designed staffing arrangements and responsibilities within DTE were instrumental in ensuring smooth implementation and coordination among different training institutes and sectoral institutions. The Project was reasonable in terms of the activities it included and the objectives that it strived to achieve, within its given implementation period. Despite minor delays at the start of implementation, both the reasonableness of the Project’s scope and the ultimate strong performance of the PIU ensured its early progress and the continued robust implementation through each of its AFs. Quality of Supervision 73. The World Bank’s quality of supervision is rated Satisfactory. Throughout implementation, the World Bank’s implementation support team, led by Country Office-based staff, sustained a continuous dialogue with the PIU and other sector Institutions (BMET, BTEB) and training institutes. The team was pragmatic and results- focused and incorporated the findings of the many assessments and surveys effectively in the design of subsequent assistance under the AFs. The team also conducted periodic workshops with stakeholders of different project-supported programs to obtain feedback. Implementation progress, workshops’ and studies’ findings were used to introduce minor adjustments to project indicators and indicator targets and to feed into the preparation of a proposed follow-on project. The preparation and documentation of the AFs were timely, comprehensive, and responded to the Government’s emerging priorities (e.g., RPL) in addition to lessons of implementation experience. The team helped forge formal training partnerships with different countries to the benefit of managers and teachers. The implementation missions’ Aide Memoires were succinct and focused on both results and the next steps to advance implementation. The Project’s Implementation Status and Results Reports were timely, transparent, and results-focused and provided a strong basis for understanding implementation issues and progress towards the Project’s PDO. Justification of Overall Rating of Bank Performance 74. Overall World Bank performance is rated Satisfactory based on Satisfactory ratings for both quality at entry and quality of supervision as described above. D. RISK TO DEVELOPMENT OUTCOME 75. The Risk to Development Outcome is considered Modest. First and foremost, the Project’s objectives continue to count upon the Government’s highest priority. A well-functioning TVET sector, capable of delivering graduates with high-quality skills as required by the labor markets is fundamental for Bangladesh’s sustained growth in the future. A mechanism for targeting Page 29 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) individuals from less advantaged backgrounds and females is also a Government priority, supported by its policy framework, and essential for helping further reduce poverty and ensuring that the benefits of growth are shared among all Bangladeshis. The Government has already internalized some of the Project’s features, including a continuation of the stipend program with internal budget resources. The World Bank is assisting the Government in the preparation of a follow-on Accelerating and Strengthening Skills for Economic Transformation (ASSET) Project, for which proposed IDA financing is expected to be submitted for consideration by the Board of Directors in September 2020. Some challenges are anticipated, such as, among others, securing continued financing for the ever-changing training equipment requirements needed to maintain relevance to the constant introduction of new technologies. V. LESSONS AND RECOMMENDATIONS 76. Programs similar to Recognition of Prior Learning financed by the Project can have a positive impact on the many workers without formal education. First, these programs work well in an environment of excess demand for skilled labor. Still, there is a strong case for budget financing of these programs as industries have few, if any, incentives to improve the qualifications of workers that are already performing tasks required by their trade. On the other hand, these programs work as a very targeted one-time (and one-cost) social protection initiative that aims to increase the qualifications and pay for many workers that possess skills required but are not compensated accordingly due to lack of formal training. Finally, while obtaining a certification of qualification improves workers' confidence and standing vis-à-vis colleagues and their community, the impact on workers’ pay is not immediately apparent. Further, longer-term monitoring would be required to conclude the results. 77. Continued communication is fundamental to alter public opinion on improvements to TVET, and Skills Competition is a highly effective intervention to improve social awareness on TVET. Although the Project did make substantial progress in improving public perception of technical education, many people still have negative opinions on technical programs. The importance of a sustained effort to disseminate development and examples of success cannot be underestimated, especially when significant improvements have occurred to a previously under-performing system. The skills competition under the Project helped bring recognition to the TVET institutions and stimulated students to develop and document their innovations. This not only helps boost students’ confidence but serves to promote TVET and attract the participation of industries and help strengthen linkages. 78. The coordination of interventions and division of roles by separate donor-financed operations that support the common policy reform agenda is key to maximizing development impact for skills. The Project was an Investment Project Finance that supported the implementation of essential policy innovations in the TVET sector. It did not contain specific requirements or activities aimed at enacting different policies, but only supported their implementation. The devising, defining, and implementing of policy innovations was supported by financing from the EC-ILO. With close coordination among the Government, the EC-ILO and the World Bank, this arrangement worked very fluidly and produced outstanding results, without overburdening the World Bank’s investment loan. Page 30 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) 79. An innovative shared leadership structure of PIU allowed the Project to successfully involve multiple line ministries to implement well-aligned and coordinated skills interventions. To accommodate the need for working across skills training institutions and agencies under different ministries, the Project devised an innovative way for harmonization of operation by placing multiple agencies under one PIU, which had a Project Director (PD) from MOE and Deputy Project Directors from relevant agencies. One shared designated account was used to finance all the project activities with the approval authority of the PD. This arrangement enabled the Project to minimize the management costs and maximize coordination and synergy across operations for different agencies, including synchronization of training curriculum standards across DTE/BTEB and BMET. 80. Training institutions providing housing facilities need to create friendly environments to attract female students, and school leadership needs to support this. There are often legitimate concerns among both women and their families in sending their daughters off to reside away from home. Schools that have been successful in attracting women have had support from the top to help attract women students and provide them with an accommodating atmosphere away from home. This could include, for instance, close circuit cameras, separate dormitories where they report feeling safe, separate leisure rooms and washrooms, and friendly relations with teachers, where females are strongly represented. . Page 31 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improving training quality of vocational training Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Pass rates of students from Percentage 50.00 60.00 65.00 86.50 supported Diploma programs (%) 01-Sep-2009 30-Jun-2016 30-Jun-2019 30-Jun-2019 Female Percentage 66.00 68.00 68.00 89.50 30-Oct-2015 30-Jun-2019 30-Jun-2019 30-Jun-2019 Disadvantaged Percentage 55.00 60.00 60.00 92.50 30-Oct-2015 30-Jun-2019 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): National average:77.9% Page 32 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Objective/Outcome: Improving employability of trainees Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Employment rates for trainees Percentage 30.00 44.00 46.00 51.00 of short-term courses within 6 months of course completion 01-Sep-2009 30-Jun-2016 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Objective/Outcome: Improving access to training for those from disadvantaged socio-economic backgrounds Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of disadvantaged Number 0.00 160000.00 175000.00 240118.00 students supported through stipends selected based on 01-Sep-2009 30-Jun-2016 30-Jun-2019 30-Jun-2019 household information based selection criteria Comments (achievements against targets): Page 33 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) A.2 Intermediate Results Indicators Component: Component 1: Improve the Quality and Relevance of Training Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct project beneficiaries Number 240000.00 420000.00 706847.00 19-Aug-2015 30-Jun-2019 30-Jun-2019 Female beneficiaries Percentage 14.00 17.00 20.70 Comments (achievements against targets): A new indicator introduced at the Additional Financing in 2015 Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion % of utilization of Institutional Percentage 0.00 95.00 96.60 Development Grants under AF 30-Jun-2015 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Page 34 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 1.1: Enrollment as Percentage 55.00 65.00 95.00 99.00 a proportion of sanctioned intake capacity in supported 25-Jan-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 courses Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 1.1: Share of Percentage 25.00 10.00 5.00 3.00 vacancies in sanctioned faculty posts in supported courses 25-Mar-2010 30-Jun-2016 30-Jun-2019 28-Sep-2018 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 1.1: Transition Percentage 80.00 90.00 95.00 98.00 rates of students from second to third semester in supported 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 courses Page 35 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Comments (achievements against targets): This indicator intends to measure retention of students in polytechnics. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of faculty in supported Number 1500.00 6000.00 7980.00 courses trained 15-Apr-2015 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): A new indicator introduced at the Additional Financing in 2015 Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 1.2: completion Percentage 50.00 70.00 95.00 96.00 rates for trainees in supported short-term training courses 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Indicator Name Unit of Baseline Original Target Formally Revised Actual Achieved at Page 36 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Measure Target Completion Share of girl trainees in Percentage 5.00 20.00 30.00 32.00 supported short-term training courses 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Component: Component 2: Pilots in TVET Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 2.1 Number of Number 0.00 2.00 7.00 6.00 Industry Skills Councils functioning supported by the 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 project Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 2.2: Evaluation of Text 0 Evaluation Completed Evaluation Completed Evaluation Completed SSC (Voc) 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Page 37 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of individuals assessed Number 1900.00 30000.00 31804.00 for RPL in supported centers 15-Apr-2015 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Component: Component 3: Institutional Capacity Development Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of staff trained in Number 22.00 150.00 635.00 Leadership and Management areas. 15-Apr-2015 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Another intermediate results indicator for Component 3 "Establishment of Central Job and Information Portal in DTE" was achieved and subsequently removed from the results framework. Component: Component 4: Project Management, Communications, and Monitoring and Evaluation Page 38 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 4.1: Awareness Number 0.00 12.00 30.00 36.00 raising campaigns 25-Mar-2010 30-Jun-2016 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Satisfaction Survey (Student, Text Nil Survey Completed Survey Completed Faculty, and Employers’) for IDG recipient institutions 15-Apr-2015 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Component 4.2: Assessment Text Nil 2nd tracer study on 2nd tracer study on Studies polytechnic polytechnic completed; RPL completed; RPL impact evaluation impact evaluation completed completed 15-Apr-2015 30-Jun-2019 30-Jun-2019 Page 39 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Comments (achievements against targets): Page 40 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 : Strengthen training institutions to improve quality Outcome Indicators 1. Pass rates of students from supported Diploma programs 1. % of utilization of Institutional Development Grants under the Additional Financing 3. Share of vacancies in sanctioned faculty posts in supported Intermediate Results Indicators polytechnics (regular and contractual) 4. Transition rates of students from second to third semester in supported courses 5. Number of faculty in supported courses trained 6. Number of staff trained in Leadership and Management areas. 1. Improved teaching and learning environment in polytechnics and short-course training institutions Key Outputs by Component 2. Improved teaching skills and subject knowledge of teachers (linked to the achievement of the Objective/Outcome 1) 3. Improved teacher deployment at institutions 4. improved teaching and learning materials Objective/Outcome 2 : Strengthen training institutions to improve the employability of trainees 1. Employment rates for trainees of short-term courses within 6 Outcome Indicators months of course completion 1. Completion rates for trainees in supported short-term training courses Intermediate Results Indicators 2. Number of Industry Skills Councils functioning supported by the project 3. Evaluation of the SSC (Voc) Pilot Page 41 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) 4. Satisfaction Survey (Student, Faculty, and Employers’) for IDG recipient institutions 5. Assessment studies in TVET 1. Enhanced curriculum and practical skills training in line with industry needs 2. Enhanced industry participation in institutional management of Key Outputs by Component training institutions (linked to the achievement of the Objective/Outcome 2) 3. Enhanced industry attachment and apprenticeship program 4. Enhanced job placement services for trainees 5. Enhanced employer feedback and graduate tracking processes Objective/Outcome 3 : Improve quality of training and employability for trainees from a disadvantaged socio-economic background 1. Number of disadvantaged students supported through stipends selected based on household information based selection criteria 2. Pass rates of students from supported Diploma programs for Outcome Indicators females and disadvantaged 3. Employment rates for trainees of short-term courses within 6 months of course completion for female 1. Enrollment as a proportion of sanctioned intake capacity in supported courses (%) Intermediate Results Indicators 2. Share of girl trainees in supported courses % 3. Number of individuals assessed for RPL in supported centers. 4. Number of awareness raising campaigns 1. Increased enrollment of female students in training institutions 2. Increased enrollment of low-income students in training institutions Key Outputs by Component 3. Increased access to RPL assessment for informal sector workers (linked to the achievement of the Objective/Outcome 2) 4. Increased awareness among vulnerable communities about skills development opportunities Page 42 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Venkatesh Sundararaman, Nalin Jena Task Team Leader(s) Pema Lhazom Senior Operations Officer Mokhlesur Rahman ET Consultant Subrata S. Dhar Senior Education Specialist Sangeeta Goyal Education Economist Syed Rashed Al-Zayed Economist Rajendra Dhoj Joshi Senior Education Specialist Marta E. Molares-Halberg Lead Counsel Burhanuddin Ahmed Sr. Financial Management Specialist Marghoob Bin Hussein Procurement Specialist Muhammed Riadul Islam Procurement Analyst Shakil Ahmed Ferdausi Environmental Specialist Fabio Pittaluga Senior Social Development Specialist Sabah Moyeen ET Consultant Bertha Mburugu Program Assistant Nazma Sultana Program Assistant Tanusree Talukdar Program Assistant Sushila Rai Team Assistant Supervision/ICR Mokhlesur Rahman, Shiro Nakata Task Team Leaders at ICR Amit Dar, Venkatesh Sundararaman, Nalin Jena, Shinsaku Former Task Team Leaders Nomura Ishtiak Siddique Senior Procurement Specialist Page 43 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Zafrul Islam Senior Procurement Specialist Marghoob Bin Hussein Senior Procurement Specialist Mohammad Reaz Uddin Chowdhury Senior Financial Management Specialist Mohammed Atikuzzaman Senior Financial Management Specialist Arvind Prasad Mantha Senior Financial Management Specialist Syed Rashed Al-Zayed Senior Economist Tashmina Rahman Education Specialist Subrata S. Dhar Senior Education Specialist Sabah Moyeen Senior Social Development Specialist Fabio Pittaluga Senior Social Development Specialist Shakil Ahmed Ferdausi Senior Social Development Specialist Nadia Sharmin Senior Environmental Specialist Karthika Radhakrishnan Operations Officer Mahtab Alam Program Assistant Nazma Sultana Program Assistant Shourov Kumar Sharma Team Assistant Aftab Uddin Ahmad Consultant Mustahsin-Ul Aziz Consultant Ahamad Tanvirul Alam Chowdhury Consultant Syeda Kashfee Ahmed Consultant Suzana Nagele de Campos Abbott ICR Main Author B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY06 1.200 5,337.45 Page 44 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) FY07 5.200 26,723.73 FY08 6.542 38,839.70 FY09 54.293 131,598.73 FY10 68.233 252,400.78 FY11 4.960 21,793.71 Total 140.43 476,694.10 Supervision/ICR FY09 .325 1,466.44 FY10 .670 2,683.75 FY11 40.151 153,292.43 FY12 52.083 125,911.67 FY13 31.830 134,479.84 FY14 56.520 483,076.27 FY15 69.599 481,553.89 FY16 67.892 560,028.90 FY17 68.925 636,482.39 FY18 41.840 315,765.41 FY19 81.111 390,884.16 FY20 15.880 143,999.32 Total 526.83 3,429,624.47 Page 45 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Improve the Quality and 162.2 155.65 95.96 Relevance of Training Pilots in TVET 7.8 6.17 79.10 Institutional Capacity 40.8 34.75 85.17 Development Project Management, Communications, and 14.7 13.00 88.44 Monitoring and Evaluation Total 225.50 209.5725 92.94 25 Difference with the total disbursed amount in the data sheet is due to exchange rate differences. Page 46 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 4. EFFICIENCY ANALYSIS The economic impact of STEP is estimated as the increased lifetime labor earnings of the graduates affected by the STEP. This increased lifetime labor earnings are estimated using the present discounted value (PDV) method. Three major benefits of STEP are generated from the following beneficiaries: (i) polytechnic graduates who wouldn’t have graduated in the absence of the STEP’s institutional grants for improving the quality of training (Sub-component 1.1); (ii) polytechnic graduates who wouldn’t have graduated in the absence of stipends provided by STEP (Sub-component 1.1); and (iii) the employed graduates of short-term programs (Sub-component 1.3). The Present Discounted Value (PDV) of expected economic costs comprised both direct (project cost of STEP and students’ out-of-pocket cost) and indirect (forgone labor earnings of the students) costs of completing the TVET diploma. Economic analysis at the appraisal The cost-benefit analysis of the STEP AF at the appraisal focused on the largest activity, Component 1.1, including the institutional development grant (IDG) and stipend support to polytechnics. Benefits: Two main channels of benefits stream are quantified: (i) increased number of polytechnic graduates as a result of increased enrollment and increased pass rates, and (ii) increased wage premiums as a result of a better quality of training. Costs: Given that the focus was on the Sub-component 1.1, the costs include the direct project cost of Sub-component 1.1 ($US47 million), out-of-pocket expenses per polytechnic students, and indirect costs of foregone earnings due to the education. Table 1. Basic Assumptions and Input Parameters Values Description used Note Sustained benefits of the institutional Assumption - The benefits continue 10 grant 10 years years after the completion. The annual growth rate of polytechnic enrollment 3% Assumption Intake ratio with STEP (%) 95 Result indicator Intake ratio without STEP (%) 70 Result indicator Pass rate with STEP (%) 65 Result indicator Pass rate without STEP (%) 55 Result indicator Wage increase due to quality improvement (%) 5 Active working years (years) 40 From age 20 to 60 Average start salary of TVET graduates ESS 2010; adjusted for the inflation from (BDT) 10,077 2010 to 2015 Average start salary of SSC graduates ESS 2010; adjusted for the inflation from (BDT) 7,687 2010 to 2015 Page 47 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Employment rate of TVET graduate to be 7% lower than that of SSC graduates (LFS Employment rate for TVET graduates (%) 95 2010) Employment rate for SSC graduates (%) 88 LFS 2010 Net annual out-of-pocket expense (the average total expense less the amount of stipend) (BDT) 3,690 BDT 13,290 - BDT800*12 months (HIES 2010) The average of the past 20 years from 1994 Real interest rate (%) 8.1 to 2013 (World development indicators) Exchange rate (BDT/US$) 77.95 As of Jan 1, 2015 (Bangladesh Central Bank) Result: The cost-benefit analysis using 9 different scenarios with the rates of graduation (pass rates) and the wage premium due to quality improvement was conducted. The scenario (65 percent pass rate and 3 percent wage premium) yields a net present value of US$314 million with an internal rate of return of 12 per cent. NPV ranges between US$137 million and US$620 depending on the scenario. Economic Re-evaluation The cost-benefit analysis is evaluated using the outcome indicators (real pass rates and the number of polytechnic graduates and the short-term program graduates supported by STEP over the project period). By the end of 2019, approximately 113,270 students26 have or will be graduated from polytechnics supported by the STEP over the project period of AF. Among them, approximately 41,600 polytechnic graduates have benefited from the stipend program. The pass rate (graduation rate) has improved from 64 percent in 2015 (the end of the original STEP project) to 86 percent in 2018. In this analysis, the impact of the IDG and the impact of stipend are estimated separately since the pass rates of stipend students (91 percent, on average) are distinguishably higher than the overall pass rate (87 percent, on average). In addition, it is reasonable to assume that the program impact of the stipend along with institutional grants on enrollment would be higher than the one of the grants only. In order to calculate the net impact of the IDG and the stipend, the counterfactual (the number of graduates in the absence of STEP) is estimated using the pass rate at the national level. Other input parameters that are used for the economic benefit of these sub-components are the following. Table 2. Updated input parameters Input Parameters Value Note Assuming that the impact will be linearly Sustained impact on pass rate 4 years depreciated over 4 years after the completion. Intake ratio with STEP (%) 95 Value at appraisal Intake ratio without STEP (%) 70 Value at appraisal Enrollment increase (%) with 30% ~ 50% Scenario base, empirically it is shown as high 262016 and 2017 graduate numbers are 37,319 and 19,316, respectively. The 2018 and 2019 graduate numbers are estimated based on past two years graduate numbers. Page 48 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) STEP stipend as 65% Pass rate with STEP (%), no 87 Updated according to the result indicator stipend Pass rate with STEP (%), stipend 91 Updated according to the result indicator Pass rate without STEP (%) 78 Pass rate at the national level Wage Increase due to quality improvement (%) – TVET diploma 5 Base scenario value at the appraisal Value at appraisal, adjusted using an annual Monthly Average Salary of TVET wage increase of 3 % (LFS 2015-16 and 2017- Graduate (unemployment 18) from 2016 to 2019, an unemployment adjusted), BDT 10492 rate of 95% Value at appraisal, adjusted using an annual Monthly Average Salary of SSC wage increase of 3 % (LFS 2015-16 and 2017- Graduate (unemployment rate 18) from 2016 to 2019, an unemployment adjusted), BDT 7414 rate of 88% Scenario base. Within 2 years, the wage increase for STEP supported short term programs graduate ranges between 27 and 34 %, in the meantime, the wage increase of non-STEP Wage premium due to quality short-term program graduate is only 9% improvement (%) – Short term (Nomura et al. 2014). programs 10~30 Assuming it lasts 5 years after completion. The difference in employment rates between the STEP supported and non-supported Impact of STEP short course on graduates is between 6% and 10% (Nomura employment 8 et al.) Real interest rate (%) 4 Average from 2015 to 2018 Exchange rate (BDT/US$) 84.4 September, 2019 i) Economic benefit of IDG Over the project period, the pass rate of students from the STEP-supported institutes was 9 percent higher than the national pass rate of polytechnic students. Along with the assumption of 25 percent higher enrollment due to the STEP, it is estimated that the STEP generated 11.5 percent more graduates over the project period. The economic benefit of the IDG is estimated by the increased earnings of these graduates (11.5 percent) over their economically active years (40 years). In addition, it is assumed that the impact of IDG on the pass rate will be depreciated over 4 years after project completion. Besides its impact on enrollment and pass rate, it is assumed that the quality improvement of institute results in a 5 percent increase in average wages of TVET graduates. In this analysis, the economic benefit of the stipend is separated from the overall impact of grants in the analysis. The present value of the economic benefit of institutional grants is estimated to be US$161 million. Page 49 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ii) Economic benefit of stipend Over the project period, the pass rate of the stipend beneficiary students was 13 percent higher than the national pass rate of polytechnic students. In the case of the stipend, the impact on the enrollment (take-up) would be higher than the case of IDG. Empirically, the impact of the voucher (financial support) in vocational training is shown to be as high as 65 percent (Vocational Education in Kenya - a Randomized Evaluation, 3rd Working Paper, 2015). However, since the STEP supported only the partial cost of students’ living expenses, its impact on the enrollment would be lower. In this analysis, three different scenarios of 30 percent, 40 percent and 50 percent increase in enrollment are considered. Combined with its impact on the pass rate, these scenario cases are translated into 47 percent, 58 percent, and 70 percent increases in the number of graduates resulting from the stipend. The economic benefit of the stipend is estimated by increased earnings of these affected stipend graduates over their economically active years (40 years). All other assumptions and input parameters are the same as the case of IDG. The present value of the economic benefit of the stipend is estimated to range between US$155 million and US$231 million. iii) Economic benefit of short-term programs A total of 35,106 graduates (45%) of short-term programs are identified as currently employed. Given that the short-term programs are targeted to improve the competency of industry-demanded skills, the economic benefit of these programs (wage premium) could be sustained in the short- to medium- term. In addition to the wage premium, an increase in the employment rate can be counted as a result of the STEP. According to the tracer study (Nomura et al. 2014), the difference in employment rates between the STEP-supported and non-supported graduates ranges between 6 percent and 10 percent. Under the assumption of wage premium of 10 percent to 30 percent for 5 years after project completion and with the life-long earnings of those graduates who would not have found employment without the STEP supported short-term program, the present values are estimated to be ranging between US$187 (10 percent wage premium) and US$232 million (30 percent wage premium). Table 3. Summary - Total benefit of each component (Present Value,2019 US$) Institutional grant 161,278,962 Stipend - 30% enrollment increase 155,814,770 Stipend- 40% enrollment increase 193,496,878 Stipend- 50% enrollment increase 231,065,859 Short term course -10% wage premium 187,685,913 Short term course -20% wage premium 210,119,197 Short term course -30% wage premium 232,552,481 iv) Economic cost of STEP The cost includes present values of total project expenditures over the project period, and direct (total out of pocket students’ expenditure over 4 years) and indirect costs (forgone earning during schooling years). Page 50 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) v) Results The lowest impact scenario (30 percent enrollment increase for the stipend beneficiaries and 10 percent wage premium for short-term programs) yields an NPV of US$495 million and an IRR of 18.7 percent, and the highest impact scenario (50 percent enrollment increase for the stipend beneficiaries and 30 percent wage premium for short-term programs) yields an NPV of US$576 million and an IRR of 19.2 percent. The re-evaluated NPV of the STEP is higher compared to the one at the appraisal, mainly due to the inclusion of benefits from the short-term programs and the actual pass-rate of STEP supported graduates (87 percent) being higher than assumption used at appraisal (high case of 70 percent). In summary, the results justify the investment made through STEP project. Table 4. Summary–Cost Benefit Analysis (NPV in millions 2019 US$; IRR in %)27 Assumptions: increase in enrollment rate resulting from the stipend 30% 40% 50% NPV IRR NPV IRR NPV IRR Wage premium for short term program: 10% 495 18.7 513 17.6 531 16.7 Wage premium for short term program: 20% 518 20.3 536 19.0 553 17.9 Wage premium for short term program: 30% 540 22.0 558 20.5 576 19.2 27The NPV is higher for the higher increase in enrollment rate scenario while the IRR is lower. This is due to cost stream is different across the scenario - faster increase in first 4 years of opportunity cost due to higher enrollment. Page 51 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Page 52 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) Page 53 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) World Bank, Country Assistance Strategy for the People’s Republic of Bangladesh for the Period FY11-14, Report No. 54615-BD dated July 30, 2010. World Bank, Country Partnership Framework for Bangladesh for the Period FY16-FY20, Report No. 103723-BD dated March 8, 2016. World Bank, Project Appraisal Document, Skills and Training Enhancement Project, Report No. 53549-BD dated May 11, 2010. World Bank, Employability of Post-Secondary TVET in Bangladesh, Tracking Survey of Graduates of Polytechnics, Report No. AUS0000063, dated January 2, 2019. World Bank, How Does the Short-Term Training Program Contribute to Skills Development in Bangladesh? A Tracer Study of the Short-Term Training Graduates, Report No. 82 dated September 2015, and Center for Research and Management Counseling, SRG Bangladesh Limited, Bangladesh: Skill and Training Enhancement Project (STEP). World Bank, Bangladesh, Vocational Training Project, P009393, Staff Appraisal Report No. 2316a-BD, dated May 3, 1979 and World Bank, Bangladesh, Technical Education Project, P009440, Staff Appraisal Report No. 4852-BD, dated April 25, 1984. World Bank, Learning for Job Opportunities: An Assessment of Vocational Education and Training in Bangladesh, Bangladesh Development Series Paper No 19, Report No. 45189 dated June 2007. World Bank, Breaking the Glass Ceiling: Challenges to Female Participation in Technical Diploma Education in Bangladesh, Report No. AUS0000002 dated January 2017. World Bank, Employability of Post-Secondary TVET in Bangladesh: Tracking Survey of Graduates of Polytechnics, Report No. AUS0000063 dated January 2018. World Bank, Implementation Status and Results Reports and Mission Aide-Memoires. Skills and Training Enhancement Project (STEP) Project Completion Report, dated June 2019. Tracer Study of Graduates of Short Course Training Programs (2nd Round), dated May 24, 2018. Report on Recognition of Prior Learning (RPL) Assessment, dated August 2016. Evaluation of SSC (Voc) Apprenticeship Piloting Program dated July, 2018. Ahmed, Shaikh S., Stipend Effectiveness of the Skills and Training Education Project (STEP), July, 2015. Assessment of Stipend Program of the Skills and Training Enhancement Project, April 2019. Page 54 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) ANNEX 7. SUMMARY OF BORROWER ICR PROJECT COMPLETION REPORT : IMED 04/2003 (Revised) A. PROJECT DESCRIPTION : 01. Name of the Project : Skills and Training Enhancement Project (STEP). 02. Administrative Ministry/Division : Technical and Madrasah Education Division 81. under Ministry of Education. 03. Executing Agency : Directorate of Technical Education (DTE). 04. Location of the Project : The geographical coverage of the project is Country 82. wide. 05. Objective of the Project : The project development objective is to strengthen selected public and private training institutions to improve training quality, and employability of trainees, including those from disadvantaged socio-economic backgrounds. The specific objectives of the project are to: i) To enhance quality and relevance of technical and vocational education and training as a part of meeting the strategic options of the poverty reduction agenda of the government; ii) To strengthen the overall Technical and Vocational Education and Training (TVET) system through (a) direct start-up and operational support to Industry Skills Councils (ISC) and the National Skills Development Council and (b) support to SSC (Vocational) schools; iii) To strengthen the capacity of key institutions (DTE, BTEB and BMET) which play important role in the management and quality assurance of TVET sector; and iv) To establish a project management and implementation structure, implement communication strategy and undertake monitoring and evaluation. It is expected that on successful implementation, the project will result in with improving the quality; relevance and efficiency of the TVET education and training in Bangladesh with increased employability of the graduates coming out of the system which will ultimately help reduce poverty and improve the quality of life of the general mass in particular. D. ACHIEVEMENT OF OBJECTIVES OF THE PROJECT: Objectives as per PP Actual achievement i) To enhance quality -Enrollment has been increased from 55% to 99% as a proportion of and relevance of technical and sanction intake capacity vocational education and training as a part of meeting 83. -Provide stipend to 711,000 student years for all female the strategic options of the student and disadvantage male students poverty reduction agenda of - Percentage of pass rate achieving from 50% to 86.46% the government; 84. -75% attendance must be needed for stipend 85. - Transition rate of students from second to third semester increased from 80% to 98% Page 55 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) 86. -BDT 33994.11 lakh will be disbursed to 56 polytechnic institutes for strengthened with modern equipment, faculty training, industry visit etc.- 87. - Competition between public and private TVET institutes has been established 88. -trained 139477 unemployed youth through short term training with free of cost 89. - completion rate has been increased from 50% to 96% regarding short term training (6 months/360 hours) 90. -Share of girls trainees will be increased from 5% to 30% regarding short term training (6 months/360 hours) 91. -Employment rate has been increased from 30% to 49% regarding short term training (6 months/360 hours) - Trained 138600 unskilled and semi-skilled labor through short term training (1 month) with free of cost and employment rate was 73.16% ii) To strengthen the -Project Provided seed financing and operational cost to overall Technical and NSDC and provided some equipment to the ISCs through Vocational Education and NSDC Training (TVET) system - Project provided support to strengthening and through (a) direct start-up and established of ICT ISC operational support to -Strengthening Secondary School Certificate (vocational) Industry Skills Councils (ISC) and the National Skills 92. - Apprenticeships piloting for SSC (voc) implemented in 20 Development Council and (b) institutes to 9125 students support to SSC (Vocational) 93. Schools, iii) To strengthen the 95. capacity of key institutions 96. -Utilized BDT 26370.06 lakh for development of the key (DTE, BTEB and BMET) TVET organizations ( DTE, BTEB, & BMET) which play important role in 97. -Web-based MIS developed for DTE, BETB and BMET the management and quality 98. -Provide contractual teacher (Full teaching strength) to the assurance of TVET sector; public institutes and reduced vacant post from 71% to 3% and 99. - xxxxx Numbers of faculty trained through local training 94. 100. -xxxx Numbers of staff trained in core competencies in DTE, BTEB and BMET through partnership with different renowned institutes in the world 101. -xxxxx candidate certified under RPL system - Project established a technical teachers training Institute at BK-TTC for enhanced training capacity of short term training provider - Provided support for vertical expansion of 1 story of BMET & 4- storied of DTE building 102. - iv) To establish a project Project staff and contractual teacher appointed by the management and Project Page 56 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) implementation structure, -Consultants are selected according to PPR 2008 implement communication Project provide stipend to the selected student strategy and undertake monitoring and evaluation. -Project provide institutional grants to the selected institutes -Provide training to teachers, other officials & staff of key TVET institutes -Capacity of DTE, BTEB BMET and NSDC strengthen - Skill Competition (i. Inaugural ii. Institutional, iii. Regional and iv. National ) held 4 times since 2014 (each time around 5500 student participated). --Procurement done in accordance with GOB/World Bank guidelines - Mass people aware about TVET & STEP - Digital contents and contents management system developed and distribution - Strengthen M&E activities 4.4 Impact of the project- This is a social sector project where benefits of projects cannot be quantified in the real sense of the term. Quantification of benefits in financial terms is not only a complex exercise, its outcome is also not easily understood. It does not give any clear picture of the benefits the project may accrue. However, project try to articulate the direct and indirect impact of the project. 4.4.1 Direct: Direct impact of the project interventions will be measured through an impact study at a later stage. However, to measure immediate impact or result achieved the Semi-Annual Monitoring Reports and various study reports are relevant. Since inception significant progress was made by the project based on different key indicators. The direct impacts are given below as per the Revised Project Result Framework: • In base year pass rates of students from supported diploma programs were 50% (Overall). In the year 2018 overall target was 65%; female 68% and disadvantage (Stipend students) were 60% as of 2018 actual achievement were overall 86.46%; female 89.48% and disadvantage (Stipend students)92.52%. Whereas, In 2018 National average pass rate was 77.92%. • Employment Rates for trainees of short-term courses within 6 months of course completion was estimated at 30% percent in baseline, the achievement was 49% in 2018, while the project end target was 46%; • Number of disadvantaged students supported through stipends selected based on household information based selection criteria was 240118 compared to end target was 175,000 Page 57 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) • The targeted total number of beneficiaries and the share of female were estimated to 420,000 and 17% respectively in YR7 (2018). The achievement was 665,180 in 2018, of which the share of female students was 20.29%. • Enrollment has been increased from 55% to 99% as a proportion of sanction intake capacity • Transition rate of students from second to third semester increased from 80% to 98% • Competition between public and private TVET institutes has been established • In base year the Employment Rates for trainees of short-term courses within 6 months of course completion was 30%. As of 2018 actual achievement was 49% compared to target 46%; • project providing 1100 contractual teacher to the 50 (49 of DTE and 1 for BMET) polytechnics institutes offering diploma level training programs to ensure full teaching strength by reducing the vacant posts from 71% to 3%. • completion rate has been increased from 50% to 96% regarding short term training (6 months/360 hours) • Share of girls trainees will be increased from 5% to 30% regarding short term training (6 months/360 hours) • trained 139477 unskilled and semi-skilled labor through short term training (1 month) with free of cost and employment rate was 73.16% • Since 2014 project organized national skills Competition to showcase and promote TVET. The event created a much-needed promotion of TVET and awareness-building among stakeholders and general public. • STEP provided web based MIS including necessary hardware, software and network for key TVET organizations • STEP provide workplace experience supports quality training and employment prospects for 9125 SSC (Voc) students. • Since 2014 STEP Organized national skills Competition to showcase and promote TVET • Project create a pool of NTVQF certified assessor up to Level IV for strengthened the BTEB and the overall TVET sector’s capacity as a whole to deliver skills training, assessment and certification under the NTVQF framework. • STEP conducted RPL assessment through 20 institutes. The RPL program has been a huge success with over 31,840 candidates assessed, 24,620 certified competent and an overall achievement rate of around 77 percent. • The project supported the establishment of Institutional Management Committees (IMC) in all STEP- supported polytechnics and short course training providers to review the progress of the training provider and advise on ensuring the relevance of the training to local industry demands. Page 58 of 60 The World Bank Bangladesh - Skills and Training Enhancement Project (P090807) • The project has supported the establishment of employment cells in all beneficiary polytechnics and short course for expanding access to employment • STEP evaluated the RPL assessment and found that RPL assessment creates the opportunities of skills workers to participate in higher level vocation and technical training and education as well as upscale the work force. Considering the comparison before and after RPL assessment for nature and type of job of competent and partially competent assesses as well as control respondents, the significant increase was observed for both competent and partially competent assess in respect of full-time and part time wage employment in public and private entity. From the chi squared value it was revealed that there was significant difference found between before and after RPL assessment in terms of responses on the job title/position. Most of the competent (77%) and partially competent (91%) assesses reported that they were benefitted through social recognition. Majority of the competent (63%) assesses reported that the real benefit was the increase of opportunity for higher level of RPL and 73% partially competent assesses reported that real benefit was recognition by colleagues and friends. The majority 42% competent and 43% partially competent assesses were interested to participate further in RPL assessment without financial assistance. • We evaluated the SSC (VoC) apprenticeship program and found that, out of 700 treatment group respondents, 85.9% (593) students reported that the apprenticeship attachment training increased their skills in expertise; 80.1% (553) students stated that industry training increased their self confidence; 39.3% (271) students reported that they achieved the BTEB/NTVQF Competency Certificate through assessment; 19.7% (136) students reported that the apprenticeship pilot program created opportunity to get better job; 12.0% (83) students reported that theindustry training increased the opportunity for higher education. In terms of benefits of BTEB/NTVQF Competency Certificate, out of 435 students who achieved competency certificates, 65.2% of them (283) reported that certificate created the opportunity to get better job; 41.2% (179) students stated it increased social image; 40.1% (174) students said it created opportunity to get higher income in the job; 31.6% (137) students opined that it created opportunity for higher study; and 8.5% (37) students reported that the self-satisfaction was achieved through apprenticeship assessment. Among 435 treatment group students who reported about the benefits of certificate, 91.8% (393) students highly confident on skills achieved through BTEB/NTVQF assessment and certification program; while 8.2% (32) students were confident. • Benefit Cost Ratio (1:10.33) of apprenticeship pilot program evidently revealed that the return against investment was very high. This return will be continued for the STEP supported graduate throughout his/her working life and that will be gradually increased through his/her life. Page 59 of 60