Document of The World Bank Report No: ICR2874 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-43000) ON A CREDIT IN THE AMOUNT OF SDR 2.4 MILLION (US$3.5 MILLION EQUIVALENT) TO THE ST. LUCIA FOR THE OECS SKILLS FOR INCLUSIVE GROWTH PROJECT IN SUPPORT OF THE FIRST PHASE OF THE OECS SKILLS FOR INCLUSIVE GROWTH PROGRAM March 14, 2014 Human Development Sector Caribbean Country Management Unit Latin American and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 11, 2014) Currency Unit = Eastern Caribbean Dollar EC$2.7 = US$1.00 US$0.37 = EC$1.00 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AAA Analytical and Advisory Activities AHLEI American Hotel and Lodging Education Institute APL Adaptable Program Loan CANTA Caribbean Association of National Training Agencies CARICOM Caribbean Community CAS Country Assistance Strategy CIIHM Caribbean Institute of International Hospitality Management CSME Caribbean Single Market and Economy CVQ Caribbean Vocational Qualification CXC Caribbean Examinations Council FM Financial Management GCTVET Grenada Council for Technical and Vocational Education and Training GDP Gross Domestic Product IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IERR Internal Economic Rate of Return ID Identification Document IO Intermediate Outcome ISR Implementation Status Report KPI Key Performance Indicator M&E Monitoring and Evaluation MoE Ministry of Education, Human Resource Development and Labor MIS Management Information System NSDC National Skills Development Centre NPV Net Present Value NVQ National Vocational Qualification OECD Organization of Eastern Caribbean States PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PRF Project Result Framework PSR Project Status Report QAG Quality Assurance Group QEA Quality at Entry Assessment QSA Quality of Supervision Assessment TVET Technical and Vocational Training and Education TTL Task Team Leader Vice President: Hasan A. Tuluy Country Director: Sophie Sirtaine Sector Manager: Reema Nayar Project Team Leader: Harriet Nannyonjo ICR Team Leader: Harriet Nannyonjo ST. LUCIA OECS Skills for Inclusive Growth Project (First Phase of the OECS Skills for Inclusive Growth Program) CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 3 3. Assessment of Outcomes ............................................................................................ 8 4. Assessment of Risk to Development Outcome......................................................... 14 5. Assessment of Bank and Borrower Performance ..................................................... 16 6. Lessons Learned ....................................................................................................... 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 19 Annex 1. Project Costs and Financing .......................................................................... 20 Annex 2. Outputs by Component ................................................................................. 21 Annex 3. Economic and Financial Analysis ................................................................. 22 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 24 Annex 5. Beneficiary Survey Results ........................................................................... 26 Annex 6. Stakeholder Workshop Report and Results................................................... 27 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 28 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 42 Annex 9. List of Supporting Documents ...................................................................... 43 MAP A. Basic Information OECS (LC) Skills for Country: St. Lucia Project Name: Inclusive Growth Project ID: P097141 L/C/TF Number(s): IDA-43000 ICR Date: 2/18/2014 ICR Type: Core ICR GOVERNMENT OF ST. Lending Instrument: SIL Borrower: LUCIA Original Total XDR 2.40M Disbursed Amount: XDR 2.40 M Commitment: Revised Amount: XDR 2.40M Environmental Category: C Implementing Agencies: Ministry of Education and Culture Co-financiers and Other External Partners: None B. Key Dates Revised / Process Date Process Original Date Actual Date(s) Concept Review: 06/12/2006 Effectiveness: 12/19/2007 12/19/2007 12/14/2011 Appraisal: 01/17/2007 Restructuring(s): 04/02/2013 Approval: 05/08/2007 Mid-term Review: 07/17/2012 07/14/2010 Closing: 03/15/2012 09/15/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project at No Quality at Entry (QEA): None any time (Yes/No): Problem Project at any time Quality of Supervision Yes None (Yes/No): (QSA): DO rating before Moderately Satisfactory Closing/Inactive status: i D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 24 24 General public administration sector 4 4 Other social services 21 21 Vocational training 51 51 Theme Code (as % of total Bank financing) Education for the knowledge economy 24 24 Improving labor markets 25 25 Other Private Sector Development 13 13 Social Inclusion 25 25 Vulnerability assessment and monitoring 13 13 E. Bank Staff Positions At ICR At Approval Vice President: Hasan A. Tuluy Pamela Cox Country Director: Sophie Sirtaine Caroline D. Anstey Sector Manager: Reema Nayar Eduardo Velez Bustillo Project Team Leader: Harriet Nannyonjo Andreas Blom ICR Team Leader: Harriet Nannyonjo ICR Primary Author: Cecilia Zanetta F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To assist the Government of St. Lucia to increase the employability of youth through private-sector driven training. Revised Project Development Objectives (as approved by original approving authority) Not Applicable (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 Percentage of enrolled youth employed fifteen months after the start of the training 65% (Dec. 2011) Value quantitative or n.a. 75% n.a. Dropped (April Qualitative) 2013) Date achieved Dec. 31, 2006 March 15, 2012 April 2012 Comments Not Applicable. This indicator was dropped under the April 2013 restructuring (incl. % because it is was deemed not to be a good measure of the Project’s impact on the achievement) “employability of youth” since it greatly depends on exogenous factors such as market conditions. Indicator 2 Number of youth certified at Level 1 or above Value 726 (Dec. 2011) 1,119 youth certified 0 1,265 quantitative or 407 (April 2013) at Level 1 or above ii Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Exceeded (88%, 154%, and 275% level of achievement with respect to Appraisal, 2011 and 2013 restructuring targets, respectively). Originally an Intermediate Outcome Indicator, this indicator was made a PDO Indicator under the April 2013 restructuring, as it was deemed to better capture the Project’s impact on “employability of youth” given that certification is an asset that enhances ability of a Comments trainee to get a job and/or keep one. The target was scaled down under the 2011 and (incl. % 2013 restructurings to reflect the slower than expected pace of implementation in the achievement) aftermath of the 2008 global crisis which affected, among other things, employers’ ability to offer paid internships to trainees. A total of 1,146 certifications were awarded 1 to 1,119 youth, including 694 Level 1 NVQ/CVQ, 218 received Level 2 NVQ/CVQ 2 and 234 international certifications . (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Intermediate Outcome 1 (i.e., Component 1) To increase levels of training among unemployed youth through the establishment of a competitive training scheme that finances private sector-driven training and traineeships. Indicator 1 Number of unemployed youth enrolled in the training Value 870 (Dec. 2011) quantitative or 0 1850 1,170 945 (April 2013) Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Comments Exceeded (63%, 135%, and 124% level of achievement with respect to Appraisal, (incl. % 2011 and 2013 restructuring targets, respectively). achievement) Percentage of enrolled youth that become certified one year after the start of the Indicator 2 training Value quantitative or 0 71% 65% (Dec. 2011) 95.6% Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Comments Exceeded (135%, and 147% level of achievement with respect to Appraisal, and (incl. % 2013 restructuring target, respectively). achievement) Indicator 3 Percentage of enrolled youth in training in other sectors than hospitality Value 0 30% 38% (April 2013) 50% quantitative or 1 27 youth received two certifications 2 International certifications are those awarded by institutions that are recognized internationally beyond the Caribbean Region, for example the American Hotel and Lodging Education Institute and Monroe College. This means that trainees awarded international certification could potentially secure employment both within and outside the Caribbean. iii Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Comments Exceeded (167%, and 132% level of achievement with respect to Appraisal, and (incl. % 2013 restructuring target, respectively). achievement) Intermediate Outcome 2 (i.e., Component 2) To develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training. Indicator 4 Number of pre-qualified or accredited training providers Value 65 (Dec. 2011) quantitative or 0 100 122 117 (April 2013) Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Comments Exceeded (122%, 188%, and 104% level of achievement with respect to Appraisal, (incl. % 2011 and 2013 restructuring targets, respectively). achievement) Indicator 5 Number of Standards Vetted Value 85 (Dec. 2011) quantitative or 0 n.a. 101 87 (April 2013) Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Exceeded (119% and 116% level of achievement with respect to 2011 and 2013 Comments restructuring targets, respectively). (incl. % achievement) Note: This indicator was added during the Nov. 2011 restructuring. Indicator 6 National Qualifications register in operation Value quantitative or No Yes Yes Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Comments (incl. % Achieved. achievement) Indicator 7 Action plan for the inclusion of vulnerable groups in the economy Value Dropped (Dec. quantitative or No Yes 2011) Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Comments Not Applicable. This indicator was dropped under the 2011 restructuring, since the (incl. % Project already targeted this population group achievement) Action plan to improve the relevance and sustainability of training, including: a) Study Indicator 8 for career mobility; b) Labor market needs assessment; and c) Future financing of training. Value a) Yes a) Yes quantitative or 0 b) Yes c) Dropped b) Yes Qualitative) c) Yes Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Sept. 15, 2013 Achieved. The study of career mobility and the labor market needs assessment were Comments completed in 2009 and 2012, respectively. The third study (i.e., Future Financing of (incl. % Training) was dropped under the Dec. 2011 restructuring since it was partially covered achievement) under the labor market needs assessment. Intermediate Outcome 3 (i.e., Component 3) To strengthen institutional capacity to better implement, monitor and plan training through the strengthening of the Project implementing agencies Percentage of above indicators and of additional implementation indicators reported Indicator 9 on a yearly basis by NSDC Value 0 100% 100% iv quantitative or Qualitative) Date achieved Dec. 31, 2006 March 15, 2012 Sept. 15, 2013 Comments (incl. % Achieved (100% level of achievement). achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 06/22/2007 Moderately Satisfactory Moderately Satisfactory 0.00 2 10/17/2007 Moderately Satisfactory Moderately Satisfactory 0.00 3 05/08/2008 Moderately Satisfactory Moderately Satisfactory 0.25 4 11/02/2008 Moderately Satisfactory Moderately Satisfactory 0.25 5 06/29/2009 Moderately Unsatisfactory Moderately Unsatisfactory 0.38 6 12/19/2009 Moderately Unsatisfactory Moderately Unsatisfactory 0.38 7 06/28/2010 Moderately Unsatisfactory Moderately Unsatisfactory 0.63 8 02/27/2011 Moderately Satisfactory Moderately Satisfactory 0.85 9 09/12/2011 Moderately Unsatisfactory Moderately Satisfactory 1.99 10 03/25/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.99 11 09/19/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.99 12 03/31/2013 Moderately Satisfactory Moderately Satisfactory 2.99 13 07/24/2013 Moderately Satisfactory Satisfactory 2.99 H. Restructuring (if any) ISR Ratings at Amount Disbursed Restructuring Board Approved Restructuring Reason for Restructuring & Key at Restructuring in Date(s) PDO Change Changes Made DO IP USD millions Level-Two Restructuring to respond to the economic downturn that had affected the pace of implementation of the demand-driven, private sector- oriented training under the Project, 12/14/2011 No S S 1.99 incl.: (a) changes to the scope of activities; (b) revisions to Project financing; (c) an update of the Project Results Framework (PRF); and (d) an extension of the closing date to Sept. 15, 2013. Level-Two Restructuring to revise the PDO indicator and PRF targets to 04/02/2013 No MU MS 2.99 better measure the Project’s impact on the employability of youth over the long term. v I. Disbursement Profile vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. In the second half of the 2000s, economic growth was steadily improving in the countries of the Organization of Eastern Caribbean States (OECS), yet their economies continued to face obstacles to competitiveness and economic development, including the shortage of skills. Over the previous decade, countries, in the region had undergone a dynamic shift, transitioning from mono-crop based agriculture to a service-oriented economy, placing significant pressure on labor markets. Tourism and construction had become the main drivers of economic growth, as illustrated by the case of St. Lucia, where 1,100 new hotel rooms (equivalent to a 25 percent increase) were built in 2006 alone. The upsurge in tourism-related activity was expected to continue to fuel the demand for skilled labor in the short and medium terms. 2. Government officials in the OECS saw the region’s high economic growth as a window of opportunity to address the urgent need of providing relevant skills for unemployed youth to enter the labor market, as well as an opportunity to help transition the labor force into higher-end services. In St. Lucia, youth unemployment was 39 percent in 2005 compared to 13 percent for the whole workforce. Youth unemployment imposed a high cost to society as a result of increased risks of deviant behavior and forgone earnings. It was estimated that bringing youth unemployment down to the level of the whole workforce could increase GDP by 1.3 percent. 3 3. Enhancing the efficiency of labor markets was also an important regional priority, as increasing movement of artisans and skilled persons across the Caribbean Community was seen as a crucial element to the success of the incipient Caribbean Single Market and Economy (CSME). 4 In this regard, the Caribbean region had taken steps to provide a solid human capital foundation for economic growth and integration, including adopting a Caribbean Vocational Qualifications (CVQ) framework that set the foundations for common standards and recognition of technical and vocational qualifications throughout the region, and alignment of Technical and Vocational Training and Education (TVET) with workplace demands. 4. Approved in 2007, the Bank-financed OECS Skills for Inclusive Growth Adaptable Program Loan (APL) aimed to provide support to OECS countries to 3 World Bank (2003), Caribbean Youth Development: Issues and Policy Directions, Washington, D.C., 4 The Caribbean Single Market and Economy (CSME), also known as the Caribbean Community (CARICOM) Single Market and Economy, is an integrated development strategy envisioned at the 10th Meeting of the Conference of Heads of Government of the Caribbean Community which took place in July 1989 in Grand Anse, Grenada. One of the key elements of the CSME is the free movement of labor - through measures such as removing all obstacles to intra-regional movement of skills, labor and travel, harmonizing social services (education, health, etc.), providing for the transfer of social security benefits and establishing common standards and measures for accreditation and equivalency (http://www.caricom.org). 1 undertake crucial reforms needed for improving labor skills development to, in turn, meet the labor demands required to sustain economic growth. St. Lucia was the first country to join the Program. 1.2 Original Project Development Objectives (PDO) and Key Indicators 5. The objective of the first phase of the APL was to assist the Government of St. Lucia to increase the employability of youth through private-sector driven training. Employability was defined as the knowledge, skills and behavior of beneficiaries to help youth transition to the labor market (p. 5 of the PAD). Employable skills sought out by employers include basic literacy and numeracy, higher order skills such as decision making and problem solving and life skills such as dependability, positive attitude, self- confidence, punctuality). 6. The PDO Indicator was the following one:  Percent of enrolled youth employed fifteen months after the start of the training program (Target value: 70 percent average over the life of the Project and 75 percent for Year 4). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 7. While the PDO remained unchanged throughout the life of the Project, modifications were introduced to the PDO Indicators, as follows:  Percent of enrolled youth employed fifteen months after the start of the training program – This PDO Indicator was dropped under the April 2013 restructuring, as it was deemed not to be a good measure of the Project’s impact on the “employability of youth” since it greatly depends on exogenous factors such as market conditions.  Number of youth certified at Level I and above was added as a PDO Indicator under the April 2013 restructuring because certification was deemed to be a strong long-term measure of employability since it is an asset that enhances the ability of a trainee to get a job and/or keep one. 1.4 Main Beneficiaries 8. The Project’s original direct beneficiaries were unemployed youth ages 17 to 30 with less than three Caribbean Examinations Council (CXC) passes. This population group was targeted for training under the Project as it was considered to have insufficient academic qualifications to obtain employment or proceed to higher education. Under the 2011 restructuring, the pool of direct beneficiaries was expanded to include those aged 16 to 40 with less than four CXC passes, thus expanding the targeted at-risk population in the context of widespread unemployment. 9. Indirect beneficiaries included employers, who would benefit from a labor pool with enhanced, certifiable labor skills, as well as training providers and other individuals involved in the various aspects of labor skills certification, who would benefit from expanded professional and business opportunities. 2 1.5 Original Components 10. Component 1 - Skills training for unemployed youth (US$3.56 million equivalent to 67 percent of Project cost): This component provided financial support to the training of the targeted youth population. 11. Component 2 - Improving the policy framework for training (US$0.84 million, equivalent to 16 percent of Project cost): This component supported policies and introduced competency-based standards to improve the quality of training and assistance to youth under Component 1. 12. Component 3 - Project management and institutional strengthening (US$0.92 million, equivalent to 17 percent of the Project cost): This component provided financial support to strengthen project management. 1.6 Revised Components 13. To address the deteriorated economic context in the aftermath of the 2008 global crisis and the corresponding decline in demand for workers, the relative importance of Components 1 and 2 was revised under a Level Two restructuring in 2011. Specifically, the number of trainees under Component 1 was reduced in response to a lower than anticipated market absorption capacity while more emphasis was placed on Component 2 to strengthen the policy framework for competency-based training, including building capacity for assessment, certification, and quality assurance. 1.7 Other significant changes 14. Level Two-Project Restructuring – December 2011: The 2011 Level Two restructuring also introduced the following changes: a) Revisions to Project Financing, including: i) a reduction in Project costs; ii) the elimination of the monetary contribution of employers to the training scheme in order to offset the economic downturn and encourage their participation; and iii) a reallocation of proceeds in accordance with the changes to the scope of activities. b) Updating of the Project Results Framework, (PRF) including revision of the targets to reflect the changes in scope of activities as a result of the economic downturn. c) Extension of the closing date, from March 15, 2012 to September 15, 2013, to allow sufficient time for full Project implementation, including training of additional cohorts. 15. Level Two-Project Restructuring – April 2013: The 2013 Level Two restructuring also introduced changes to the end of Project targets of IO Indicators to reflect changes in the pace of implementation (see Annex 2 for a detailed description). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 3 16. While the Project was grounded in a robust technical design, it also exhibited some weaknesses, as summarized below: Strengths:  The Project’s overall focus and PDOs was fully aligned with the OECS Country Assistance Strategy (CAS) for 2006-2009, which emphasized “supporting growth and competitiveness” as well as “reducing vulnerabilities, particularly targeting at-risk youth.”  Its innovative design incorporated state-of-the-art knowledge acquired from similar Bank-financed skills development projects targeting unemployed youth in the region, as well as findings from three Analytical and Advisory Activities (AAA) products. 5 Particularly notable was the emphasis on demand-driven training to ensure the alignment with market needs and the incorporation of soft skills training, which had been identified elsewhere as being highly valued by both employers and trainees.  The system of incentives embedded in the Program’s operational rules, such as outcome-based payment schemes to training providers, was critical to its successful and more efficient implementation. Fee payment to training providers was subject to their trainees successfully obtaining certification.  The Program’s strong regional dimension effectively built on the ongoing momentum toward regional integration, by ensuring compatibility of labor skills competency frameworks, maximizing cross-fertilization and capitalizing on the potential for economies of scale. In this regard, the so-called “Horizontal” APL was an appropriate choice of lending instrument, as it effectively supported operations in several countries under a common program framework, ensuring consistency at the regional level, and reducing Project preparation and supervision costs.  The Project benefited from an in-depth labor market assessment and a close working relationship with the hotel and tourism industry during preparation.  The Project’s implementation arrangements effectively built upon St. Lucia’s institutional framework, assigning responsibilities for regulation and quality assurance to the TVET Unit within the Ministry of Education, Human Resource Development and Labor (MoE) and responsibilities for coordinating the provision of training to the National Skills Development Centre (NSDC), fully capitalizing on its capacity in terms of counselors and instructors, basic skills training curriculum, management information system (MIS), and physical facilities. Weaknesses: 5 Caribbean Development Bank, Department for International Development, the European Commission for Barbados and the Eastern Caribbean, the United Nations Development Programme and the World Bank (2005), Promoting an Integrated Social Protection Framework for the Caribbean, Twelfth Meeting of the Council for Human and Social Development, Caribbean Community, April 27 – 29, 2005, Georgetown, Guyana.; World Bank (2003), Caribbean Youth Development: Issues and Policy Directions, Washington, D.C. World Bank (2006), OECS Skills Enhancement Policy Note, Washington, D.C. 4  The Project design may have been overambitious, as it called for the simultaneous development of a vocational qualification framework and the establishment of a demand-driven, competency-based training mechanism. Not only were both tasks logistically challenging in and of themselves but the establishment of a competency- based training also depended to a large extent on the progress made in the development of the vocational qualification framework.  The PRF exhibited weaknesses, which are described under Section 2.3. 2.2 Implementation 17. Although the Project was implemented under unusually adverse macroeconomic conditions, it benefitted from the sustained political support from top Government authorities under two different administrations, strong leadership within the MoE and NSDC, and effective project management. The specific factors affecting implementation can be summarized as follows: Factors outside the control of the Government and the implementing agency 18. Unfavorable economic environment: The impact of the 2008 global financial crisis on St. Lucia’s highly dependent economy was severe. Over the previous decade, Saint Lucia had been growing at an average rate of 2.1 percent, with the highest growth rate of 7.4 percent, in 2006. Following the recession, growth rates declined to an average of 0.14 percent during the 2009-2012 period. The decline in economic activity was due in part to reduced activity in the construction sector, as several sizeable investments in the tourism sector were postponed. 19. Sharp drop in the demand for labor skills: To stimulate public-private partnerships and move towards more sustainable training, the Project’s training scheme originally required the private sector to contribute a nominal 5 to 10 percent towards the cost of technical training (EC$120-240) and payment of the stipend for a trainee during the traineeship to cover transportation costs (EC$1,200). While payment of a small stipend for trainees had been a common practice, the nominal contribution of 5 to 10 percent was new but considered feasible in the economic conditions prevailing at appraisal. Ultimately, the ripple effects of the global crisis reached Saint Lucia’s economy and the demand for employer-paid internships—and even unpaid ones— virtually vanished. This became a substantial stumbling block for the implementation of the training component, which was eventually addressed under the Project’s restructuring in 2011. Factors Generally Subject to Government Control 20. Sustained support on the part of authorities: While the Project was prepared under one administration and implemented under another one from the opposition party, the Project’s objectives of providing demand-driven technical and life skills training among vulnerable youth and developing the corresponding institutional framework remained unchanged. 5 Factors Generally Subject to Implementing Agency Control 21. Key changes were introduced to the training program: Ultimately, the Project was successfully implemented as a result of the flexibility and proactive attitude exhibited by the PCU, the MoE, and the NSDC, which were able to effectively expand the focus of the training to other sectors beyond tourism that were not as hard hit by the global financial crisis and to accommodate changes demanded by employers. In addition, efforts to expand the number of employers offering internships were successful. Finally, critical operational changes were also introduced to the training program, including increasing the stipend amount, expediting payment, and expanding training offerings to better match the interests of the trainees. These changes led to drastic reductions in drop- out rates, from 26 percent in 2012 to 8 percent in 2013. 22. International certification: The MoE partnered with the Caribbean Institute of International Hospitality Management (CIIHM), the American Hotel and Lodging Education Institute (AHLEI) and Monroe College to provide international qualifications in the areas related to hospitality. As a result of this collaboration, 234 Level 2 international certifications were awarded under the Project. 23. Ongoing collaboration with Grenada under the APL framework: Collaboration between St. Lucia and Grenada 6 under the OECS Skills for Inclusive Growth Program helped both countries learn from each other and move faster with implementation, as they routinely shared documents and experiences. For example, St. Lucia’s quality assurance documents were based on those of Grenada, while Grenada’s Life Skills document was an adaptation of the St. Lucia’s one. This collaboration also included the joint development of the information monitoring system (IMS). These are just some of many concrete examples of collaboration between the two countries that took place during implementation and are expected to continue after Project closing as a result of the strong links that have developed between professional staff in both countries. 24. Strong collaboration with CARICOM countries and agencies: The St. Lucia team worked closely with their counterpart in Grenada, other countries in the region like Jamaica, Trinidad and Tobago, and the regional organizations, such as the Caribbean Association of National Training Agencies (CANTA), as they developed labor certification standards and certification of key quality assurance staff. This close collaboration led, among others, to the MoE’s TVET Unit’s accreditation by CANTA, which enables it to issue CVQs, effectively expanding the possibilities for employment of the Project graduates to the entire CARICOM region. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 6 A US$3.00 million credit was approved for Grenada on January 14, 2009 for the Second phase of the OECS Skills for Inclusive Growth Program. 6 25. Design: The Project’s M&E design effectively built upon the NSDC’s information system, which was already in place, and its PRF included a battery of robust intermediate indicators. However, it exhibited the following weaknesses regarding the PDO indicators:  It became apparent during implementation that the employment based indicator was not effective in capturing the Project’s impact, as employment can be substantially affected –both positively and negatively – by factors outside the orbit of the Project.  The PRF also failed to capture the establishment of a vocational qualifications framework, which was a major focus of the Project. 26. Implementation: The M&E system went beyond the PRF to include gender, drop- out rates, geographical location and occupational area. A more ambitious information management system (IMS) to support the M&E needs of the newly established vocational qualifications framework has also been developed under the Project in collaboration with Grenada. 27. Utilization: The M&E system for the TVET Unit made it possible to support all stages of implementation of the training program, as well as closely monitor the progress being made and the achievement of expected results, including identifying reasons for trainee drop out. It also served as the first register of labor skills certificates being issued. Once fully implemented, the IMS could also provide the basis for a regional labor-skills certification registry. 2.4 Safeguard and Fiduciary Compliance 28. Environmental and social safeguards: The Project had an environmental category C rating given that a neutral environmental impact was expected. No social safeguards were triggered either. 29. Financial Management (FM): The Project’s FM was conducted in accordance with Bank’s policies and procedures. Audits were submitted on time. 30. Procurement: Albeit slow, procurement activities were conducted for the most part in accordance with Bank policies and procedures. In 2012, procurement was downgraded because the PCU failed to follow Bank guidelines, causing delays in the issuing of No Objections on the part of the Bank. Corrective actions were subsequently adopted and procurement was upgraded later that same year. 2.5 Post-completion Operation/Next Phase 7 31. The Project’s implementation fully relied on Saint Lucia’s permanent institutions (i.e., the MoE’s TVET Unit, and the PCU, also a permanent unit within the Ministry of Finance and Economic Affairs). As a result, there are no significant changes to be made in the post-completion phase, except for incorporating into the permanent staff a handful of technical staff positions that were financed by the Project. The MoE has conducted an Impact Assessment of the Project and Self-Evaluation Study to fully capitalize on the lessons learned from the Project. In addition, the MoE’s TVET Unit has finalized its new Strategic Plan and a Plan for Sustainability, which was informed by the results from the Labor Market Needs Assessment (2013). In addition, St. Lucian authorities have decided to continue the integration of competency-based training program into the NSDC’s training programs as well as TVET education at the secondary level. Assessment of prior learning, based on the competency framework established by the Project, has also started to facilitate certification of employed youth who do not have certification. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Substantial 32. Objectives: The relevance of the Project’s PDO is deemed to be Substantial. As pointed in the 2013 Budget Speech, 7 increasing the pool of labor skills as a way to enhance quality and productivity while tackling stubbornly high unemployment is critical for igniting St. Lucia’s hidden growth potential and moving up the development ladder. In turn, the Government’s TVET strategic plan 2014 - 2019 responds to the findings of the 2013 Labor Market Needs Assessment, which also underscores the mismatch of the skills set for the available jobs in the labor market. 33. The importance assigned to skills training and outreach to unemployed youth is a priority to Saint Lucia as clearly articulated in the Ministry of Education and Culture Education development Plan 2009-2014 and beyond which placed continued emphasis on raising the level of achievement of all learners, reforming the management of education, and improving the relevance of the secondary curriculum and introduction of alternative modes of certification. In addition, the PDO is fully aligned with the current OECS Education Sector Strategy 2012-2021, 8 which includes as one of its seven strategic imperatives, the need to provide opportunities for all TVET learners to develop the knowledge, skills and attitudes to enable them to progress to further education and training and productive employment through the development of a competency-based curricula linked to the CVQ framework established across secondary and tertiary education and in all other educational settings in work and in community. 34. The Project’s PDO is also consistent with the Bank’s current Regional Partnership Strategy for OECS Countries 2010-2014 (Report No. 53762-LAC, May 3, 2010), which 7 Budget Statement 2013, delivered by the Hon. Dr. Kenny Davis Anthony, Prime Minister and Minister for Finance, Economic Affairs, Planning & Social Security, May 14, 2013; Castries, Saint Lucia 8 The OECS Education Sector Strategy 2012-2021 reflects the overarching education strategy of the OECS and is used by Member States to align their national Strategies and Plans. 8 had two objectives: i) to build resilience; and ii) enhance competitiveness and stimulate growth over the medium term. By focusing on the development of labor skills among vulnerable youth, the Program contributed directly to these two objectives. In addition, the Project’s PDO is fully aligned with the World Bank Education Strategy 2020, which focuses on learning for all, beyond schooling in recognition that the level of skills in a workforce predicts economic growth rates far better than do average schooling levels. 35. Design and implementation: The relevance of the Project’s design is deemed to be Substantial. With its emphasis not just on technical and life skills training provision per se but also on internships and development of a regionally relevant labor skills competency framework, the Project’s design continues to be highly relevant. The original focus on internships paid by employers within the tourism and construction sectors was relevant in the context of the economic conditions at appraisal. After economic conditions deteriorated, the focus shifted toward publicly paid internships in sectors beyond tourism and construction, which ensured the continuous relevance of the Project’s design. Project’s restructurings contributed to maintaining the operation’s relevance in the context of the prevailing economic conditions and demand on the part of employers, significantly expanding its scope beyond the tourism and construction sectors as well as its target population to include more educated youth. The Project design continued to be highly relevant as denoted by the strong focus on development of the institutional framework and management skills needed to support the development and implementation of a regionally relevant labor skills competency framework, the decision of the GOSL to continue with a similar youth certification-based training program under the NSDC, and the National Initiative for Creating Employment (NICE). The strong collaboration with other CARICOM countries and regional agencies that took place under the Project also contributed toward regional economic integration, a top priority for St. Lucia and other OECS countries. 3.2 Achievement of Project Development Objectives Rating: Substantial The PDO was to assist the Government of St. Lucia to increase the employability of youth through private-sector driven training. PDO Indicator: Number of youth certified at Level I or above. 36. PDO achievement is considered to be Substantial, as demonstrated by the substantial progress toward achieving the PDO of increasing the employability of youth through public/private sector partnerships for demand-driven technical and life skills training. 37. The number of youth awarded NVQs/CVQs 9 certification exceeded the 2011 and 2013 restructuring targets. A total of 1,119 youth received certification at Level 1 or 9 The Caribbean Vocational Qualification (CVQ) is an award that represents achievement of a set of competencies which define core work practices of an occupational area, consistent with the levels articulated within the regional qualifications framework. To earn an award, trainees must demonstrate 9 above compared to the targets of 726 and 407 youth defined as part of the 2011 and 2013 restructurings (154 percent and 275 percent level of achievement, respectively). Achievement was also considerable compared to the original target of 1,265 youth defined at Appraisal (88 percent level of achievement). Of the 1,119 youth that received certification, 694 received Level 1 NVQ/CVQ and 234 received international certification, and 218 Level 2 NVQ/CVQ 10 which may explain why over 43 percent of the certified trainees were employed within less than six months of start of training. It is important to emphasize that certification outcomes capture not just training outputs (i.e., the number of trainees that received soft and technical training and completed their internship) but achievement of industry-vetted, independently certified, competency- based skills that are recognized both at the national, regional 11 and international 12 levels. Prior to certification, all trainees regardless of level of training, were assessed on technical skills and a common core of employable skills 13, including basic literacy and numeracy, higher order skills such as decision making and problem solving and life skills such as dependability, positive attitude, self- confidence, punctuality and other aptitudes required to function effectively on the job. The purpose of the assessment was to determine whether trainees have met the requirements outlined in the competency standards for the qualification they seek. This assessment was based on job performance during internship and on a portfolio compiled by the trainee to demonstrate mastery of skills. Those who complete the training and pass the assessment are then certified. 38. By focusing on the development of skills—both life skills and technical skills— through a demand-driven training program, the Project directly contributed to improving the employability of youth. Life skills training provided under the Project focused on the development of socio-emotional or non-cognitive skills (also known as “life skills/productivity enhancement training (PET)” or “soft skills”) related to behaviors and attitudes, such as dependability, positive attitude, team work, and conflict resolution. Recent evidence from econometric and qualitative analyses point to the importance of non-cognitive skills as determinants of success in the labor market and workers’ productivity. 14 Moreover, recent studies indicate that employers in the Latin America and Caribbean region are placing increasing value on life skills vis-à-vis technical competence in reaching CARICOM-approved occupational standards developed by practitioners and employers (i.e. the industry experts). 10 27 Youth Received two certifications 11 In case of CVQ 12 In the case of international certification. 13 There was a common core of life skills for all occupational fields irrespective of the level of training. This means that those trained for example in Housekeeping were required to go through the same life skills training as those under Data Operations. 14 See Heckman, Stixrud and Urzua, 2006;Fazio, 2011. 10 skills. 15 Similarly, by requiring training providers to demonstrate employers’ commitment to provide on-the-job internships as part of their proposal, the Project ensured that employers played a role in determining the allocation of training resources among the various occupations and economic sectors, thus ensuring the relevance of the technical skills developed under the Project. 39. The large number of trainees certified was enabled by several Project activities: First, the Project succeeded in mobilizing trainees, training providers and employers resulting in 1,170 trainees being enrolled in training against a revised end-target of 945. The percentage of these trainees enrolled in sectors other than hospitality reached 50 percent against a revised end-target of 38 percent. A total of 122 training providers received training in competency-based training and were pre-qualified to provide training under the newly established vocational qualification framework, far exceeding the appraisal, 2011 and 2013 restructuring targets by 122%, 188% and 104% respectively. In addition, a total of 277 employers provided traineeships to trainees. The large number of employers and training providers facilitated both technical training and internships, and contributed to the percentage of enrolled youth that were certified within a year after start of the training reaching 95.6 percent against a revised 65 percent end-target and an original 71 percent target. 40. Second, the Project supported implementation of an action plan for inclusion of vulnerable groups. This included enhancing trainees’ basic numerical and literary skills, which was identified as one of the factors limiting trainees’ ability to participate and capitalize on the training. A literacy electronic tool/software was installed at fifteen institutions island-wide, including high schools and libraries, to help bridge basic learning deficiencies. Training providers also placed special emphasis on strengthening basic numerical and literacy skills as part of the technical training. This additional support helped more trainees to qualify for training and be able to complete portfolios that were part of the assessment prior to certification. 41. Third, the Project supported establishment of a robust policy and institutional framework for delivering training by enhancing OECS collaboration in training, introducing occupational standards and a quality assurance system to increase quality and value of training. This included the establishment of an industry-vetted, regionally recognized occupational standards framework, supported by a robust quality assurance system that will serve to permanently improve the quality and value of training in Saint Lucia. In May 2013, the TVET Unit of the Ministry of Education Human Resources Development and Labor received CVQ granting status from CARICOM, which is an independent endorsement of the robustness of Saint Lucia’s newly established occupational standards framework, enhancing relevance, quality, and increasing the possibility of employment outside Saint Lucia. As part of this framework, a total of 101 15 Using data from 1,200 firms in the LAC region, Bassi et al. (2012) showed that employers value non- cognitive skills more than cognitive ones, with the results being statistically robust across countries, economic sectors, and occupations. These results are consistent with those rendered by studies recently conducted in Peru and the Dominican Republic (World Bank, 2011; Martinez, 2011). 11 qualifications were validated by 30 industry advisory committees that were set up across the various occupational areas and subsequently adopted by the TVET Council (compared to an end of Project target of 87). In addition, there has been good progress in the development of the human resources needed to support the labor skills competency framework, with a total of 67 certified assessors (against a target of 30), and 40 verifiers and auditors, to support the assessment process. This system is supported by a corresponding manual that articulates the guidelines and procedures relating to issuance of NVQs/CVQs, 16 and trainee assessment which provided the basis for certification. The qualifications helped to diversify the fields of certification based on indications from the labor market, and matched availability of placements for traineeships that were an integral part of the training program, and would serve to increase employability of youth beyond Saint Lucia. The availability of qualified personnel facilitated assessment and therefore certification of a large number of youth. In the context of the newly established labor skills competency framework, industry advisory groups play a central role in determining the “content” of training—i.e., the specific skills to be built through training—thus, ensuring their relevance. In addition, by putting in place objective certification standards and an independent certification system, the Project ensured the quality of the technical skills developed under the Project. 42. The Project strengthened the institutional capacity of MoE’s TVET Unit and the NSDC to implement, monitor and plan training. This was central to ensuring that the Government of Saint Lucia had the necessary institutional capacity to support the provision of training and the establishment of the labor skills competency framework both under and beyond the Project. The increasingly stronger capacity also partly contributed to the significant decrease in trainee drop-out during the life of the Project, from 26 percent under the 2012 training cycle to only 8 percent under the 2013 training cycle, as a result of several operational improvements, including delivering the training over a reduced time span but with the same number of hours required for certification and expediting the payment of stipends. 3.3 Efficiency Rating: Modest 43. The Project’s efficiency is considered to be modest. Although the unit certification cost (i.e., cost of certified trainee) was about 1.5 percent lower than (i.e., US$2,767.00 per certification anticipated at appraisal compared to actual per capita cost of US$2,726 17 and the actual average unit cost of training was higher US$1,361.9 16 These documents include: i) Learner’s Handbook, ii) Center Approval Guidelines, iii) Assessment Guidelines, iv) Quality Assurance Manual, and v) Accreditation Manual, vi) Procedures Manual. 17 Unit certification costs are estimated based on the estimated and actual cost of Component 1 (USD3.5 million and USD3.05 million, respectively) and the number of estimated and actual number of youth certified under the Project (1,265 and 1,119 youth certified at Level 1 or above under the training component at Appraisal and end-project result, respectively). 12 compared to US$907 at appraisal, the rate of return remains high. This is mainly the result of two factors. First, actual certification outcomes were substantial (88 percent compared to appraisal) and exceeded those specified at the 2011 and 2013 restructurings (154 percent and 275 percent, respectively). Second, the GOSL made the strategic decision to increase the focus on Level 2 NVQ/CVQ and international certification, which represented higher training costs than Level 1 training, but resulted in a higher probability of employment. Out of the 1,119 youth certified, 234 received international certification and 218 received 2 NVQ/CVQ. 44. The project’s economic efficiency is also demonstrated by the results from an economic analysis of the training component, which yields an 8.5 percent internal economic rate of return (IERR) 18 for component 1 alone. It assumed that, over the next ten years, those who received training and certification under the Project will have a 20 percent increase in the probability of employment relative to their peers who did not receive any training. It is assumed that the impact will be 50 percent higher among those that received Level 2 NVQ/CVQ or international certification (i.e., 30 percent increase). Having obtained regionally recognized CVQs and international certification, trainees now have expanded employment opportunities within and outside CARICOM, thus effectively increasing the probability of being employed. The analysis also indicates that an increase in average wage income of 15-20 percent is needed (i.e., 18.2 percent) to ensure a positive rate of return with discount rates of 5, 7.5 and 10 percent compared to 20 percent at appraisal. 45. Having obtained regionally recognized CVQs and international certification, trainees now have expanded employment opportunities within and outside CARICOM thus effectively increasing the probability of being employed. The positive economic impact of the Project is also supported by the establishment of an improved policy and institutional framework that includes an industry-vetted, regionally recognized occupational standards framework that will serve to permanently improve the quality and value of training in Saint Lucia. Thus, the economic benefits under the Project should include benefits from competency-based training and certification in terms of higher probabilities of employment and wages for future cohorts of trainees and employers. In addition, a system of evaluation and certification of labor competency may induce workers mobility among firms; a byproduct that will increase a premium paid for the retention of employees. In contrast, it can also reduce the screening cost and initial training expenses of workers to be hires, leading to more efficient labor markets. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 46. The Project’s relevance of objectives and design are considered substantial, efficacy is substantial, and efficiency is considered modest, resulting in an overall Satisfactory rating. 18 No IERR was calculated at appraisal. 13 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 47. As originally envisioned, the training supported under the Project targeted solely uneducated, unemployed youth. While this population group continued to be the main beneficiary group, as part of the restructuring the targeting was expanded to include more educated, employed youth to address growing unemployment even among more educated youth. 48. In terms of gender, the larger majority of trainees were female, accounting for 76 percent. In addition to targeting vulnerable youth, Saint Lucia’s authorities made an effort to reach special-needs population groups, such as HIV infected youth, inmates at the Borderlands Correctional Facility, and girls at risk. (b) Institutional Change/Strengthening 49. The establishment of a labor skills competency framework was perhaps the most notable Project outcomes in terms of its institutional impact. In this regard, it is important to note that Saint Lucia is one of only four entities in the CARICOM region accredited by CANTA to issue CVQs. This status could only be achieved with a strong quality assurances system that was established with support from the Project. (c) Other Unintended Outcomes and Impacts (positive or negative) 50. The Project contributed toward strengthening collaboration and coordination between government agencies, in particular the MoE, the NSDC and other line ministries, particularly the Ministry of Agriculture. South-to-South collaboration cooperation with Jamaica, Trinidad and Tobago, Grenada and regional agencies, including CANTA, is another important Project outcome. There was strong collaboration with other development partners, including the European Union (EU) and Canada. Synergies were identified with an EU-funded project that allowed for training of plumbers, butchers, bee keepers and farmers utilizing the regionally approved standards purchased and validated under the Project. In addition, the development of the MIS and the labor market needs assessment was done in collaboration with a Canada-funded education project. These collaborations enabled cost savings opportunities for the GOSL as well as the promotion of TVET regionally approved framework and certification of more trainees. Trainees also obtained birth certificates and national IDs and opened bank accounts, which facilitated their mainstreaming into society. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not Applicable. 4. Assessment of Risk to Development Outcome Rating: Moderate 14 Factors contributing toward sustainability 51. There is strong political support. The newly established vocational framework and youth development continue to be areas of strategic importance for Saint Lucia’s authorities, which positively affects the sustainability of the Project’s outcomes. This importance was denoted by the high profile given to the 2013 graduation ceremony, which was attended by the country’s highest authorities, including the Governor General, the Permanent Secretary (PS) in the Office of the Prime Minister, the Minister of Education, the PS in the Minister of Finance PS, members of the diplomatic core, and foreign ambassadors. 52. Periodic regional accreditation by CANTA will help ensure that the vocational qualification framework remains intact and continues to be improved. A new accreditation review is planned for August 2014. In addition to their value as an external and periodic quality assurance process, accreditation reviews can be expected to contribute to the sustainability of the vocational qualifications framework, as losing Accreditation would have great visibility and a high reputational risk. 53. Efforts are already underway to mainstream the utilization of competency-based standards in publicly funded training programs. For example, the MoE is already working on revising the TVET curriculum at the secondary level to grant Level 1 CVQs upon high school graduation. This initiative was already piloted under the Project, with 26 high school students receiving CVQs under the Project. In addition, the NSDC is in the process of mainstreaming CVQ certification standards into its other training programs. Finally, the Ministry of Agriculture has stated its commitment to adopt competency-based training for all its extension services training. 54. The robust framework that has been established to provide quality and relevant skills. The policy and institutional framework, including occupational standards and a quality assurance system with trained assessors, verifiers and auditors is in place to support the continued operation of the CVQs and expansion of TVET in Saint Lucia. Factors posing a risk on sustainability 55. It is critical to ensure the financial sustainability of the newly established vocational qualifications framework over the medium and long term. Establishment of a financing mechanism to support the TVET Unit, giving it the ability to collect fees and charge for services, including certification, and ensure that these resources are earmarked for the TVET system would enhance the system’s financial sustainability. 56. Employers need to be made aware of the existence and potential benefits of the regional vocational qualification framework. The recently completed Labor Needs Assessment Survey shows that only 9 percent of the firms that participated in the survey were aware of the CVQ, only 2 percent utilized it to some extent, and just 1 percent did so at a greater extent. These results point to the need to undertake aggressive 15 communication efforts, perhaps involving employers from other Caribbean countries where vocational qualification framework is already well established. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 57. The Bank’s team prepared a technically robust operation, which reflected the state-of-art knowledge in active labor markets programs targeting youth as well as sound operation design that put in place a system of incentives to support the demand-driven nature of the training and the result-oriented behavior of the various actors. In addition, the Bank team worked closely with various stakeholders, particularly employers from the hotel and tourism sectors, to identify expected shortages in labor skills to be addressed under the Project. Bank support was also a key factor in fostering horizontal fertilization among OECS countries, particularly between Saint Lucia and Grenada’s officials, as well as with regional organizations, such as CANTA. The Project’s design, however, exhibited moderate shortcomings, including a PRF that did not adequately capture the Project’s development objective. (b) Quality of Supervision Rating: Moderately Satisfactory 58. The Project was under close supervision, with regular supervision missions that included field visits to training providers and employers throughout the country, which helped identify the challenges being faced at various stages of Project implementation. The Bank worked closely with the Saint Lucia authorities and provided the flexibility needed to achieve a successful implementation of Project activities, including processing two Level-2 restructurings to adapt the Project to the challenging economic landscape that ensued as a result of the global financial crisis and enhance its PRF. There were moderate shortcomings. The timing of the two restructurings, however, was questionable, as the impact of the economic slowdown on the demand for employer-paid internships was evident as early as 2009-2010. Likewise, the need to revise the PFR should have been identified and addressed earlier during the Project lifespan, rather than six months before closing. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 59. Overall Bank performance is considered Moderately Satisfactory, given the moderately satisfactory performance at quality at entry and supervision. 5.2 Borrower Performance 16 (a) Government Performance Rating: Satisfactory 60. Government performance is deemed Satisfactory. The Project received the full support from Saint Lucia’s authorities from the two administrations that were in office during its implementation. Moreover, adequate technical and financial resources were assigned to the Project despite the tight fiscal conditions that prevailed during its lifespan. Even though the Steering Committee did not meet regularly during early stages of implementation, there was strong inter-government coordination between not only the MoE and the NSDC, but also other line ministries such as Agriculture, which was crucial to the Project’s successful implementation. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 61. The performance of the PCU is considered Satisfactory. It effectively coordinated actions with key stakeholders and the private sector and maintained regular communication with the Bank team. The PCU provided effective support to the TVET Unit and the NSDC, responding swiftly to all their needs. By the end of the Project, all Project indicators were being monitored and reported on a regular basis. The TVET Unit provided strong leadership and a clear vision in the process of developing a labor skills certification framework. It worked closely with key stakeholders within the country as well as regional partners. It also exhibited ability to identify and respond to the incipient demand for certification standards. The performance of the NSDC is also deemed Satisfactory. Top authorities were actively involved in Project implementation. The NSDC’s institutional and physical resources were critical to the success of the training component, while its information system was critical in monitoring both implementation progress and results. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 62. Overall Borrower performance is considered Satisfactory, given the Satisfactory outcome rating. 6. Lessons Learned a) Lessons of wide general application 63. Measuring the impact of skills building interventions on employability poses a difficult challenge. This is particularly true in projects that, by virtue of their size and other institutional and technical limitations, do not lend themselves to randomized-type impact evaluations. While the impact on employment is undoubtedly the ultimate measure of success, it is also affected by factors external to the intervention, most notably economic conditions. Thus, in the design of skills development projects such as this one, it is important to have reasonable expectations, realizing that at best these programs can 17 help improve the alignment between the supply of labor skills as determined by market forces. 64. There are trade-offs between equity and efficiency objectives in the context of skills development programs. Targeting the most vulnerable sectors of the population, particularly poor at-risk youth who tend to have structurally less access to labor markets than their peers from other socio-economic groups, is a clear priority from an equity perspective. The argument for also including more educated, unemployed youth is supported on the grounds of both equity and efficiency, particularly in times of high unemployment. 65. Focusing on soft-skills can provide an effective focus of training programs. Both employers and trainees appear to value them greatly. Under the Project, soft skills were emphasized throughout the training, including the technical training and the internship. 66. The development of a competency certification framework is a crosscutting effort at the intercept of the orbit of several line ministries. As illustrated by the Project implementation experience, its success depended on the collaboration between several entities, primarily the MoE and the NSDC. There are opportunities for broader collaboration with the MoF and the MoL. 67. The success of the training programs is highly susceptible to operational factors such as amount and timeliness of stipends. The remarkable decrease in drop-out rates as a result of fine-tuning operational details attest to their importance. b) Project-specific lessons learned 68. The costs of materials should be taken into account when determining amounts to be paid to training providers. Rather than costs per student being uniform regardless of the type of training, the costs of inputs needed—both in terms of equipment and materials—should be taken into account in the cost structure. 69. While the output-based payment to training providers was critical to its successful implementation, there is room for further fine-tuning. To take into consideration factors affecting drop-outs that are beyond the control of training providers, such as pregnancy, illness, or lack of interest, other mechanisms should be considered, such as 100 percent payment for a minimum certification threshold (e.g., 95 percent), to strike a greater balance between the Program’s efficiency and the interest of training providers. 70. It is important to not unduly burden employers providing internships with excessive administrative work. As reported by employer representatives during the ICR mission, the feedback required from employers (i.e., the completion of two questionnaires over a four month period) was overly complex and time consuming not only for employers but also for training providers who were responsible for ensuring that they were submitted. 18 71. Training providers could benefit from having access to support from counselors and social workers. This is particularly important in projects targeting vulnerable population groups, as training providers are not necessarily equipped to deal with the wide array of personal and inter-personal issues that arise during training. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Borrower’s ICR is included in Annex 9. (b) Co-financiers Not Applicable (c) Other partners and stakeholders Not Applicable 19 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Actual/Latest Components Estimate Estimate % of Appraisal USD millions USD millions % 1. Skills Tralning for Unemployed Youth 3.50 3.05 0.87 2. Strengthening the Policy Framework for Trainlng 0.80 0.56 0.69 3. Project Management and Institutional Strengthening 0.87 1.92 2.21 Total Baseline Cost 5.17 5.52 1.07 Physical Contingencies 0.01 0.0 Price Contingencies 0.14 0.0 Total Project Cost 5.32 5.52 103.8 (b) Financing Appraisal Actual/Latest Percentage Source of Funds Type of Co-financing Estimate Estimate of Appraisal USD millions USD millions % Private Sector Contribution to training cost and 0.95 0.19 19.7 Government Budgetary resources 0.87 1.44 165.1 IDA (*) Credit 3.50 3.69 105.5 Total 5.32 5.32 100.0 (*) Higher credit amount results from more favorable exchange rates. 20 Annex 2. Outputs by Component 1. The Project’s expected targets under Components 1 and 2 were modified under the December 2011 and April 2013 Project restructuring:. Outputs by Component Intermediate Outcome 1 (i.e., Component 1) To increase levels of training among unemployed youth through the establishment of a competitive training scheme that finances private sector-driven training and traineeships. Indicator 1 Number of unemployed youth enrolled in the training (Target value: 945) Exceeded (124% level of achievement). A total of 1.170 unemployed youth were enrolled in the program. Indicator 2 Percentage of enrolled youth that become certified one year after the start of the training (Target value: 65%) Exceeded (121% level of achievement). Average certification rate was 95.6 percent (to be updated). Indicator 3 Percentage of enrolled youth in training in other sectors than hospitality (Target value: 38%) Exceeded (132% level of achievement). Percentage of enrolled youth in training in other sectors than hospitality was 50 percent. Intermediate Outcome 2 (i.e., Component 2) To develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training. Indicator 4 Number of pre-qualified or accredited training providers (Target value: 117) Exceeded (104% level of achievement). Number of pre-qualified or accredited training providers was 122. Indicator 5 Number of occupational standards vetted (Target value: 87) Exceeded (116% level of achievement). A total of 101 occupational standards vetted and formally adopted. Indicator 6 National Qualifications register in operation (Target value: Yes) Achieved. A National Qualifications register is operational. Indicator 7 Action plan to improve the relevance and sustainability of training, including: a) Study for career mobility; and b) Labor market needs assessment (Target value: Yes) Achieved. The study for career mobility and the labor market needs assessment were completed in 2009 and 2012, respectively. Intermediate Outcome 3 (i.e., Component 3) To strengthen institutional capacity to better implement, monitor and plan training through the strengthening of the Project implementing agencies Indicator 9 Percentage of above indicators and of additional implementation indicators reported on a yearly basis by NSDC (Target value: 100%) Achieved (100% level of achievement). All indicators are regularly monitored and reported. 21 Annex 3. Economic and Financial Analysis 1. The Project’s efficiency is considered to be Modest. Although the unit certification cost (i.e., cost of certified trainee) was about 1.5 percent lower than anticipated (i.e., US$2,767.00 per certification anticipated at appraisal versus an actual per capita cost of US$2,726.00), and the actual average unit cost of training was higher US$1,361.9 compared to US$907 at appraisal, the rate of return remains high. This is mainly the result of two factors. First, actual certification outcomes were substantial (88 percent compared to appraisal) and exceeded those specified at the 2011 and 2013 restructurings (154 percent and 275 percent, respectively). Second, the GoSL made the strategic decision to increase the focus on Level 2 NVQ/CVQ and international certification, which represented higher training costs than Level 1 training, but resulted in a higher probability of employment. Out of the 1,119 youth certified, 234 received international certification and 218 received Level 2 NVQ/CVQ. 2. The Project’s economic efficiency is also illustrated by the results from an economic analysis of the training component, which yields an 8.5 percent Internal Economic Rate of Return (IERR) 19 for Component 1 alone. It assumed that, over the next ten years, those who received training and certification under the Project will have a 20 percent increase in the probability of employment relative to their peers who did not receive any training. It is also assumed that the impact will be 50 percent higher among those that received Level 2 NVQ/CVQ training or international certification (i.e., 30 percent increase). Having obtained regionally recognized CVQs and international certification, trainees now have expanded employment opportunities within and outside CARICOM, thus effectively increasing the probability of being employed. 3. The analysis also assumes a 15 percent increase in per capita wages among those that received training and certification at Level 1. For those that received training at Level 2, the impact is expected to be twice as much among those that obtained Level 2 NVQs/CVQs or International certification (i.e., 30 percent). Research in other countries indicates that competency-based training and certification can yield comparable increases in wage. For example, in its analysis of the Mexican National System of Competency Standards, Skills Testing, Calderón (2000) findings suggest a 17 percent improvement in hourly wages, relative to not having been trained. 20 4. The benefits from training and certification derive from the increase in wage mass for the target group that can be attributed to the training program taking into account: i) 19 No IERR was calculated at appraisal. 20 Calderón, A. (2000), The Mexican National System of Competency Standards, Skills Testing and Certification: Measuring the benefits of on the job training and discussing the economic rationale for not passing the government the bill for the certification of workers skills, Paper prepared for the Conference “Lifelong Learning as an Affordable Investment” Hosted by the OECD and Canada, Montreal December 6- 8, 2000. 22 unemployed persons who would otherwise have no wages; and ii) employed persons who would receive higher wages. 5. A sensitivity analysis was also conducted to assess the efficiency of the training under different scenarios, as the Project’s full impact on the probability of being employed and wages cannot yet be determined due to several reasons, including the fact that most trainees completed their training in August 2013. As shown in Table 2, an increase in average wage income of 15-20 percent is needed (i.e., 18.2 percent) to ensure a positive rate of return with discount rates of 5, 7.5 and 10 percent compared to 20 percent at appraisal. These results indicate that the Project is almost as efficient as at appraisal despite the lower than anticipated number of youth certified and the unfavorable economic conditions currently prevailing in Saint Lucia as a result of the global financial and economic crisis. Table 2. Rate of return of training under several discount and wage increase scenarios d=discount rate 5% 7.5% 10% 1% -82.26% -83.07% -83.76% B = Increase in wage mass 5% -52.91% -56.99% -60.40% 10% -16.23% -24.38% -31.21% 15% 20.45% 8.23% -2.02% 20% 57.14% 40.84% 27.17% 25% 93.82% 73.45% 56.36% 30% 130.50% 106.05% 85.55% 35% 167.18% 138.66% 114.74% 40% 203.86% 171.27% 143.92% 6. The positive economic impact of the Project is also supported the establishment of an improved policy and institutional framework that includes an industry-vetted, regionally recognized occupational standards framework that will serve to permanently improve the quality and value of training in Saint Lucia. Thus, the economic benefits under the Project should include benefits from competency-based training and certification in terms of higher probabilities of employment and wages for future cohorts of trainees and employers. In addition, a system of evaluation and certification of labor competency may induce workers mobility among firms; a byproduct that will increase a premium paid for the retention of employees. In contrast, it can also reduce the screening cost and initial training expenses of workers to be hires, leading to more efficient labor markets. 23 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Theresa Parrish Beltramo Junior Professional Associate LCC3C Angela Demas Senior Education Specialist HDNED Emmanuel N. Njomo Consultant LCSFM Cornelia M. Tesliuc Sr Social Protection Specialis AFTSE Christel M. J. Vermeersch Senior Health Specialist LCSHH Evelyn Villatoro Senior Procurement Specialist EASR1 David Seth Warren Sector Leader LCC3C Supervision/ICR Nancy N. Agwu Finance Analyst CTRLD Harriet Nannyonjo Senior Education Specialist LCSHE Mary A. Dowling Program Assistant LCSHE Angela Demas Senior Education Specialist LCSHE Erik Bloom Senior Economist LCSHE Que P. Bui Procurement Asst. LCSHH Chingboon Lee Sector Manager LCSHE Reema Nayar Sector Manager LCSHE Henry Forero Ramirez Senior Information Officer TWICT Viviana A. Gonzalez Program Assistant LCSHH Saman Karunaratne Finance Analyst CTRLD Svetlana V. Klimenko Sr Financial Management Specia LCSFM Daniela Marotta Senior Economist MNSED Thomas Patrick McArdle Consultant LCSHE Judith C. Morroy Consultant LCSHH Yuki Murakami Education Economist LCSHE Maria Lourdes Noel Senior Program Assistant LCSEN Rolande Simone Pryce Senior Operations Officer AFTG2 Alonso Sanchez Consultant HDNED Nazumi Takeda Consultant HDNED Cornelia M. Tesliuc Sr Social Protection Specialis AFTSE Christel M. J. Vermeersch Senior Health Specialist LCSHH David Seth Warren Sector Leader LCC3CMDI Janet Entwistle Senior Operations Officer LCSHE 24 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY06 120.69 FY07 131.23 FY08 0.00 Total: 251.92 Supervision/ICR FY06 0.00 FY07 2.53 FY08 81.99 FY09 59.53 FY10 62.19 FY11 66.30 FY12 119.30 FY13 75.02 FY14 21.78 Total: 488.66 25 Annex 5. Beneficiary Survey Results Not Applicable 26 Annex 6. Stakeholder Workshop Report and Results Not Applicable 27 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 1. Background: The Government of Saint Lucia signed a financing agreement with the International Development Association for the financing of the OECS Skills for Inclusive Growth Project. The Project became effective on December 19, 2007. The funding was to assist the Government of Saint Lucia in increasing the employability of youth, which includes marginalized sections of the population, through private sector driven training. This would however require training/retraining, enterprise developmental initiatives and certification The Project was designed to build a sustainable base by (i) increasing the levels of training among unemployed youth through the establishment of a competitive training scheme that finances private sector-driven training, (ii) to develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training, and (iii) to strengthen institutional capacity to better implement, monitor and plan training. However, due to the global economic down turn the Project was restructured twice December 2011 and March 2013 which resulted in a change of the indicators, an increase to the cost of training and a revised closing date of September 15, 2013. The implementing agency was the Ministry of Education, Human Resources and Labour (MOE) through the Technical, Vocational, Education and Training (TVET) Unit. In addition, through a Subsidiary Agreement the National Skills Development Centre (NSDC) an approved Training Centre was selected to implement the Skills Training Component as well as certain aspects of Component three while the Project Coordination Unit (PCU) of the Ministry of Finance, Economic Affairs, Planning and Social Security was responsible for the fiduciary aspects of the Project. 2. Skills training and outreach to unemployed youth is a priority to Saint Lucia and is clearly articulated in the: (i) The 2005-2008 Macroeconomic Prospects and Medium-term Economic Strategy which highlighted the need to reduce unemployment and poverty rates, as one of the seven medium term goals; (ii) The Ministry of Education Development Plan for 2006-2010 which places continued emphasis on raising the level of achievement of all learners, reforming the management of education, and improving the relevance of the secondary curriculum and introduction of alternative modes of certification. and the (iii) The 2005 Tourism Sector Strategy and Action Plan emphasizes the goal of building the human resource base to meet the existing and developing needs of the sector and, in particular, to develop occupational standards for tourism industry jobs. 3. Saint Lucia’s commitment to these strategies is demonstrated through the establishment of a framework for TVET which includes legislation, TVET Council and a national Technical and Vocational Education and Training (TVET) Unit to serve as the operating arm of the Council. Saint Lucia’s commitment to the CARICOM TVET strategy was reinforced through the implementation of the OECS Skills for Inclusive Growth Project which sought to provide quality skills training and certification that would allow mobility of skills trained nationals across the region. 28 4. Despite the major global financial crisis and the many challenges encountered the GOSL was able to meet the developmental objective of the project. The assistance provided by the World Bank was invaluable which allowed for a more flexible and responsive approach to implementation which enabled Saint Lucia to be better able to meet the PDOs. Quality at Entry 5. Skills Training in particular technical and vocational education training continues to be a priority for Saint Lucia as the Country is confronted with an upsurge of crime, high levels of dropouts from secondary schools and the poor achievement of students from secondary schools which is heavily driven by academia. In addition, many employees at major hotels and other workplaces lack the necessary certification for mobility. While they possess the necessary experience they have not been certified. Therefore, lack of certification makes it difficult for them to take advantage of emerging opportunities that will become available through the OECS integration and CARICOM CSME. The project design should have catered for persons already employed seeking to obtain certification and assessment of prior learning. Despite that Saint Lucia sought to partner with private sector to undertake prior learning assessments to enable employees to be equipped with the necessary certification. 6. At the start of the project consultations were held mainly with the NSDC (the agency brokering training for out of school youth) and the Hotel and Tourism Association. However, with the installment of a new administration in 2006 there was a request to include other sectors so it would have been useful to hold consultations with other donor partners as well as other private sector organizations. To support this request, the project included a consultancy to carry out a study on expanding training into the other sectors. The main results were identification of new training areas based on demand that could be offered under the project. Perhaps the implementation of a social and economic assessment would have better contributed to the project design as many of trainees had several social vulnerabilities that required interventions and involvement of the Division of Human Services. 7. A disadvantage for Saint Lucia was that the project design was the replication of a training fund model from Latin America and the wider Caribbean region in particular Jamaica. However, with assistance from the Bank the Project was able to adapt to the needs of Saint Lucia. While funding was provided to strengthen the institutional capacity of the MOE through the provision of additional staff it would have been more beneficial for the Project Manager to be housed at the MOE for effective monitoring and evaluation of the Project. 8. Conceivably, it might have been better to source grant funding or project preparatory funds towards establishment of the qualification framework and then the second phase could have focused on skills training and certification. Immense focus was placed on skills training therefore this component was seen as priority. Extensive emphasis was placed on the establishment of a training fund in the absence of an operationalized 29 certification framework. Saint Lucia already had in place the necessary legislation for issuance of the CVQ however a cadre of trained assessors and verifiers was needed for its implementation. Given that one of the PDOs was the number of CVQs issued; this was presumptuous and could not be achieved until Saint Lucia received CVQ status in 2013. If there had been regard to such preparatory work; the operational framework for achieving the PDO would have been met much earlier. 9. The institutional arrangements for project implementation were clearly defined however leadership and ownership of the project was a major concern. The Project Coordination Unit, which was established prior to effectiveness of the project, was satisfactorily structured to effectively coordinate project activities. The Project required the establishment of a Steering Committee which included MOE, MOF. During implementation Saint Lucia made changes to the composition of the Steering Committee to include members of the TVET Council to ensure sustainability of initiatives. An assessment of the Operations, Objective, Design, Implementation and Operational Experience 10. The Project objective had three intermediate outcomes with associated lines of action: (i) to increase levels of training among unemployed youth through the establishment of a competitive training scheme that finances private sector-driven training, (ii) to develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training, and (iii) to strengthen institutional capacity to better implement, monitor and plan training. Overall the objective and intermediate outcomes were met and the project implementation can be rated as satisfactory. The project was restructured so that it was better able to respond to the realities of Saint Lucia thereby adjusting the IDA’s allocation to meet the nonexistent private sector contributions and inclusion of other qualifications apart from the CVQ certification. The restructuring allowed for the issuance of NVQ and international certification. 11. The original project design was flawed and may have contributed to the initial delays experienced by the project. The project design required the execution of the Subsidiary Agreement between the implementing agency, MOE and the NSDC thereby shifting the focus to skills training rather than on certification. It was not necessary for the execution of the Subsidiary Agreement to be a condition for effectiveness of the loan given that Saint Lucia was in the process of developing the system for certification which included quality training for training providers, assessors, verifiers, validation of standards. In addition, the executing agency, the Ministry of Education already has oversight over the National Skills Development Centre (NSDC) which forms part of the Agency therefore a Subsidiary Agreement was not required. Prior to project effectiveness the NSDC was under the auspices of the Ministry of Social Transformation however by project negotiations and after the general elections of 2006 that Ministry became part of the MOE. 30 12. Due to the global economic downturn the Project experienced several delays. Other contributors to delays were (a) the absence of the operational systems (b) staffing to support the system for certification and (c) limited training providers with the necessary equipment to undertake training. Some of the Trainees selected for training were not functionally literate to participate in training therefore; mechanisms had to be effected to address literacy and numeracy gaps. 13. While the Financing Agreement emphasized the need for regional collaboration with the OECS through monitoring and evaluation for impact across the OECS, the Project was being implemented in only two OECS countries, Grenada and Saint Lucia. Despite this the Project was able to effectively collaborate with Grenada on the development of the monitoring and evaluation framework for both Projects. Such collaboration included the sharing of information, templates for the design of forms and development of a common and compatible management information system used to evaluate the impact of the project across the two countries. Several meetings were held in both countries to develop and finalise the monitoring and evaluation systems for the project and TVET. Other regional collaborations were undertaken with CARICOM CANTA, Barbados and Jamaica which included the acquisition of approved standards, workshops on standards preparation and the establishment of a qualification databank that can be accessed by other OECS countries through the joint MIS initiative. This enabled a strong regional dimension and building on the ongoing thrust of regional integration (CARICOM and OECS). This also facilitated the maximization of ongoing synthesis within the region and recapitalization on the economies of scale. 14. There was also collaboration with another development partner; the European Union (EU) which funded training in the Agricultural Sector and other sectors. The EU funded project and the World Bank OECS Skills for Inclusive Growth Project identified synergies that allowed for training of plumbers, butchers, bee keepers and farmers. This enabled sensitization and wide spread use of regionally approved standards by implementers of EU funded projects. These collaborations enabled cost savings opportunities for the GOSL as well as the promotion of TVET regionally approved framework and certification of more trainees. Collaboration was undertaken with the CIDA “Education for Employment Caribbean” Project of Canada which allowed Saint Lucia to share on the development of the MIS and the labour market needs assessment. 15. The implementation arrangements functioned effectively most of the times although there were inherent challenges with some of the processes. The fiduciary arrangements were such that the PCU undertook financial and project management which worked well with the MOE and NSDC responsible for the management and training during the project. The World Bank Task Team Leader and the entire team responsible for the project were excellent in providing guidance and assistance during the challenging periods and for this the Government of Saint Lucia (GOSL) is forever grateful. 31 16. The success and satisfactory rating achieved at project closure was as a result of the level of commitment of the staff of the Bank and the GOSL. Through a South-South Exchange program financed by the World Bank and the hosting of the annual fiduciary workshops the Project team were able to participate in a study tour to Columbia to visit various training centres and to observe the various innovative technology used for the delivery of training. The fiduciary training of 2009 hosted by the World Bank allowed the Project team to collaborate with Jamaica on the development of the terms of reference for the MIS. 17. This project has been able to achieve in a few years what may have taken longer for the GOSL to achieve on its own recognisance and so efforts to sustain the momentum must be a priority of the education sector. For sustainability and to leverage funding the GOSL has developed a strategic plan and carried out an assessment of the TVET Unit to determine skills gaps and develop a talent management strategy. These investments under the OECS Skills for Inclusive Growth will enable the MOE to reorganize the work functions of existing units and sections of the ministry to better support the functions of TVET. Saint Lucia took advantage of the economic downturn experienced globally and locally by ensuring that at least 916 individuals were well-trained and certified to complement the developed workforce to meet labour market needs essential to improving the economy. Restructuring: problems encountered, actions taken, solutions. 18. The Project was restructured twice in an effort to respond to the global changes that had negatively impacted implementation of the project and contributed to the significant delays under the project. The first restructuring was done on December 15, 2011 to reallocate financing under the Project such that the contribution that was to be made by the Private sector was financed by the IDA resources and the GOSL; the closing date was extended to September 15, 2013. The second major change done on March 13, 2013 sought to revise the PDO and indicators in order to better measure the Project’s impact on the employability of youth over the long term. Assessment of the Outcome 19. Through the collaborative approach with the involvement of private sector, sensitization workshops, participation as industry advisors, and training opportunities the project was able to meet the developmental objective; to increase the employability of youth through private-sector driven training. Saint Lucia also obtained accreditation status from CARICOM CANTA to issue the CVQ prior to project closure; which is a tremendous success. The achievements, successes, challenges and lessons learned during the life of the project are outlined below. 32 Main Project Achievements Project Development Project Outcome Indicators (POI) Outcome Objectives To increase the employability 1) Percentage of enrolled youth of youth through private sector employed fifteen months after the start of 43% driven training the training. 2) Number of Youth Certified at Level 1 Total – 1,119 or Above. Target 407 (increase over the target - 175%) Component 1: Skills Training for Unemployed Youth This component implemented by the NSDC focused on the operation and expansion of a training scheme for disadvantaged, unemployed youth. This scheme was originally intended to provide training for 1,850 unemployed youth and placement in traineeships with private sector firms to gain on-the-job experience. Through the restructuring this target was reduced to 945. By project close 1,170 trainees were enrolled in training. Initially the firms were to contribute to the training costs and to pay the trainees a stipend however; due to the global economic downturn the private sector firms were unable to do so. The GOSL remained committed to the project and provided counterpart funding. Training was delivered in a number of training areas within the four sectors: Hospitality, Construction, Agriculture and Information Communication and Technology. 20. All Trainees (1,170 youth enrolled) had to undergo Productivity Enhancement Training (PET) for a minimum of 60 hours. This included life skills training to equip trainees with the necessary skills, aptitudes and attitudes required to function effectively in the course and subsequently on the job. Career counseling was also offered as part of the training to allow the trainees to be placed in an industry suitable to their skill set and aptitude in order for the matching process to be effective. This was based on feedback received after delivery of training in Cycle 1; training Providers and employers were included in that process. During implementation Life Skills Counsellor and Career Counsellors performed similar roles although the Terms of Reference for these positions were different. This was necessary given the limited expertise, with only five counsellors providing services to such a large number of individuals as well as to persons enrolled in other ongoing skills training initiatives directly under the NSDC. This enabled knowledge sharing and capacity building among the counselors who had developed in– house Life Skills Modules as a means of mitigating the negative impact and ensuring quality. These Life Skills Modules also served well as guidelines for mentors at the work place. 33 21. During the screening process it was found that many of the trainees lacked the necessary literacy skills to adequately function/participate in the training process. Therefore under the project, a literacy tool “AutoSkill” was procured and installed at fifteen (15) centres 21 island-wide to bridge learning deficiencies. The number of user licenses obtained under the Project was 175 concurrent user licenses; which enables 350 youth to access the software simultaneously for numeracy and literacy training. This initiative was realized through collaboration with the SSDF funded by the EU. Under the SSDF the Master License for Saint Lucia for a three year term had been procured which the Supplier agreed to extend given that Saint Lucia had not effectively utilized the software. 22. The MOE also procured 5 user licenses for five schools located in rural communities (Fond Assau, Roseau, Marchand, Methodist, Odsan) using funding from the Peace Corps while 15 user licenses had already been procured for the Bordelais Correctional Facility through the Saint Lucia HIV/AIDS Prevention and Control Project funded by the World Bank. Based on the successes made by the BCF, Bahamas and Jamaica in using the software the GOSL agreed to expand this initiative at the national level for easy access to youth seeking to enroll in skills training or desirous of improving their literacy levels while undertaking skills training. To support this initiative literacy training centre administrators were hired. 23. A challenge for trainees was the late receipt of stipends. This often resulted from late submission of attendance reports and trainees not having personal bank accounts for the funds to be deposited. This was later regularized but may have contributed to the dropout rate of trainees during the project. The Project however, assisted a number of trainees in obtaining birth certificates, National ID cards, Social Security Cards and the opening of bank accounts which are essential to function effectively as productive citizens. Many trainees had other personal and social issues which may have impacted the high dropout rates. While the project was an education project it depended heavily on interventions with Human Services and the health sector. Maybe provision should have been made for the hiring of social workers or the project team could have fostered greater involvement and partnership with the Department of Human Services. 21 Centres include Secondary Schools, Libraries, Combined Schools and other institutions 34 Intermediate Outcomes Intermediate Outcome /Output Outcome Indicators Component 1: Skills Training for Unemployed Youth 1) Number of unemployed youth enrolled in training. Target 945 1170 Sub-component 1.1: Training Scheme To increase levels of training among 2) Percentage of enrolled youth 95.6(1,119/ unemployed youth through the that become certified one year 1170) establishment of a competitive after the start of the training. training scheme that finances private Target 65% sector-driven training and 50(590/1170 traineeships 3) Percentage of trainees in sectors ) other than tourism. - Target 38% 1) Number of Employers enrolled Employer Support for Traineeships in providing support for the 277 training. Other Achievements: Training offered in 26 skill areas The public private partnership allowed several private companies to participate in a government project; this enhanced relationships. A total of 277 companies were involved in this project 122 Training Providers trained in areas such as methods, quality assurance, use of standards Certification issued by way of NVQ, CVQ and International qualification Training facilities at schools, NSDC and CARE upgraded to provide quality training Procured training equipment to support skills training in hospitality, electrical installation, construction and ICT Procurement of training equipment for the NSDC and CARE Development of Life Skills Modules Training provided to Career and Life Skills Counsellors A Communication strategy was developed and implemented throughout the life of the project Promotional items and paraphernalia were designed and published for promotion, advocacy and sensitization AutoSkill literacy and numeracy software implemented 12. Procurement of training kits for trainees in nail technology, cookery, electrical Component 2: Improving the policy framework for training 24. This component was intended to support activities to improve the policy framework for delivering training. However, funding was used primarily to build/establish the operational framework for TVET training and certification. Given that certification is based on competency standards and industry standards assessors were trained on the use of standards and to carry out assessments. Through extensive collaboration with CARICOM, CANTA and the OECS the project was able to reap benefits of economies of scale. This was realized through the procurement of standards and the introduction and development of competency based standards used to increase the quality and value of 35 training. The introduction of competency based standards was achieved through exchanges with HEART Trust of Jamaica; which was contracted to supply 101 standards in different training areas for the key industries including: Hospitality, Agriculture, Construction and IT. Training was also delivered on the development of standards with assistance from TVET, Barbados. These investments enabled the GOSL to strengthen the training system for TVET thus allowing for sustainability beyond the life of the project. 25. To govern the operations relating to issuance of the CVQ a Quality Assurance framework and Manual was developed to guide the implementation of the regionally approved policies for TVET and certification endorsed by CARICOM member states. The Manual clearly articulates the guidelines and procedures to be adhered to as it relates to the issuance of the NVQ/ CVQ as per the regionally endorsed qualification framework and in conformity to relevant standards. Local industry Advisory Committees were set up for the validation of regionally approved standards and endorsed by the Minister of the MOE for use by Saint Lucia. Industry Advisory groups comprised pedagogy experts in the subject area and industry personnel. Approval of standards by the policy directorate allowed Saint Lucia to issue the NVQ while approval of the quality assurance framework enabled Saint Lucia to attain the CVQ status in May 2013 conveyed by CARICOM to issue the CVQ. Accreditation of CVQ status to Grenada and Saint Lucia on the same day was unprecedented as both countries were the first in the OECS sub-region to acquire this prestigious award. 26. Through extensive outreach and sensitization the Project prequalified 122 training providers that will form part of the Accreditation system following periodic reviews. These training providers who are nationals will assist in sustainability of initiatives as they would be available to provide future training towards the attainment of CVQ and NVQ certification. With a view to testing the system and increasing the number of youth certified the MOE expanded the project deliverables through collaborations with regional and international partners. 27. Through these public private partnerships other programs such as the Caribbean Youth Empowerment Program (CYEP) got the opportunity to register 64 trainees through the Project to obtain international certification conveyed by Monroe College, Saint Lucia Campus. These trainees enrolled by the NSDC had followed the training model of the project. The MOE took advantage of every opportunity to increase the number of youth certified and therefore undertook pilot initiatives with secondary schools, a half-way house for girls, CARE, Bordelais Correctional Facility and the Ministry of Agriculture. As a result of this, 100 youth were certified at the CVQ level and 234 youth obtained international certification from AHLEI and Monroe College, Saint Lucia Campus. This was in six areas namely: Office Administration, Bar Operations & Mixology, Data Operations 1 & 2, Housekeeping, Culinary Arts and Customer Service, General Maintenance. 36 28. Overall through collaboration with NSDC, Monroe College and CIIHMC the MOE was able to provide certification to 1,119 youth that will allow them to become more marketable and to move freely within CARICOM. Certified youth achieving CVQ can now migrate to any of the member states of CARICOM to obtain employment as per CSME and OECS integration policies. Trainees obtaining international qualification have new opportunities as these trainees are able to pursue higher education (bachelor’s degree) at any college recognised by AHLEI or Monroe College in Saint Lucia (or the United States). 29. The project envisaged the development of policy action plans. To meet this requirement a Labour Market Needs Assessment was undertaken; which will enable Saint Lucia to identify training needs and job opportunities based on demand in the sectors. This will also assist Saint Lucia to prioritize future training needs. 30. These achievements will improve TVET considerably and will lead to greater employment opportunities and employability of the youth from marginalized sections of the population. Trained Assessors, Verifiers and Auditors available are an invaluable asset to support TVET and will enable continuity, sustainability and expansion of TVET. The public private partnership approach with private training institutions and employers is a solid foundation and a good model for continuity of the investment. 31. The GOSL remains committed to TVET and therefore conducted an exercise to evaluate the outcomes and impact of the Project. Through this exercise the Ministry intends to improve the effectiveness of TEVT through a self-evaluation process, obtain lessons learnt, share knowledge and experiences, and demonstrate accountability for the project results. Component 2: Improving the policy framework for training Sub-component 2.1: 1) Number of To develop an improved policy framework NVQ, CVQ or TOTAL – 1,146 for delivering training by enhancing OECS International collaboration in training and introducing certifications occupational standards to increase quality awarded. Target and value of training: 407 Quality assurance for the certification system 2) Number of 122 Prequalified training providers Operationalization of a certification system 3) National Qualifications Yes register in operation 37 Sub-component 2.2: 1) Labour Market Completed To improve policy framework for delivering Needs training by development of Policy Action Assessment Plans 2) Action plan to Strategic Plan developed improve the To evaluate outcomes and relevance and leverage funding sustainability of training Percentage of Indicators reported Annually 100% Other Achievements: Subcomponent 2.1 Introduce occupational standards, certification, and quality assurance 101 standards were acquired and validated A cadre of trained assessors, verifiers and auditors; Sixty seven (67) assessors and forty (40) verifiers in sub disciplines under four (4) core areas (Hospitality, Construction, Agriculture and ICT) were trained Industry advisory groups formed Partnerships formed with international institutions such as Monroe College of the USA and CIIHMC 12 persons trained as Master Trainers Attainment of CVQ status from CARICOM 30 Industrial Advisory Committees A quality assurance system to support certification Partnerships with other projects and companies from various industries on the use of industry standards Partnership formed with private training institutions and employers Labour Market Needs Assessment conducted to identify training needs and new employment areas. Component 3: Project management and institutional strengthening Achievements 32. The MIS was established through a joint procurement process with Grenada and will be a repository for all TVET activities. The MIS was launched and the system will provide both countries with a web based application system that can be used to monitor and track the entire training process including assessments. The system can be accessed by trainees, training providers and employers. Equipment was procured for the NSDC to support the National MIS. Initially, it was intended that the system be housed and managed at the NSDC however management decided that the system be housed at the MOE given that TVET is a national system. In the final year of implementation of the Project the IT Unit of MOE became involved and provided technical support to TVET on 38 the system. Prior to the inclusion of the IT Unit of the MOE the IT Unit of the NSDC provided the necessary technical support. The Project also provided assistance to staff at the NSDC to undertake training in IT necessary for the development of the MIS. 33. A Monitoring and Evaluation Specialist (M&E) was hired to provide support to the Project. That expertise resided at the NSDC. In retrospect the M&E should have formed part of TVET to provide information for national decision making purposes and to support the TVET framework. However, the M&E provided support to the NSDC research and data entry clerks. Training was provided on data collection, use of forms and analyses. Information exchange between the NSDC and MOE was regular to support calls for demand training and further analysis and decision making. Staff of the NSDC and MOE also participated in a general M&E training provided by the WB under the Second Disaster Management Project. 34. A Project Officer, Secretary and Quality Assurance Officer were hired to provide support to TVET. These staff provided support in standards validation, review of reports from assessor, verifiers and auditors. The PCU of the MOF provided support to the Project with the coordination of project activities, preparation of evaluation reports for training proposals, convening of steering committee meetings. The PCU also assisted by providing training in procurement. 35. It was a requirement that a steering committee be formed comprising MOEA and MOE to provide oversight and guidance on policy. At the initial stages there were difficulties in convening a meeting of the steering committee. Given that the Education Act makes provision for a TVET Council and the project was developing the quality assurance system for TVET it would have been better to use the TVET Council as the Steering Committee with agencies such as MOEA, NSDC and TVET reporting to the Council for further decision making. Evaluation of the Borrower's performance 36. Being cognizant of the global financial situation during the period of implementation the GOSL performed satisfactory. Evaluation of the Bank’s Performance 37. In the initial stages the Bank appeared to be inflexible which resulted in the Project having three Task Team Leaders during its implementation. The changes made towards project closure contributed to the overall success of the Project; therefore the Bank’s performance is rated as satisfactory. Proposed Arrangements for Future Operations 38. The GOSL has developed a strategic plan including an implementation plan, a plan for sustainability and a talent management strategy that will be used to strengthen operations within TVET. Through analyses of the outcome of the Project the GOSL is 39 well ahead in providing TVET with the necessary resources that it requires for managing and operating TVET nationally. The PCU has transferred essential equipment procured under the Project to the MOE to assist with digitalizing of all certification issued. The TVET Unit is collaborating with other initiatives such as CARCIP and NICE as new partnerships in ensuring that persons trained have access to the MIS and can apply directly to TVET for certification. The GOSL is also committed to continued collaboration with the partnering agencies with which relationships have already been formed. The MOE is moving steadily to commence assessment of prior learning through the creation of partnerships with companies such as LUCELEC. Having established a quality regionally accredited framework Saint Lucia should utilize the partnerships with developed countries for temporary labour opportunities. This would assist in alleviating the current high rates of unemployment among the youth and other marginalized population. Currently, there are farming, nursing, home help, hotel worker and construction opportunities in Canada and the United States. Challenges 39. The availability of financial resources will always pose a challenge in particular for small island states. The emerging opportunities that TVET will provide will require that training providers remain current as well as for training agents and counselors to acquire the necessary skills and techniques to effectively monitor the delivery of training. Funding is also required for the MOE to continue to build on the TVET operational systems. Sustainability of investments in particular systems, structures and the delivery of quality of education must be provided for, therefore, continued collaboration and partnerships must be fostered. Lessons Learnt  Projects of this nature require a multi-sectorial approach for its successful implementation  Collaboration with other line ministries is important for the success of TVET  Increased collaboration can enhance advocacy and promotion of the regionally endorsed certification framework  Increased collaboration will contribute to leverage of funding and cost savings for the GOSL  The inclusion of life skills programs add value for trainees, program managers and employers  At the preparation stage of projects all stakeholders must be informed and be involved  Project design should not be presumptuous such that plans are not articulated for changes in the economy (risk analysis needs to be strengthened)  This project clearly demonstrated that “one size does not fit all” – the Latin America and larger country models may not work for small island states  Implementing agencies must be strengthened to adequately implement the project  Project management and associated staffing should reside within the implementing agency 40  While the Bank has its guidelines Task Team Leaders must understand the environment in which they work and should be willing to make the necessary changes when the need arises  The Project required the establishment of a Steering Committee which included MOE, MOF. This Steering Committee must include members of the TVET Council to ensure sustainability of initiatives.  Principal Performance Ratings  (HS = Highly Satisfactory; S = Satisfactory; U = Unsatisfactory; HL = Highly Likely; L = Likely; UN = Unlikely; HUN = Highly Unlikely; HU = Highly Unsatisfactory; H = High; SU = Substantial; M = Modest; N = Negligible)  Outcome: S Borrower Performance: S  Sustainability: L Project at Risk at Any Time: Yes  Quality at Entry: S Institutional Development Impact: H  Bank Performance: S 41 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable 42 Annex 9. List of Supporting Documents Organization of Eastern Caribbean States (2012), Education Sector Strategy 2012-2021: Every Learner Succeeds, May 2012. St. Catherine, E. (2013), Analysis of the Saint Lucia Labour Needs Assessment Survey 2012, Central Statistics Office, May 14, 2013, Castries, Saint Lucia St. Lucia (2013), An Agenda For Economic Growth and Fiscal Sustainability, Budget Statement 2013 Delivered by the Hon. Dr. Kenny Davis Anthony Prime Minister and Minister for Finance, Economic Affairs, Planning & Social Security to support the appropriation bill for the financial year ending March 31, 2014 Tuesday May 14, 2013, Parliament Chamber, Castries, Saint Lucia. World Bank (2007-2013), St. Lucia - OECS (Grenada) Skills for Inclusive Growth Project. Aide Memoires, Implementation Status Results Reports (ISRs) Sequences 1 to 11, and other documents in Project Files, World Bank, Washington D.C. World Bank (2013), St. Lucia - OECS Skills for Inclusive Growth Project; Restructuring, World Bank, Washington D.C. http://documents.worldbank.org/curated/en/2013/03/17462493/st-lucia-oecs-skills- inclusive-growth-project-restructuring World Bank (2011), World Bank Education Strategy 2020: Learning for All - Investing in People’s Knowledge and Skills to Promote Development, Washington D.C. - The World Bank. 9http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTEDUCATION/0,,co ntentMDK:22474207~menuPK:282402~pagePK:210058~piPK:210062~theSitePK:2 82386,00.html0 World Bank (2011), St. Lucia - OECS Skills for Inclusive Growth Project (Adaptable Program Lending), Restructuring Datasheet, Vol. 2, World Bank, Washington, D.C. http://documents.worldbank.org/curated/en/2011/12/15556339/st-lucia-oecs-skills- inclusive-growth-project-adaptable-program-lending-restructuring-vol-2-2-data-sheet World Bank (2011), St. Lucia - OECS Skills for Inclusive Growth Project (Adaptable Program Lending), Restructuring Main Report, Vol. 1, World Bank, Washington, D.C. http://documents.worldbank.org/curated/en/2011/12/15556338/st-lucia-oecs-skills- inclusive-growth-project-adaptable-program-lending-restructuring-vol-1-2-main- report World Bank (2010), Organization of Eastern Caribbean States (OECS) - Regional partnership strategy for the period 2010 - 2014. Washington D.C. - The World Bank. http://documents.worldbank.org/curated/en/2010/05/12298466/organization-eastern- caribbean-states-oecs-regional-partnership-strategy-period-2010-2014 World Bank (2007). St. Lucia - OECS Skills for Inclusive Growth Project (Adaptable Program Lending), Washington D.C. – The World Bank. http://documents.worldbank.org/curated/en/2007/04/12329910/st-lucia-oecs-skills- inclusive-growth-project-adaptable-program-lending World Bank (2005), Organization of Eastern Caribbean States (OECS) - Country Assistance Strategy. Washington D.C. - The World Bank. http://documents.worldbank.org/curated/en/2005/09/6227004/organization-eastern- caribbean-states-oecs-country-assistance-strategy 43 Meetings Conducted during ICR Mission Ministry of Education, Human Resource Development and Labour- MoE Mr. Robert Lewis, Minister Ms. Esther Brathwaite, Permanent Secretary Mrs. Estellita Rene – Education Officer (TVET) Ms. Ruth Charlemagne, Project Officer Ms. Euthalia Philgence, Quality Assurance Ms. Marietta Edwards Mr. Joshua Vernor, ICT Euthalia Philgence National Skills Development Center Ms. Selma St Prix, General Manager Mrs. Faith- Harry Jn Baptiste Ms. Mary Francis Ms. Tara Leonard Mr. Barry Williams Bina Mathurin Ana-Elisa Cadasse Nathalie Charles Sharon Fearon Project Coordinating Unit - PCU Ms. Cheryl Mathurin, roject Coordinator Ms. Sonia Alexander, Consultant Mrs. Rosilia Joseph Ms. Yasmine Anatole Mrs. Myrtle Drysdale-Octave Monroe College Dr. Alex Ephrem, Vice-President Trainees Ms. Laureen Bruno (Customer service) Ms. Theresa Mathwrine (Office administration) Training Providers, Assessors, Auditors, and Members of Industry Advisory Committees Ms. Elizabeth Andrew Rose (Hospitality);Training provider, assessor, auditor, industry advisor. Mr. George Deyon (Construction); Assesor, auditor, industry advisor. Mr. Nigel Fulgence, Member of the St. Lucia TVET Council, assessor, auditor, industry advisor. Mr. Andrew Henry (Commercial food preparation): Training provider, assessor, auditor. Mr. Kim Montrope (Carpentry; furniture making; general construction); training provider, assessor. Mr. Andrew Meroe (Electrical installation); training provider. World Bank Harriett Nannyonjo, TTL 44 IBRD 33487 61°00'W 60°50'W Saint Lucia Channel Cap Point Pigeon Island SAINT Gros Islet LUCIA Rodney Bay Cape Marquis GROS Dauphin Choc Bay ISLET Pointe Seraphine DAUPHIN CASTRIES Grand Grand 14°00'N Anse Anse 14°00'N Grande Cul Bay de Sac Bay CASTRIES Sans Soucis Cul de Sac Marigot Bay La Croix Maingot Anse La Raye Denn er y ANSE DENNERY Fond d'or Bay Caribbean LA RAYE RAYE Denner y Dennery Sea Canaries CANARIES Mt. Gimie Praslin Praslin Bay (950 m) Tabac Mt. Tabac Grand Caille Point (678 m) PRASLIN Mon Trou Gras Point Soufrière Repos Soufrière Bay SOUFRIERE 13°50'N Ca 13°50'N Mt. Grand ne s ll e Anse de Piton Magazin (616 m) MICOUD Micoud Vierge Point Gros Piton Point Desurisseau CHOISEUL Londonderr y Londonderry Choiseul LABORIE VIEUX FORT 60°50'W Laborie Laborie Bay S A I N T L U CIA Vieux Fort SELECTED CITIES AND TOWNS 0 1 2 3 Kilometers Maria I. Vieux Fort NATIONAL CAPITAL Bay Cape Moule RIVERS 0 1 2 3 Miles Chique MAIN ROADS This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank QUARTER BOUNDARIES Group, any judgment on the legal status of any territory, or any Saint Vincent Passage INTERNATIONAL BOUNDARIES endorsement or acceptance of such boundaries. 61°00'W DECEMBER 2004