Document of The World Bank Report No: ICR2872 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-4534-GD) ON A CREDIT 4534-GD IN THE AMOUNT OF SDR 2.10 MILLION (US$3.0 MILLION EQUIVALENT) TO GRENADA FOR A OECS SKILLS FOR INCLUSIVE GROWTH PROJECT IN SUPPORT OF THE SECOND PHASE OF THE OECS SKILLS FOR INCLUSIVE GROWTH PROGRAM February 28, 2014 Human Development Sector Caribbean Country Management Unit Latin American and the Caribbean Regional Office CURRENCY EQUIVALENTS (Exchange Rate Effective February 26 , 2014) Currency Unit = Eastern Caribbean Dollar EC$2.7 = US$1.00 US$0.38 = EC$1.00 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS APL Adaptable Program Loan CANTA Caribbean Association of National Training Agencies CARICOM Caribbean Community CAS Country Assistance Strategy CSME Caribbean Single Market and Economy CVQ Caribbean Vocational Qualification CXC Caribbean Examination Council FM Financial Management GCTVET Grenada Council for Technical and Vocational Education and Training GDP Gross Domestic Product GHTA Grenada Hotel and Tourism Association IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report ISR Implementation Status Report KPI Key Performance Indicator M&E Monitoring and Evaluation NPV Net Present Value NTA National Training Agency NVQ National Vocational Qualification MIS Management Information System MoE Ministry of Education PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PRF Project Result Framework PSR Project Status Report QAG Quality Assurance Group QEA Quality at Entry Assessment QSA Quality of Supervision Assessment TTL Task Team Leader Vice President: Hasan A. Tuluy Country Director: Sophie Sirtaine Sector Manager: Reema Nayar Project Team Leader: Harriet Nannyonjo ICR Team Leader: Harriet Nannyonjo GRENADA OECS SKILLS FOR INCLUSIVE GROWTH PROJECT (SECOND PHASE OF THE OECS SKILLS FOR INCLUSIVE GROWTH PROGRAM) CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 1 3. Assessment of Outcomes ............................................................................................ 2 4. Assessment of Risk to Development Outcome........................................................... 3 5. Assessment of Bank and Borrower Performance ....................................................... 3 6. Lessons Learned ......................................................................................................... 3 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............ 3 Annex 1. Project Costs and Financing ............................................................................ 5 Annex 2. Outputs by Component ................................................................................... 6 Annex 3. Economic and Financial Analysis ................................................................... 7 Annex 4. Bank Lending and Implementation Support/Supervision Processes .............. 8 Annex 5. Beneficiary Survey Results ............................................................................. 9 Annex 6. Stakeholder Workshop Report and Results................................................... 10 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 11 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 12 Annex 9. List of Supporting Documents ...................................................................... 13 MAP A. Basic Information OECS (Grenada) Country: Grenada Project Name: Skills for Inclusive Growth Project ID: P095681 L/C/TF Number(s): IDA-45340 ICR Date: 01/27/2014 ICR Type: Core ICR Government of Lending Instrument: APL Borrower: Grenada Original Total XDR 2.10M Disbursed Amount: XDR 1,992,690.56 Commitment: Revised Amount: XDR 1.99M Environmental Category: C Implementing Agencies: Ministry of Education Co-financiers and Other External Partners: N.A. B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept 04/18/2008 Effectiveness: 07/15/2009 07/15/2009 Review: 4/11/2011 Appraisal: 09/08/2008 Restructuring(s): 3/05/2013 Mid-term Approval: 01/14/2009 07/02/2012 07/18/2012 Review: Closing: 09/01/2013 09/01/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Satisfactory Satisfactory Quality of Moderately Implementing Satisfactory Supervision: Satisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Satisfactory Performance: Satisfactory Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 10 10 Public administration- Education 50 50 Vocational training 40 40 Theme Code (as % of total Bank financing) Education for all 17 17 Education for the knowledge economy 50 50 Improving labor markets 16 16 Social safety nets 17 17 E. Bank Staff Positions At ICR At Approval Vice President: Hasan Tuluy Pamela Cox Country Director: Sophie Sirtaine Yvonne M. Tsikata Sector Manager: Reema Nayar Eduardo Velez Bustillo Project Team Leader: Harriet Nannyonjo Angela Demas ICR Team Leader: Harriet Nannyonjo ICR Primary Author: Cecilia Zanetta F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective was to support the Government of Grenada (GoG) to increase the employability of youth through public/private sector partnerships for technical and life skills training that is demand driven. ii Revised Project Development Objectives (as approved by original approving authority) Not Applicable. (a) PDO Indicators Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years The percentage of youth enrolled in the training scheme who are Indicator 1 employed fifteen months after the start of their training. Value quantitative or N/A 65% Dropped N/A Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Comments Not Applicable. This indicator was dropped under the March 2013 (incl. % Level-2 restructuring because it is was deemed not to be a good achievement) measure of the Project’s impact on the “employability of youth” since it greatly depends on exogenous factors such as market conditions. Indicator 2 Number of youth certified (NVQ/CVQ) at Level I or above Value quantitative or 0 814 500 918 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (113% and 184% level of achievement with respect to Appraisal and March 2013 restructuring targets, respectively). The target was scaled down under the March 2013 restructuring to reflect Comments the slower than expected pace of implementation in the aftermath of (incl. % the 2008 global crisis which affected, among other things, employers’ achievement) ability to offer paid internships to trainees. A total of 918 youth received NVQs/CVQs certifications (i.e., 663, 200 and 55 youth certified at Levels 1, 2 and 3, respectively). No youth had received more than one certification at the time of Project closing. (b) Intermediate Indicators Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values 1 Target Years Intermediate Outcome 1 (i.e., Component 1) To increase job-related competencies among unemployed youth through the establishment of a competitive training mechanism that supports the financing and delivery of demand-driven training 1 Changes to the Project’s Result Framework were introduced under the March 2013 restructuring. iii Indicator 1 Number of youth enrolled in the training scheme at Level 1 Value quantitative or 0 1,160 800 832 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Achieved (72% and 104% level of achievement with respect to Comments Appraisal and March 2013 restructuring targets, respectively). The (incl. % target was scaled down under the March 2013 restructuring to place achievement) more emphasis on Level 2 training and certification. Indicator 2 Number of youth enrolled in the training scheme at Level 2 Value quantitative or 0 40 80 220 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (550% and 275% level of achievement with respect to Comments Appraisal and March 2013 restructuring targets, respectively). The (incl. % target was scaled up under the March 2013 restructuring to reflect achievement) more emphasis on Level 2 training and certification in response to private sector interest in higher level training. Number of youth successfully completing a training program (i.e., Indicator 3 completing includes: graduating from the technical and life skills modules, and completing the traineeship in a public/private company). Value quantitative or 0 969 500 886 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Achieved (91% and 177% level of achievement with respect to Appraisal and March 2013 restructuring targets, respectively). The Comments target was scaled down under the March 2013 restructuring to reflect (incl. % reduction in the amount of training funded under the Project. High achievement) training completion rates (equivalent to 93%) were one of the factors contributing to the successful Project implementation. Number of youth trained in the project that are certified at Level I in Indicator 4 the occupation for which they were trained Value quantitative or 0 742 Dropped Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Comments Not Applicable. This indicator was dropped under the March 2013 (incl. % restructuring since it is a subset of the PDO indicator. achievement) Number of private sector companies that take on trainees from the Indicator 5 project Value 0 45 90 240 iv quantitative or Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (533% and 267% level of achievement with respect to Comments Appraisal and March 2013 restructuring targets, respectively). The (incl. % target was scaled up under the March 2013 restructuring to address achievement) insufficient demand in the tourism sector. Indicator 6 Percentage of trainees in sectors other than tourism Value quantitative or 0 30% 48% 55% Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (183% and 115% level of achievement with respect to Comments Appraisal and March 2013 restructuring targets, respectively). The (incl. % target was scaled up under the March 2013 restructuring to address achievement) insufficient demand in the tourism sector. Number of employed youth assessed and certified at Level I or II (i.e., Indicator 7 Prior Learning) Value quantitative or 0 N/A 50 35 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Partially Achieved (70% level of achievement with respect to March Comments 2013 restructuring target). This indicator was added under the March (incl. % 2013 restructuring, which allowed for assessment and certification of achievement) prior learning for employed youth without certification. This is part of the 918 certifications under the PDO indicator. Indicator 8 Percentage of employed youth trained only in life skills Value quantitative or 0 N/A 75 103 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (137% level of achievement with respect to March 2013 Comments restructuring target). This indicator was added under the March 2013 (incl. % restructuring, which enabled employed youth to acquire life skills achievement) training. Intermediate Outcome 2 (i.e., Component 2) To develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training Number of qualifications acquired, validated, adopted and published Indicator 9 on the web by the GCTVET Value 0 65 68 74 quantitative or v Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (114% and 109% level of achievement with respect to Appraisal and March 2013 restructuring targets, respectively). The Comments target was scaled up under the March 2013 restructuring to reflect the (incl. % faster than expected progress made by the NTA toward the acquisition, achievement) validation, formal adoption and publication of labor skills qualifications. Indicator 10 Number of certified assessors Value quantitative or 0 39 58 63 Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Exceeded (162% and 109% level of achievement with respect to Appraisal and March 2013 restructuring targets, respectively). The target was scaled up under the March 2013 restructuring to reflect the faster than expected progress made by the NTA toward the development of human resources needed to support the new labor Comments skills certification framework. Certified assessors include both training (incl. % providers and industry representatives (i.e., employers and workers). achievement) Note: This indicator was originally defined as “Number of certified assessors who have assessed at least five people during the year.” However, the qualification regarding the number of students was dropped under the restructuring as assessors under the Project typically assessed a much greater number of students (Project Restructuring Paper, March 5, 2013, p. 4). Indicator 11 CVQs introduced Value quantitative or 0 N/A Yes Yes Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Comments Achieved (100% level of achievement). This indicator was added (incl. % under the March 2013 restructuring to better capture achievement achievement) under the restructured Project Intermediate Outcome 3 (i.e., Component 3) To strengthen institutional capacity to plan, implement, and monitor training Indicator 12 Establishment of a National Training Agency (NTA) Value quantitative or 0 N/A Yes Yes Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Sept. 1, 2013 Comments Achieved. This indicator was added under the March 2013 (incl. % restructuring to better capture achievement under the restructured achievement) Project. Established under the Grenada Council for Technical and vi Vocational Education and Training Act of 2009, the NTA is the national coordinating agency for technical and vocational education and training. National Qualifications register in operation: Certification activity is Indicator 13 being recorded in the NTA MIS, and reports are available Value quantitative or 0 Yes Yes Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Comments (incl. % Achieved. achievement) Percentage of above indicators and of additional implementation Indicator 14 indicators reported on a yearly basis by the NTA Value quantitative or 0 100% 100% Qualitative) Date achieved Dec. 31, 2008 Sept. 1, 2013 Sept. 1, 2013 Comments (incl. % Achieved (100% level of achievement). achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 04/19/2009 Satisfactory Satisfactory 0.00 2 10/05/2009 Satisfactory Satisfactory 0.00 3 11/15/2009 Satisfactory Satisfactory 0.40 4 06/26/2010 Satisfactory Satisfactory 0.40 5 02/26/2011 Satisfactory Satisfactory 0.52 6 08/14/2011 Moderately Satisfactory Moderately Satisfactory 0.72 Moderately 7 04/29/2012 Unsatisfactory 1.10 Unsatisfactory Moderately 8 11/15/2012 Moderately Satisfactory 1.42 Unsatisfactory 9 03/20/2013 Moderately Satisfactory Satisfactory 2.14 10 07/24/2013 Moderately Satisfactory Satisfactory 3.19 vii H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Amendment to Schedule 2 to include “non-consultant services” as a procurement category to cover services 04/11/2011 No S S 0.52 such as the logistical service expenses for capacity building events and the printing of training materials and media campaigns. Level-Two project restructuring that introduced changes to: (i) the scope of Project activities, (ii) the Results Framework, and (iii) Project costs and financing in response to the economic 03/05/2013 No MU MS 1.63(*) downturn that had affected activities in the key sectors of the economy. In addition, the PDO indicator was revised to a stronger measure of employability over the long term. (*) As of 2/27/2013, as indicated in Restructuring Datasheet (Section 5, p. iv). viii I. Disbursement Profile ix 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. In the second half of the 2000s, shortage of skills was a major constraint for Grenada’s economic development, as the country was experiencing a turnaround in economic activity, with growth in most economic sectors led by tourism and agriculture, as well as a rebound in the manufacturing and transport sectors. This upsurge in economic activity was expected to increase the demand for skilled labor, placing significant pressure on the labor market over the following years. A number of new large projects—particularly in the hotel and tourism industries—were being planned at the time, which, when completed, would require significant numbers of trained staff. In addition, recent trends in terms of the number of and type of tourists had pointed to the need for the Grenada tourism industry to concentrate on high-margin and niche markets, which, in turn, would require specialized skills for hospitality and tourism industry workers. 2. Enhancing the efficiency of labor markets was also an important regional priority, as increasing movement of artisans and skilled persons across the Caribbean Community was seen as a crucial element to the success of the incipient Caribbean Single Market and Economy (CSME). 2 In this context, the Caribbean region had taken steps to provide the human capital foundation for economic growth and integration, including adopting a Caribbean Vocational Qualifications (CVQ) framework that sets the foundations for common standards and recognition of technical and vocational qualifications throughout the region, and alignment of technical and vocational training and education (TVET) with workplace demands. 3. Approved in 2007, the Bank-financed Organization of Eastern Caribbean States (OECS) Skills for Inclusive Growth Adaptable Program Loan (APL) aimed to provide support to OECS countries to undertake the crucial reforms needed for improving labor skills development to, in turn, meet the labor demands to sustain economic growth. St. Lucia was the first country to join the Program, 3 followed by Grenada, which was the second OECS country to meet all of the eligibility criteria. 2 The Caribbean Single Market and Economy (CSME), also known as the Caribbean Community (CARICOM) Single Market and Economy, is an integrated development strategy envisioned at the 10th Meeting of the Conference of Heads of Government of the Caribbean Community which took place in July 1989 in Grand Anse, Grenada. One of the key elements of the CSME is the free movement of labor - through measures such as removing all obstacles to intra-regional movement of skills, labor and travel, harmonizing social services (education, health, etc.), providing for the transfer of social security benefits and establishing common standards and measures for accreditation and equivalency (http://www.caricom.org). 3 Approved on April 5, 2007 by the Board of Directors of the World Bank Group. 1 4. The objective was to support the Government of Grenada (GoG) to increase the employability of youth through public/private sector partnerships for demand-driven technical and life skills training. Based on St. Lucia’s PAD, which initiated the APL series, employability was defined as the knowledge, skills and behavior of beneficiaries to help youth transition to the labor market (p. 5). Employable skills sought out by employers include basic literacy and numeracy, higher order skills such as decision making and problem solving and life skills such as dependability, positive attitude, self- confidence, punctuality, etc.). 5. The Key Outcome Indicators (KOIs) were:  The percentage of youth enrolled in the training scheme who is employed fifteen months after the start of their training (Target value: 65 percent).  The number of youth certified at Level I or above (Target value: 814). 1.3 Revised PDO and Key Indicators, and reasons/justification 6. Key Outcome Indicators were modified as follows:  Percentage of enrolled youth employed fifteen months after the start of training: This PDO indicator was dropped under the March 2013 restructuring, as it was not deemed an appropriate measure of the Project’s impact on the “employability of youth,” since employment opportunities are driven not just by the supply of labor skills but also a number of exogenous factors beyond the scope of the Project. The state of the economy, which had been particularly unfavorable during the Project’s lifetime, figures prominently among those exogenous factors.  The number of youth certified at Level I or above: This PDO indicator was retained as it was deemed a strong long-term measure of employability since certification is an asset that enhances the ability of a trainee to get a job and/or keep one. The target value was reduced from 814 to 500 youth to reflect the slower than expected pace of implementation in the aftermath of the 2008 global crisis which affected, among other things, employers’ ability to offer paid internships to trainees. 1.4 Main Beneficiaries 7. The Project’s direct beneficiaries were originally unemployed youth ages 18-30 who were out of secondary school (i.e., either drop-outs or graduates). This population group was targeted for training under the Project as it was considered to have insufficient academic qualifications to obtain employment or proceed to higher education. Indirect beneficiaries were employers, which would benefit from a labor pool with enhanced, certifiable labor skills, as well as training providers and other individuals involved in the various aspects of labor skills certification, which would benefit from expanded professional and business opportunities. 1.5 Original Components 8. Component 1 – Skills training for unemployed youth: It financed the training of unemployed youth through a competitive, demand-driven training scheme aimed at 2 providing quality, competency-based skills and life-skills training in line with labor market needs. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as Approved) 9. Component 2 – Establishing an occupational standards framework to improve the quality and value of training: It financed activities related to the adoption of an occupational standards framework that is both regionally recognized and adapted to the local context. This included the introduction of occupational standards, their vetting by local industries, and the establishment of quality assurance processes. 10. Component 3 – Institutional strengthening and project management: This component included three sub-components that provided support for: (a) institutional strengthening for the Grenada Council for Technical and Vocational Education and Training (GCTVET) and the associated National Training Agency (NTA); (b) project management; and (c) communications campaigns. 1.6 Revised Components 11. Changes in the scope of Project activities were introduced under the March 2013 restructuring, as follows:  Component 1: The following changes were introduced: (a) a reduction in the overall amount of training although with increased focus on Level II training in response to the drop in the demand of Level I skills typically associated with the construction and tourism industries, as well as the private sector’s interest in higher level training; (b) expansion of the eligibility criteria to include unemployed youth ages 16 to 40 regardless of educational attainment, and to allow labor skills certification for employed youth without certification to enable them to participate in competency- based training and/or acquire certification or life skills training; and (c) increase in the value of the trainee stipend to offset the cost of transportation and childcare.  Component 3: Activities were expanded to cover software development for program monitoring and the operational costs of the NTA to ease the fiscal burden of the Project on the Government. 1.7 Other significant changes 12. Level 2 Project Restructuring - April 2009: Categories of Eligible Expenditures were amended to include “non-consultant services” as a procurement category to cover services such as the logistical service expenses for capacity building events and the printing of training materials and media campaigns. 13. Level-2 Project Restructuring - March 2013: Other changes introduced under this restructuring included:  Results Framework: In addition to the changes to the Key Outcome Indicators described above, the Project’s Intermediate Outcome Indicators were also modified 3 (i.e., one was revised, four additional ones were added, and one was dropped). The changes are detailed in Sections F (a) and (b) in the Datasheet.  Project costs and financing: The total Project cost was reduced from US$4.48 million to US$3.72 million as a result of the changes in the Project’s scope described above as well as savings in the activities under Component 2. The financing plan was revised to eliminate contributions by the private sector under Component 1 (i.e., trainee stipends). Government financing was also reduced, with all trainee stipends and the NTA operational costs being covered by IDA financing. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 14. The Project’s technical design exhibited several strengths as described below. However, it also exhibited several weaknesses, including the failure to fully grasp the negative impact that the 2008 global financial crisis would have on the demand of skills, which was a crucial element in the Project’s demand-driven training scheme. Specifically, the strengths and weaknesses in the Project’s design included: 15. Strengths:  The Project’s overall focus and PDO were fully aligned with the OECS Country Assistance Strategy (CAS) for 2006-2009, which emphasized “supporting growth and competitiveness” as well as “reducing vulnerabilities, particularly targeting at-risk youth.”  Its innovative design incorporated state-of-the-art knowledge acquired from similar Bank-financed skills development projects targeting unemployed youth in the region; as well as findings from two Analytical and Advisory Activities (AAA) products and the 2007 World Development Report. 4 Particularly notable are: i) the emphasis on demand-driven training to ensure the alignment with market needs; ii) the incorporation of soft skills training, which had been identified elsewhere as being highly valued by both employers and trainees; and iii) incentives embedded in the Project’s design, such as the outcome-based payment schemes to training providers.  The Program’s strong regional dimension effectively built on the ongoing momentum toward regional integration, by ensuring compatibility of labor skills competency frameworks, maximizing cross-fertilization and capitalizing on the potential for economies of scale. In this regard, the so-called “Horizontal” APL was an appropriate choice of lending instrument, as it effectively supported operations in 4 Caribbean Development Bank, Department for International Development, the European Commission for Barbados and the Eastern Caribbean, the United Nations Development Programme and the World Bank (2005), Promoting an Integrated Social Protection Framework for the Caribbean, Twelfth Meeting of the Council for Human and Social Development, Caribbean Community, April 27 – 29, 2005, Georgetown, Guyana.; World Bank (2006), OECS Skills Enhancement Policy Note, Washington, DC.; World Bank (2007), World Development Report: Development and the Next Generation, World Bank, Washington, DC. 4 several countries under a common program framework, ensuring consistency at the regional level, and reducing Project preparation and supervision costs.  The Project benefited from an in-depth skills needs assessment, working in close relationship with the hotel and tourism industry.  The Bank provided robust technical support to the Borrower during preparation, particularly on the groundwork for the establishment of the NTA.  Key tasks, including the establishment of the National Training Agency (NTA) were identified as conditions of effectiveness, thus creating incentives for their timely achievement. 16. Weaknesses:  The Project design may have been overambitious, as it called for the simultaneous development of a vocational qualification framework and the establishment of a demand-driven, competency-based training mechanism. Not only both tasks were logistically challenging in and of themselves but the establishment of a competency- based training also depended to a large extent on the progress made in the development of the vocational qualification framework.  In hindsight, it is now clear that Project preparation exhibited an important shortcoming, namely, an incomplete risk assessment. While the full impact of the 2008 global financial collapse on Grenada’s economy could not have been anticipated, it is notable that the issue was ignored during appraisal. There is no mention of the risk of an economic downturn and its potential negative impact on the demand for paid interns under the Project’s risk assessment, even though at appraisal (i.e., the PAD is dated December 16, 2008) and approval (i.e., January 14, 2009) the severity of the situation was already evident. 2.2 Implementation 17. Although the Project was implemented under unusually adverse macroeconomic conditions, it benefitted from the sustained political support from top Government authorities under three different administrations and an effective project management. The specific factors affecting implementation can be summarized as follows: Factors outside the control of the Government and the implementing agency: 18. Unfavorable economic environment: The impact of the global financial crisis on Grenada’s highly dependent economy was severe. From 2009 to 2012 real GDP growth averaged -1.5 percent, with the hotels and restaurants sector, a proxy for tourism, declining on average by 4.7 percent during the same period. The unemployment rate doubled between 2008 and 2012, while poverty, which was already high at 37 percent, worsened significantly. 19. Sharp drop in the demand for labor skills: Although clearly desirable, having employers pay for the internships under the Project was an ambitious proposition, as 5 employers in Grenada are used to getting interns at no cost from other training programs, including that of the community college. 5 However, requiring employers to pay for the internships was considered feasible in view of the shortage of skills in the tourism and construction sectors that was anticipated at the time of appraisal. Ultimately, this demand did not materialize as the rippling effects of the global crisis reached Grenada’s economy and none of the new hotels that had been identified during appraisal materialized. Thus, the demand for employers-paid internships virtually vanished. This became a substantial stumbling block for the implementation of the training component before the 2013 restructuring under which stipends were financed with IDA proceeds. 20. Tight fiscal constraints: The Government’s finances were also negatively affected by the crisis, which affected its ability to finance some of the Project’s costs as anticipated, particularly the salaries and other operation costs of the newly established NTA. This constraint was also addressed under the 2013 restructuring, which made the NTA’s operational expenses eligible for Project financing. Factors Generally Subject to Government Control 21. Sustained support on the part of authorities: While the Project was prepared under one administration and implemented under two different administrations— including one from the opposition party 6 —the Project’s design of providing demand- driven technical and life skills training among vulnerable youth and developing the corresponding institutional framework remained unchanged. The authorities’ strong support for the Project was also demonstrated by the passage of enabling legislation establishing the country’s vocational qualification framework, including the creation of the NTA. 22. Collaboration with the Ministry of Youth Empowerment and Sports (MOYES): Capitalizing on the fact that both ministries operated youth training programs focusing on getting unemployed youth into the workforce, the MOE partnered with the MOYES to facilitate the implementation of the Project. A Memorandum of Understanding (MOU) was put in place between the MOYES and the NTA in February 2012, by which the MOYES would cover the stipend of some of the trainees under this Project during their traineeship (instead of employers), the NTA would support the MOYES in its training activities and certification of trainees, and both agencies would collaborate in recruitment of trainees for the project. This collaboration was critical in resolving the major bottleneck impeding project implementation—i.e., employers’ reluctance to pay for the internships given the sluggish economic condition. 5 The T.A. Marryshow Community College. 6 The New National Party (NNP), which had ruled Grenada since 1995, was replaced by the opposition party, the National Democratic Congress, in July 2008, which remained in power until Feb. 2013, when the NNP took power once again. 6 Factors Generally Subject to Implementing Agency Control 23. Key changes were introduced to the training program: Ultimately, the Project was successfully implemented as a result of the flexibility and proactive attitude exhibited by the PCU and NTA, which were able to effectively expand the focus of the training to other sectors beyond tourism that were not as hardly hit by the global financial crisis and to accommodate changes demanded by employers, such as the training of employed, more educated youth. In addition, the PCU and NTA succeeded in significantly expanding the recruitment of employers providing internships, which, in turn, translated in a larger number of graduates than anticipated at the time of the 2013 restructuring. Changes were also introduced to the training delivery, including condensing the training into a shorter time frame (i.e. training for more hours per day) while maintaining the same overall prescribed number of hours. This led to lower overall stipend expenses (given the reduced number of days) as well as the probability of trainees dropping out due to personal circumstances and other factors. In addition, the stipend amount was increased to more closely reflect trainees’ transportation costs and payment was expedited. These changes, which were reflected in the 2013 restructuring, were critical to the ultimately successful implementation of the Project, as they led to higher completion and certification rates and therefore reduced overall unit cost of training. 24. Ongoing collaboration with St. Lucia under the APL framework: Collaboration between St. Lucia and Grenada helped both countries learn from each other and move faster with implementation, as they routinely shared documents and experiences. For example, Grenada’s Life Skills document was an adaptation of the one from St. Lucia, while St. Lucia’s quality assurance documents were based on those of Grenada. This collaboration also included the joint development of the information monitoring system (IMS). Collaboration between the two countries is expected to continue after Project closing as a result of the strong links that have developed between professional staff in both countries. 25. Strong collaboration with CARICOM countries and agencies: The PCU and NTA worked closely with their counterpart in Saint Lucia, other countries in the region, and the regional organizations, such as the Caribbean Association of National Training Agencies (CANTA), as they developed labor certification standards and certification of key quality assurance staff. This close collaboration led, among others, to NTA’s accreditation by CANTA, which enables it to issue CVQs, effectively expanding the possibilities for employment of the Project graduates to the entire CARICOM region. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 26. Design: The Project’s Result Framework (PRF) was robust and included a battery of outcome and output indicators that were largely in alignment with the PDOs and activities supported under the operation. However, it became apparent during implementation that the employment based indicator was not effective in capturing the Project’s impact, as employment can be substantially affected –both positively and negatively – by factors outside the orbit of the Project. . Moreover, the basis for selecting 7 a 65 percent target value are not clear, particularly given the country’s lack of adequate employment data that would have been needed to make a robust data-informed employment projection. 27. Implementation: The M&E system went beyond the PRF to include gender, drop- out rates, and geographic location by parish. It also served as the embryo of the NTA’s own M&E system. Making the NTA responsible for the Project’s M&E was a key factor that permitted to capitalize on this opportunity, which was particularly important in the context of Grenada’s limited institutional resources. A more ambitious information management system (IMS) to support the NTA’s M&E needs has also been designed under the Project. 28. Utilization: The M&E system made it possible to closely monitor the progress being made under the project and the achievement of expected results. It also served as the first register of labor skills certificates being issued. The IMS that has been established could provide the basis for a regional labor-skills certification registry, as was designed concomitantly with that of St. Lucia. 2.4 Safeguard and Fiduciary Compliance 29. Environmental and social safeguards: The project has an environmental category C rating given that a neutral environmental impact was expected. No social safeguards were triggered either. 30. Financial Management (FM): FM was well managed. Audits were submitted in time. 31. Procurement: In mid-2010, there was weak coordination between the Project Manager and the Procurement team. A Procurement Post Review carried out in March 2010 revealed that some procurement procedures were handled by the Project Manager and documents were not filed in the PCU's procurement files but at the MOE office instead. Actions were subsequently implemented to improve coordination, and proper filing of documentation. Procurement was in compliance with Bank policies and procedural requirements. 2.5 Post-completion Operation/Next Phase 32. Key positions within the PCU and NTA are permanent staff within the MoE, which ensures that the management capacity built under the Project remains within Grenada’s public sector. In addition, in 2013 the NTA received its own budget allocation for salaries, with operational expenses being financed under the MoE’s budget. 33. The training model supported under the Project has been replicated under the New Imani Program, one of the flagship programs of the Government of Grenada that is being implemented under the MoY. The New Program is considered an improved version of the old Imani Program that was launched by the previous NNP Administration in 2002, 8 as it builds on the training model introduced under the Project that places greater emphasis on certification and employment. Its key objectives include: assist the trainees to develop positive values and attitudes, provide an opportunity for young people to acquire employable skills, and provide an opportunity for young people to be adequately prepared for the job market through on-the-job-training All Imani trainees will receive basic skills training building upon the training curricula developed under the Project. In addition, under the Multi-Sector Youth Skills Training Program (MYSTP), which is one of the five training modalities under the New Imani Program, trainees will receive Level 1 and 2 skills training that is fully aligned with the labor skills competencies that have already been adopted by the NTA. All MYSTP trainees will be required to apply for certification to National Occupational Standards to receive a Caribbean Vocational Qualification – CVQ (2013 Manifesto, p. 34). A total of EC$15.9 million have been allocated to provide training to 2,000 youth under the New Imani Program in the 2013 Budget (2013 Budget Statement, p. 32). 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Substantial 34. Objectives: The relevance of the PDO is deemed to be Substantial. As stated in the New National Party’s 2013 Manifesto, job creation and skills development supported by youth development programs is one of the top priorities of the GoG (2013 Manifesto, p. 2). The Program is also consistent with the current Bank’s Regional Partnership Strategy (RPS) for OECS Countries 2010-2014 (Report No. 53762-LAC, May 3, 2010), which has two objectives: i) to build resilience; and ii) enhance competitiveness and stimulate growth over the medium term. By focusing on the development of labor skills among vulnerable youth, the Program contributed directly to RPS objectives. In addition, the PDO is fully aligned with the current OECS Education Sector Strategy 2012-2021, 7 which includes as one of its seven strategic imperatives, the need to provide opportunities for all TVET learners to develop the knowledge, skills and attitudes to enable them to progress to further education and training and productive employment through the development of a competency-based curricula linked to the CVQ framework established across secondary and tertiary education and in all other educational settings in work and in community. Likewise, the Program is fully aligned with the World Bank Education Strategy 2020, which focuses on learning for all, beyond schooling is in recognition that the level of skills in a workforce predicts economic growth rates far better than do average schooling levels. 35. Design and implementation: The relevance of the Project’s design is deemed to be Substantial. With a focus on unemployed youth, skills required by the labor market, paid internships, the original design reflected the conditions of robust economic growth 7 The OECS Education Sector Strategy 2012-2021 reflects the overarching education strategy of the OECS and is used by Member States to align their national Strategies and Plans. 9 prevailing at preparation. At restructuring, the continued relevance of the Project’s design was ensured during the economic downturn by increasing the focus on Level II training, which leads to higher technical skill levels than Level I training, in response to the private sector’s interest in higher level training, expanding the target population to include more educated employed youth, and allowing for assessment and certification for prior learning for employed youth without certification. Presently, the Project’s design continues to be highly relevant given its emphasis not just on training provision per se but also on the development of the institutional framework and management skills needed to support the development and implementation a regionally relevant labor skills competency framework. This is denoted by the decision of the GoG to continue with a similar youth certification-based training program under the MOYES’s Imani Program and the continuous development of the NTA with an emphasis on CVQs. 3.2 Achievement of Project Development Objectives Rating: Substantial The PDO was to support the Government of Grenada to increase the employability of youth through public/private sector partnerships for demand driven technical and life skills training. PDO Indicator: Number of youth certified (NVQ/CVQ) at Level 1 or above. PDO achievement is considered to be Substantial, as exhibited by substantial achievements toward increasing the employability of youth through public/private sector partnerships for demand-driven technical and life skills training. The number of youth awarded NVQs/CVQs 8 certification surpassed targets set at both appraisal and restructuring. A total of 918 youth were certified at Level 1 or above, exceeding the target of 500 defined at restructuring (184 percent level of achievement) and 814 defined at appraisal (113 percent level of achievement). It is important to emphasize that certification outcomes capture not just training outputs (i.e., the number of trainees that received soft and technical training and completed their internship) but achievement of industry-vetted, independently certified, competency-based skills that are recognized both at the national and regional levels. Prior to certification, all trainees regardless of level of training, were assessed on technical skills and a common core of employable skills 9 , including basic literacy and numeracy, higher order skills such as decision making and 8 The Caribbean Vocational Qualification (CVQ) is an award that represents achievement of a set of competencies which define core work practices of an occupational area, consistent with the levels articulated within the regional qualifications framework. To earn an award, trainees must demonstrate competence in reaching CARICOM-approved occupational standards developed by practitioners and employers (i.e. the industry experts). 9 There was a common core of life skills for all occupational fields irrespective of the level of training. This means that those trained for example in Housekeeping were required to go through the same life skills training as those under Data Operations. 10 problem solving and life skills such as dependability, positive attitude, self- confidence, punctuality and other aptitudes required to function effectively on the job. The purpose of the assessment was to determine whether trainees have met the requirements outlined in the competency standards for the qualification they seek. This assessment was based on job performance during internship and on a portfolio compiled by the trainee to demonstrate mastery of skills. Those who complete the training and pass the assessment are then certified. Focus group discussions with employers confirmed the importance, demand and appreciation for life skills. Some employers who had hosted trainees for internships requested support to have their other employees trained in life-skills, an indication of the level of their appreciation of the life skills training provided under the Project. 36. The large number of trainees certified was enabled by several project activities: First, a total of 1,052 trainees who enrolled in training, including 832 at Level 1, and 220 at Level 2. Of these, 886 youth successfully completed the training program, including the on-the-job internship, exceeding the target of 500 defined at restructuring (177 percent level of achievement and close to 969 defined at appraisal (91percent level of achievement). Second, the 240 private companies that provided internships to trainees, amply exceeding the pre- and post-restructuring targets (i.e., 45 and 90 companies, respectively), of which 55 percent were in industries not related to tourism (compared to the 30 percent and 48 percent pre- and post-restructuring targets, respectively). Third, a robust quality assurance system put in place to support the newly established labor skills competency framework. This system, including a large number of staff trained under the Project, facilitated trainee assessment which provided the basis for certification. A total of 37 facility auditors, 30 internal verifiers, 39 external verifiers, 63 assessors, and three master assessors have been trained in competency-based education and training methodologies and certified under the Project. This system is supported by corresponding manuals and guidelines, 10 and facilitated accreditation of training providers and suitability of facilities, as well as assessment and certification of a large number of youth, including those from the wider national training system. In addition to the trainees under the Project, a total of 35 employed youth had their labor skills assessed and certified (i.e., prior learning assessment) and another 63 students from the national community college were also assessed and certified. 37. Forth, the regionally recognized occupational standards framework that will serve to permanently improve the quality and value of training in Grenada. In May 2013, NTA received CVQ granting status from CARICOM, which is an independent endorsement of the robustness of Grenada’s newly established occupational standards framework, enhancing relevance, quality, and increasing the possibility of employment outside Grenada. In addition, a total of 74 qualifications have been adopted by the GCTVET, compared to a target of 68, which have been validated, adopted and published on the web by the GCTVET (compared to an end-target of 68 qualifications). These qualifications helped to diversify the fields of certification based on indications from the labor market, 10 These documents include: i) Learner’s Handbook, ii) Center Approval Guidelines, iii) Assessment Guidelines, iv) Quality Assurance Manual, and v) Accreditation Manual, vi) Procedures Manual. 11 and matched availability of placements for traineeships that were an integral part of the training program, and would serve to increase employability of youth beyond Grenada. 38. The Project also developed the institutional capacity to plan, implement, monitor training, and fulfill the ongoing operational responsibilities concerning the development and adoption of occupational standards, including maintaining a National Qualification Register, recording certification activity, monitor labor market trends and manage the quality assurance system, including trainee assessment. The Grenada Council for Technical and Vocational Education and Training (GCTVET) was established by law in 2009, with the National Training Agency being created as the operational arm of the Council. The NTA is staffed with a small core of technical professionals to ensure that the GoG has the necessary institutional capacity to support the provision of training, organize assessment, and the establishment of the labor skills competency framework both under and beyond the Project. 39. By focusing on the development of skills—both life skills and technical skills— through a demand-driven training program, the Project contributed directly to the PDO achievement of improving the employability of youth. Life skills training provided under the Project focused on the development of socio-emotional or non-cognitive skills (also known as “life skills” or “soft skills”) related to behaviors and attitudes, such as dependability, positive attitude, team work, conflict resolution, etc. Recent evidence from econometric and qualitative analyses point to the importance of non-cognitive skills as determinants of success in the labor market and workers’ productivity. 11 Moreover, recent studies indicate that employers in the Latin America and Caribbean region are placing increasing value on life skills vis-à-vis technical skills. 12 Similarly, by requiring training providers to demonstrate employers’ commitment to provide on-the-job internships as part of their proposal, the Project ensured that employers played a role in determining the allocation of training resources among the various occupations and economic sectors, thus ensuring the relevance of the technical skills developed under the Project. 3.3 Efficiency Rating: Substantial 40. The Project’s efficiency is considered to be Substantial. Project funds were used efficiently as illustrated by the higher than originally expected certification outcomes within a small budget. The total number of trainees certified was 918 compared to 814 and 500 pre- and post-restructuring targets (i.e., 113 percent and 184 percent respectively) despite a 24 percent reduction in overall Project costs, from US$4.48 to US$3.39 million, and a 41 percent reduction in component 1 (i.e., from US$2.08 to 11 See Heckman, Stixrud and Urzua, 2006;Fazio, 2011. 12 Using data from 1,200 firms in the LAC region, Bassi et al. (2012) showed that employers value non- cognitive skills more than cognitive ones, with the results being statistically robust across countries, economic sectors, and occupations. These results are consistent with those rendered by studies recently conducted in Peru and the Dominican Republic (World Bank, 2011; Martinez, 2011). 12 US$1.22 million). Out of the 918 certifications 13, 255 were at Level 2 and above, and this was unexpected since only 40 trainees were expected to be enrolled at Level 2. This was achieved by condensing training into shorter periods of time (i.e., more training sessions per day), as shortening the overall duration of training reduced both overall stipend expenses and the probability of trainees dropping out due to personal circumstances and other factors. Overall, unit training costs were 23 percent lower than anticipated (i.e. US$1,657.71 per trainee anticipated at appraisal versus and actual unit cost of US$1,277). Unit training costs were lower, in spite of higher daily stipends and transportation allowances and an increase in the cost of training materials from the VAT, since on average there were fewer than the expected number of training days per course. This meant that the total amount of funds paid for stipends and transportation was lower, on average, than expected. 41. The Project’s substantial economic efficiency is also illustrated by the results from an economic analysis of the training component, which yields a 13.2 percent Internal Economic Rate of Return (IERR) 14. The economic analysis also found that an increase in average wage income of 10 percent is needed to ensure a positive rate of return with a discount rate of 7.5 percent, compared to 20 percent at appraisal. These results suggest that the training component is more efficient than at appraisal even under rather unfavorable economic conditions currently prevailing in Grenada as a result of the global financial and economic crisis. In addition, the economic cost of certification per student (or the marginal economic cost of certification) is 48 percent lower than anticipated (i.e., US$2,555.00 per certification anticipated at appraisal versus an actual cost of US$1,329.00). 15 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 42. The Project’s relevance, efficacy and efficiency are considered to be Substantial supporting an overall Satisfactory rating. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 43. As originally envisioned, the training supported under the Project targeted solely uneducated, unemployed youth. While this population group continued to be the main beneficiary group, as part of the restructuring the targeting was expanded to include more 13 All these youth had received one certification by the time of Project closing. 14 No IERR was calculated at appraisal 15 The marginal economic cost of certification is calculated based on the estimated and actual cost of Component 1 (USD2.08 million and USD1.22 million, respectively) and the number of estimated and actual number of certifications awarded under the Project (814 and 918 NVQs/CVQs, respectively). 13 educated, employed youth to address growing unemployment even among more educated youth and to respond to the interest expressed by employers in training their current employees, particularly on softer skills. 44. In terms of gender, the larger majority of trainees were female, accounting for 76 percent. (b) Institutional Change/Strengthening 45. The establishment by Parliament of the GCTVET and the NTA in 2009 was among the most notable Project’s outputs, particularly in terms of its institutional impact ensured that Grenada has the necessary institutional capacity to support the provision of competency based skills training both under and beyond the Project. The NTA is the fourth entity to be accredited to issue CVQs, expanding their employment opportunities throughout the CARICOM region. Regional cooperation with Jamaica, Trinidad and Tobago, St. Lucia and regional agencies, including CANTA, was another important Project outcome. (c) Other Unintended Outcomes and Impacts (positive or negative) 46. The Project contributed toward strengthening collaboration and coordination between government agencies, in particular the MoE, MOYES and the NTA. 47. The Project has also contributed toward South-to-South collaboration. For example, Antigua hired s Grenada’s NTA to provide guidance on setting up their quality assurance and procedures for certification, which was considered the most useful guidance it had received in the sector. St. Vincent and the Grenadines sent a three-person team to Grenada to learn from their experience and receive guidance as they prepared for certification and CVQ granting status. St. Kitts has also expressed interest in learning from the Grenada’s experience. 4. Assessment of Risk to Development Outcome Rating: Moderate 48. The GoG has taken actions to ensure the continuation of the certification-based training program sponsored under the Project and the functioning of the NTA. However, the GoG faces short- and medium-term challenges as well as opportunities to capitalize on the full potential impact these initiatives can have on enhancing Grenada’s economic and human capital development in the long run. Specifically: Risks to Development Outcomes a) The GoG needs to ensure the continuous efficacy and efficiency of the competency- based training implemented under the Project. The New Imani Program has undergone some modifications that could make it less effective and less efficient. The most important modification has been decreasing intensity of training, expanding it over a longer time span. Given the Project’s implementation experience, this 14 modification is likely to increase the drop-out rate. More importantly, it can introduce distortions in the labor market by unintentionally discouraging employers from hiring permanently. b) To ensure the sustainability of the newly established vocational qualifications framework over the medium and long terms, it is vital to ensure the financial sustainability of the NTA. The GCTVET Act does not ensure the NTA’s financial independence, as it does not considers any financial mechanisms apart from the discretionary budget and other moneys assigned by Parliament on an annual basis. The current budget allocation covers mainly salaries, and the approval of operational expenditures is left at the discretion of the MoE, which makes it difficult for the agency to engage in any meaningful long-term planning. It is important to continue to provide support for certification activity in the medium term through programs such as the New Imani Program until the demand for certification is more firmly established. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (QAE) Rating: Moderately Satisfactory 49. The Bank’s team prepared a technically robust operation, which reflected the state-of-art knowledge in active labor markets programs targeting youth as well as sound operation design that put in place a system of incentives to support the demand-driven nature of the training and the result-oriented behavior of the various actors. In addition, the Bank team worked closed with various stakeholders, particularly employers from the hotel and tourism sectors, to identified expected shortages in labor skills to be addressed under the Project. In addition, the establishment of the NTA and its enabling legislation were the result of various modalities of Bank support. For example, the Bank provided support for a MoE official to attend a summer program in Professional Education at Harvard, where a first draft of the legislation was prepared as an academic exercise. The drafting of the legislation, in particular aspects related to the NTA’s organizational structure, was subsequently supported with Bank-financed consultants as part of Project preparation. Bank support was also a key factor in fostering horizontal fertilization among ECC, particularly Saint Lucia, as well as the interaction of Grenada’s officials with regional organizations, such as CANTA. 50. The PRF exhibited some weaknesses and the macroeconomic risk derived from the collapse of world financial markets that took place in October 2008 was not identified during appraisal and, as a result, the potential negative impact of a global economic downturn on Grenada’s tourism sector and the associated drop in the demand for skills, was overlooked. (b) Quality of Supervision Rating: Moderately Satisfactory 15 51. The Project was under close supervision, with regular supervision missions that included field visits to training providers and employers throughout the country, which helped identify the challenges being faced at various stages of project implementation. The Bank worked closely with the GoG and provided the flexibility needed to achieve a successful implementation of Project activities, including processing a Level 2 restructuring to adapt the Project to the challenging economic landscape that ensued as a result of the global financial crisis. The decision to restructure the Project was appropriate, as it enabled it to be fully implemented in the six months prior to its closing, with almost 50 percent of overall disbursements and 74 percent of all labor skills certifications supported under the Project taking place during the last six-month period. 16 It would have been beneficial, however, to process this restructuring earlier in the Project’s lifetime, as it became apparent as early as 2010 that potential employers were not willing or able to offer paid internships to trainees under depressed economic conditions. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 52. Overall Bank performance is considered Moderately Satisfactory, given its moderately satisfactory performance at quality at entry and supervision. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 53. Government performance is deemed Satisfactory. The Project received the full support from GoG authorities during three different administrations. Although the Government’s financial contribution to the Project was reduced due to the tight fiscal constraints that resulted from the economic recession, adequate technical and financial resources were assigned to the Project during its lifespan. A strong inter-government coordination was exhibited across the MoE, MOYES and the NTA, which was crucial to the Project’s successful implementation, including MOYES’ decision to pay for the trainees stipends until the Project was restructured. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 54. The performance of the PCU is considered Satisfactory. It was actively involved in Project preparation as well as its implementation. Throughout the Project’s lifetime, the PCU exhibited strong ownership of the Project, willingness to learn by doing, and the ability and will to reflect upon the implementation experience and make the necessary 16 A total of 245 and 673 youth were certified at Level I or above in 2012 and 2013, respectively, which represent 26 and 74 percent of all certifications issued under the Project (NTA’s Monitoring and Evaluation Report, August 14, 2013). 16 adjustments. The PCU provided effective support to NTA, responding swiftly to all its needs. It also played a central role facilitating coordination between the MoE, MOYES and the NTA. 55. The performance of the NTA is also deemed Satisfactory. Top authorities were actively involved in Project implementation. They fully capitalized on the financial and technical support provided by the Project to build the newly established agency from scratch together with the vocational framework. The NTA effectively implemented a robust information system, which allowed it to monitor both implementation progress as well as results. 56. Despite all obstacles, including an unusually harsh economic environment, changes in administration, and a delayed restructuring, both the PCU and the NTA continued to forge forward. Stability within the PCU’s and NTA’s management brought much needed continuity in the overall’s Project management. It generously shared its implementation experience with other countries in LAC and other regions. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 57. Overall Borrower performance is considered Satisfactory, given the Government’s satisfactory level of performance and the outstanding performance of the PCU and NTA. 6. Lessons Learned a) Lessons of wide general application 58. Measuring the impact of skills building interventions on employability poses a difficult challenge. This is particularly true in projects that, by virtue of their size and other institutional and technical limitations, do not lend themselves to randomized-type evaluations. While employment is undoubtedly the ultimate measure of employability, it is also affected by factors external to the intervention, most notably economic conditions. Thus, in the design of skills development projects such as this one, it is important to realize that at best these programs can enhance the supply of labor skills within the existing, exogenously determined demand for labor skills. 59. Focusing on soft-skills can provide an effective focus of training programs. Both employers and trainees appear to value them greatly. Under the Project, soft skills were emphasized throughout the training, including the technical training and the internship. 60. The development of a competency certification framework is a transversal effort at the intercept of the orbit several line ministries. As illustrated by the Project implementation experience, its success depended on the collaboration between several entities, primarily the MOE, MOYES and the NTA, and, to a lesser degree, the MoL and the MoF. 17 c) Project-specific lessons 61. All government-sponsored internship programs operating in Grenada should have consistent sets of incentives. As originally conceived, internships under the Project required employers to pay for the stipend. This requirement, together with the fact that it targeted less educated, unemployed youth, obviously put the Project at a disadvantage with respect to other publicly sponsored internship programs, such as the one from the community college, which seeks to place college-educated trainees in internships at no cost to employers. 62. There are opportunities to eliminate duplications and ensure the most effective and efficient use of the country’s limited institutional and technical resources. For example, career counseling under skills development programs could be strengthened by utilizing the already installed capacity within the MoE and individual schools. Likewise, performance under the Project by individual training providers could be used as selection criteria under other skills development projects. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Co-financiers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 18 Annex 1. Project Costs and Financing (a) Project Cost by Component (in Million equivalent) Appraisal Percentage Actual Component Project Cost Summary Estimate (US$) of (US$) million Appraisal % million 1. Skills Training for Unemployed Youth 1. Skills Training 2.08 1.22 59 2. Improving the Policy Framework for Delivering Training 1. Introduction of Standards, Accreditations & Quality Assurance, Certification 0.40 0.25 63 3. Project Management and Institutional Strengthening 1. Institutional Strengthening 0.42 0.28 67 2. Project Management 1.05 1.45 138 3. Public Awareness 0.29 0.19 65 Total BASELINE COST 4.24 3.39 80 Physical Contingencies 0.04 0 Price Contingencies 0.23 0.00 Total PROJECT COST 4.48 3.39 76 (b) Financing Source of Funds Appraisal Actual Percentage of USD in USD Appraisal Millions Million Borrower 0.90 0.32 36 International Development Association 3.00 3.06 102 (IDA) Local sources of Borrowing Country 0.60 0.01 2 (private sector) Total 4.50 3.39 76 19 Annex 2. Outputs by Component Component 1 – Skills Training for Unemployed Youth Intermediate Outcome 1: To increase job-related competencies among unemployed youth through the establishment of a competitive training mechanism that supports the financing and delivery of demand-driven training a. Youth labor-skills competency based training - Enrolled 1052 (24% males, 76% females) - 257 males - 795 females - 7 Parishes (St. Mark, St. Andrew, St. David, St. George, St. John, Carriacou, St. Patrick) - 6 Sectors (Hospitality & Tourism, Agriculture, Construction, ICT, Fashion, Furniture) - 53 Programs across 16 Occupational areas. 42 Level 1s and 11 Level 2s. - Food and Beverage Restaurant Service: Server (4 programs; 4 Level 1) - Data Operations (18 programs; 11 Level 1 and 7 Level 2) - General Office Administration (5 programs; 5 Level 1) - Industrial Security Operations (2 programs; 2 Level 1) - Electrical Installation (2 programs; 1 Level 1 and 1 Level 2) - General Cosmetology (5 programs; 4 Level 1 and 1 Level 2) - Furniture Making (1 program; 1 Level 1) - Garment Production (5 programs; 5 Level 1) - Printing and Graphics Arts (1 program; 1 Level 1) - Commercial Food Preparation: Cookery (1 program; 1 Level 1) - Crop Production (1 program; 1 Level 1) - Livestock Rearing (1 program; 1 Level 1) - General Construction (3 programs; 3 Level 1) - Cricket Pitch & Field Maintenance (1 program; 1 Level 1) - House Keeping: Room Attendant (1 program; 1 Level 1) 1. - Fashion Design (2 programs; 2 Level 2) Component 2 – Occupational Standards Framework Intermediate Outcome 2: To develop an improved policy framework for delivering training by enhancing OECS collaboration in training and introducing occupational standards to increase quality and value of training a. Number of qualifications 2. Number of qualifications acquired, validated, adopted and published on the web by the GCTVET (Target value: 68) Achieved. A total of 74 qualifications have been adopted by the GCTVET. b. Human resources supporting the labor skills competency framework (i.e., assessors and others). 3. Number of certified assessors (Target value: 63) Achieved. A total of 63 assessors have been certified c. Regional certification 4. Introduction of CVQs Achieved. The NTA has been accredited at the regional level and can issue CVQs. 20 Component 3 – Institutional strengthening Intermediate Outcome 3: To strengthen institutional capacity to plan, implement, and monitor training. a. Establishment of a National Training Agency (NTA) 5. Establishment of a National Training Agency (NTA) Achieved. The NTA was established by an Act of Parliament in 2009. b. Operationalization of a certification system 6. National Qualifications register in operation Achieved. The NTA maintains a National Qualification Register, in which certification activity is being recorded and reports are available c. To strengthen institutional capacity to better implement, monitor and plan training through the strengthening of the project implementing agencies 7. Percentage of above indicators and of additional implementation indicators reported on a yearly basis by the NTA Achieved. The NTA routinely monitors and reports on 100% of the above indicators. 21 Annex 3. Economic and Financial Analysis 1. The Project’s efficiency is considered to be Substantial. Project funds were used efficiently as illustrated by the higher than originally expected certification outcomes within a small budget. The total number of trainees certified was 918 compared to 814 and 500 pre- and post-restructuring targets (i.e., 113 percent and 184 percent respectively) despite a 24 percent reduction in overall Project costs, from US$4.48 to US$3.39 million, and a 41 percent reduction in component 1 (i.e., from US$2.08 to US$1.22 million). Out of the 918 certifications, 255 were at Level 2 and above, and this was unexpected since only 40 trainees were expected to be enrolled at Level 2. This was achieved by condensing training into shorter periods of time (i.e., more training sessions per day), as shortening the overall duration of training reduced both overall stipend expenses and the probability of trainees dropping out due to personal circumstances and other factors. Overall, unit training costs were 23 percent lower than anticipated (i.e. US$1,657.71 per trainee anticipated at appraisal versus and actual unit cost of US$1,277). Unit training costs were lower, in spite of higher daily stipends and transportation allowances and an increase in the cost of training materials from the VAT, since on average there were fewer than the expected number of training days per course. This meant that the total amount of funds paid for stipends and transportation was lower, on average, than expected. 2. The Project’s substantial economic efficiency is also illustrated by the results from an economic analysis of the training component, which yields a 13.2 percent Internal Economic Rate of Return (IERR) 17 . The economic cost of certification per student (or the marginal economic cost of certification) is 48 percent lower than anticipated (i.e., US$2,555.00 per certification anticipated at appraisal compared to the final economic cost of US$1,329.00). 18 3. The analysis utilizes the same assumptions in terms of probability of being employed and wages of the target population estimated at appraisal. It assumed that, over the next ten years, those who received training and certification under the Project will have a 10 percent increase in the probability of employment relative to their peers who did not receive any training. It is assumed that the impact will be 50 percent higher among those that received Level 2 training and certification (i.e., 15 percent increase). Having obtained regionally recognized CVQs, trainees now have expanded employment opportunities at regional and international level, thus effectively increasing the probability of being employed. Given that the target population has lower employment rates in Grenada than in other CARICOM countries, the increase can be expected to be substantial. 17 No IERR was calculated at appraisal. 18 The marginal economic cost of certification is calculated based on the estimated and actual cost of Component 1 (USD2.08 million and USD1.22 million, respectively) and the number of estimated and actual number of certifications awarded under the Project (814 and 918 NVQs/CVQs, respectively). 22 4. The analysis also assumes a 15 percent and 22.5 percent increase in per capita wages among those that received training and certification at Levels 1 and 2, respectively. Research in other countries indicates that competency-based training and certification can yield comparable increases in wage. For example, in its analysis of the Mexican National System of Competency Standards, Skills Testing, Calderón (2000) findings suggest a 17 percent improvement in hourly wages, relative to not having been trained 19. 5. The benefits from training and certification derive from the increase in wage mass for the target group that can be attributed to the training program taking into account: i) unemployed persons who would otherwise have no wages; and ii) employed persons who would receive higher wages. Results show an IERR of 13.2 percent for Component 1 alone, and using a much lower employment rate (i.e. 10 percent compared to the trainee employment rate of 24 percent reported in September 2013). 19 Calderón, A. (2000), The Mexican National System of Competency Standards, Skills Testing and Certification: Measuring the benefits of on the job training and discussing the economic rationale for not passing the government the bill for the certification of workers skills, Paper prepared for the Conference “Lifelong Learning as an Affordable Investment” Hosted by the OECD and Canada, Montreal December 6- 8, 2000. 23 Table 1. Summary of Economic Analysis PARAMETERS Certifications awarded under the Project NVQs/CVQs awarded among those enrolled in training under Component 1 820 - Number of Level 1 NVQs/CVQs certifications awarded 649 - Number of Level 2 NVQs/CVQs certifications awarded 171 Impact on the probability of employment Employment rate of target population (without the Project) 40% Increase in the probability of employment due to Project participation - Among trainees that obtained Level 1 NVQs/CVQs certifications 10.0% - Among trainees that obtained Level 2 NVQs/CVQs certifications 15.0% Additional number of participants employed 36 - Among trainees that obtained Level 1 NVQs/CVQs certifications 26 - Among trainees that obtained Level 2 NVQs/CVQs certifications 10 Impact on earnings Monthly per capita wage among target population (ECD) 830 Annual per capita wage among target population (ECD) 9,960 Annual per capita wage among target population (USD) 3,689 Increase in per capita wages due to Project participation (%) - Among trainees that obtained Level 1 NVQs/CVQs certifications 10% - Among trainees that obtained Level 2 NVQs/CVQs certifications 15.0% Increase in per capita annual wages due to Project participation (US$) - Among trainees that obtained Level 1 NVQs/CVQs certifications 369 - Among trainees that obtained Level 2 NVQs/CVQs certifications 553 Number of years of Project benefit 10 BENEFITS Annual wage mass from trainees without the Project 1,209,956 Increase in wage mass from increased employment 148,865 - Among trainees that obtained Level 1 NVQs/CVQs certifications 105,340 - Among trainees that obtained Level 2 NVQs/CVQs certifications 43,525 Increase in wage mass among already employed 133,612 - Among trainees that obtained Level 1 NVQs/CVQs certifications 95,764 - Among trainees that obtained Level 2 NVQs/CVQs certifications 37,848 Total increase in annual wage mass due to Project 282,477 COSTS Training component (US$) 1,520,000 IERR 13.2% 6. A sensitivity analysis was also conducted to assess the efficiency of the training under different scenarios, as the Project’s actual impact on the probability of being employed and increase in wages cannot yet be determined due to the fact that most trainees completed their training in August 2013. As shown in Table 2, an increase in average wage income of 10 percent is needed to ensure a positive rate of return with a discount rate of 7.5 percent, compared to 20 percent at appraisal. These results suggest 24 that the training component is more efficient than at appraisal even under rather unfavorable economic conditions currently prevailing in Grenada as a result of the global financial and economic crisis. Table 2 Rate of return of training under several employment rate and wage increase scenarios Percentage point increase in probability of employment among trainees certified at Level 1 0% 2.5% 5% 7.5% 10% 12.5% 1% -29.9% -17.1% -10.1% -4.8% -0.4% 3.4% Percentage increase in 10% -2.3% 2.2% 6.2% 9.8% 13.2% 16.4% wage mass 15% 5.4% 9.3% 12.9% 16.3% 19.5% 22.7% among 20% 11.8% 15.5% 18.9% 22.2% 25.4% 28.6% trainees 25% 17.7% 21.2% 24.5% 27.8% 31.0% 34.2% certified at 30% 23.1% 26.5% 29.9% 33.2% 36.5% 39.7% Level 1 40% 33.2% 36.7% 40.1% 43.6% 46.9% 50.3% 7. Components 2 and 3 are also expected to have a positive economic impact, as the Project effectively supported the establishment of an industry-vetted, regionally recognized occupational standards framework that will serve to permanently improve the quality and value of training in Grenada. Thus, the economic benefits under the Project should include competency-based training and certification benefits in terms of higher probabilities of employment and wages for future cohorts of trainees and employers. In addition, a system of evaluation and certification of labor competency may induce workers mobility among firms; a byproduct that will increase a premium paid for the retention of employees. In contrast, it can also reduce the screening cost and initial training expenses of workers to be hired, leading to more efficient labor markets. 25 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Angela Demas Senior Education Specialist HDNED Viviana A. Gonzalez Program Assistant LCSHH Svetlana V. Klimenko Sr Financial Management Specia LCSFM Daniela Marotta Senior Economist MNSED Thomas Patrick McArdle Consultant LCSHE Yuki Murakami Education Economist LCSHE Rolande Simone Pryce Senior Operations Officer AFTG2 Norma M. Rodriguez Procurement Analyst LCSPT Christel M. J. Vermeersch Senior Health Specialist LCSHH Supervision/ICR Erik Bloom Senior Economist LCSHE Harriet Nannyonjo Senior Education Specialist LCSHE Svetlana V. Klimenko Sr Financial Management Specia LCSFM Thomas Patrick McArdle Consultant LCSHE Yuki Murakami Education Economist LCSHE Norma M. Rodriguez Procurement Analyst LCSPT Alonso Sanchez Consultant HDNED Nazumi Takeda Consultant HDNED (c) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands No. of staff weeks (including travel and consultant costs) Lending 224.6 Supervision/ICR 478 26 Annex 5. Beneficiary Survey Results Not Applicable 27 Annex 6. Stakeholder Workshop Report and Results Not Applicable 28 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Introduction/Background 1. A fundamental ingredient in the development of Grenada is a highly competent workforce, which will have the capacity to provide quality skills and services to the Grenadian private and public sectors. The OECS (Grenada) Skills for Inclusive Growth Project has its genesis in the steady improvement in the economic growth of Organisation of Eastern Caribbean States (OECS) in the lead up to the year 2008. The project recognizes that to avoid stagnation and increase competition especially in the tourism/hospitality, agriculture, marine and construction sectors, that, crucial reforms are needed. One of the elements highlighted in the World Bank report, "OECS Towards a New Agenda for Growth (2005), is the creation of new capacity in the labour force and the private sector to take advantage of emerging opportunities in the global market place. In Grenada, the shortage of skilled labour was cited by businesses as the number one constraint to increasing competitiveness. Moreover, a competent workforce will improve the productivity of Grenada. 2. Following several consultations with various stakeholders starting in 2007, the Government of Grenada and the World Bank signed a Credit Agreement, which was approved by the Parliament of Grenada to finance the Grenada Skills for Inclusive Growth Project. The project was launched on May 26, 2009 and was completed on September 1st, 2013. There were two restructuring with the first in April 2011 and the second in January 2013. Quality at Entry 3. The overall Quality at entry can be described as satisfactory. The development objective set for the project was relevant and appropriate to Grenada’s needs and specific objectives were consistent with the identified needs of the Grenada’s economy at the start of the Project. The Project components were also consistent with the identified priorities and needs of the Grenadian economy. In addition, the project was developed with excellent stakeholders participation and as a result of such there was greater ownership for the project throughout implementation. The PCU was staffed with well trained and experienced personnel. The phasing of the project activities was adequate initially, however as a result of the domestic economic downturn in 2009 tremendous pressure was placed on the PCU and the GNTA throughout the implementation of the Project. The relative allocation of the financing among the project component was deemed appropriate in light of the outputs defined for each component at entry. However, the costing of training was considered a bit unattractive to training providers. Experience during Implementation 4. Project Objectives: The Project’s main objective aimed to support the Government of Grenada to increase the employability of youth through public/private sector partnerships for technical and life skills training that is demand driven. This objective had three intermediate outcomes with associated lines of action: 29 (a) To increase job-related competencies among unemployed youth through the establishment of a competitive training mechanism that supports the financing and delivery of demand driven training (training); (b) To improve the quality and value of training in Grenada and enhance OECS collaboration in training through the adoption of an occupational standards framework that is validated locally and recognized regionally (framework); (c) To strengthen institutional capacity to plan, implement, and monitor training (capacity building). 5. The original project included two overall Outcome Indicators in the Results Framework: (a) The percentage of youth enrolled in the training scheme who are employed fifteen months after the start of their training. Target value: 65 percent. This indicator was dropped in the Project Restructuring of January 2013 and the target in the second changed from 800 to 500. (b) The number of youth certified at Level 1 or above. Target value: at least 800. Project Components 6. The Project has three components: (1) Skills Training for Unemployed Youth (2) Establishing the Standards Framework and (3) Project Management and Institutional Strengthening. 7. Component 1 seeks to provide training for unemployed youth in Hospitality, Marina, Construction and Agriculture sectors, through the establishment of a training scheme. The project commence with training for the Hospitality sector in year one and expanded to other sectors in years two and three, based on demand in the respective sectors. 8. Component 2 addresses the acquisition and introduction of occupational standards, their vetting by local industries, and the establishment of quality assurance processes. The latter includes: training a cadre of assessors and inspectors, accreditation of training providers, assessment of trainees and certification of trainees based on competencies. This framework serves not only the training financed by the project’s first component, but it will also serve the wider national training system including TVET at the secondary school level. 9. Component 3 ensures that sufficient resources are available to coordinate and monitor the project during implementation. The NTA handles the administrative aspects of the training component and serves as a broker for the competitive selection of the training providers. There is a subcontracting competitive process to outsource the hard skills training. The NTA organizes for the traineeships. The Project Coordination Unit (PCU) assists the NTA with the coordination of the call for proposals. Project Manager and support staffs are contracted by the NTA to facilitate project implementation. Operational costs associated with the project implementation are also being supported. 30 The NTA will upgrade its management information system and strengthen the institution’s capacity in financial management. Assessment of the Operation’s Objectives, Design, Implementation and Operational Experience; Relevance 10. Overall objective of the Project remained a high priority to the Government of Grenada at the time of ICR preparation. Project design was effective in that it facilitated broad based stakeholder participation. This was consistent throughout implementation. Project design was modified during implementation due to a downturn in the economy. The changes in the local economy had affected the full participation of the private sector as programmed under the original project design. Notwithstanding this, the project maintained a very high relevance since the operation’s objective, design, and implementation remain consistent with the country’s current development priorities as articulated in the Government’s various policy documents. Assessment of Outcome of the Operation against Agreed Objectives 11. Overall, there was tremendous progress since Project restructuring towards the achievement of the Project’s Development Objectives (PDO’s). The training component was outstanding. 12. At the end of the Project a total of 1052 youth have been enrolled in training compared to revised end-of-Project target of 800. Of this total, 886 have completed training. At the close of the project 220 of the 1052 enrolled pursued level 2 programs (compared with end-of-project target of 80). The Project therefore surpassed the targeted total of 800. (See the Project Development Objectives Framework and Monitoring Indicators attached).  949 persons have received National Vocational Qualifications (NVQs/CVQs).  Collaboration with the Ministry of Youth Empowerment in the implementation of its various programmes will allow for the sustainability of initiatives started under the project.  The Project has had an impact beyond tourism with the percentage of trainees in sectors other than tourism at 55 percent exceeding substantially the 30 percent original end of Project target.  The Project has successfully engaged with the private sector, and there were now 90 private sector companies taking on trainees compared to an original end-of-Project target of 45.  103 pre-qualified training providers, including an innovative partnership with the TAMCC.  Over 240 registered and active employers are currently involved in the Project.  All programmes included 60 hours of Life Skills Training.  Over 52 programmes including 12 Level 2 training programmes were conducted during the Project lifespan. 31  The Project required 80 persons to be trained at Level 2 but more than 230 have received Level 2 training.  Over 100 employed youth were trained in Life Skills compared to the project target of 75  More than 300 employers participated in the traineeship of the training component while the Project target was only 90.  Interviews with private sector companies involved in the Project indicate that there is a premium on workers with certified skills. This applies both to new entrants to the workforce and the ‘upskilling’ and certification of existing workers. Component two - Establishing an occupational standards framework to improve the quality and value of training 13. The NTA has a well-established policy framework for Skills training and certification which includes quality assurance policies and procedures. These are documented in 6 key documents as follows: i) Learner’s Handbook, ii) Center Approval Guidelines, iii) Assessment Guidelines, iv) Quality Assurance Manual, and v) Accreditation Manual, vi) Procedures Manual. A total of 63 Assessors have been trained in a number of areas including agriculture, information technology, office administration, and cosmetology. The Project target is 60. In addition, thirty (30) internal verifiers, thirty-nine external verifiers, 37 facility auditors, and 65 instructors have been trained in competency-based education and training methodologies. Three Master Assessor candidates have received their certification also. 14. Grenada NTA is an active member of the Caribbean Association of National Training Agencies (CANTA). In May 2013 the NTA received CVQ granting status from COHSOD. Several active industry lead bodies throughout Grenada. 15. The NTA vetted 73 occupational standards using local industry lead bodies all of which have been approved by the GCTVET. The Project target was 68 standards. 16. Over 103 applications for pre-qualification by training providers were approved in a wide range of occupational areas including open water diving, building technology, crop production, youth development, livestock rearing and building and construction site supervision. Component three -Institutional Strengthening and Project Management 17. The Grenada Council for Technical and Vocational Education and Training (GCTVET) was established in law in 2009 with the National Training Agency created as the operational arm of the Council. The NTA is manned by a small core of technical professionals. Both the GCTEVT and the NTA continued to operate effectively at the time of ICR preparation. 18. The TVET system and project implementation have been closely monitored and evaluated through in-house monitoring and evaluation and a management information system (MIS). This aspect of the project has been done in collaboration with St. Lucia and continues up to ICR preparation. 32 19. The Project has conducted communication activities aimed at the three major stakeholder groups - youth, training providers, and employers which has been fairly successful in getting youths to register for training programmes, getting training providers to become pre-qualified and fostering the support of the business sector for traineeship placements and general interest in use of occupational standards and the benefits of competency-based education and training. The successful implementation of the communication strategy was achieved. 20. Project Manager and support staffs were contracted for the duration of the Project. 21. A Management Information System (MIS) was development and installation of a unified MIS is completed. This system was developed in collaboration with St. Lucia. 22. Monitoring and evaluation: The NTA must be commended for providing gender disaggregated data. Financing 23. Costs: The total Project cost was US$3.39 million compared to the original cost of US$4.48 million (see Table 1). Table 1. Project Cost Summary by Component in EC$ and US$ Original Original Actual Component Project Cost Summary (EC$ ) (US$) (US$) million million million 1. Skills Training for Unemployed Youth 1. Skills Training 5.59 2.08 1.22 Subtotal Skills Training for Unemployment Youth 5.59 2.08 1.22 2. Improving the Policy Framework for Delivering Training 1. Introduction of Standards, Accreditations & Quality Assurance, Certification 1.08 0.40 0.25 Subtotal Improving the Policy Framework for Delivering Training 1.08 0.40 0.25 3. Project Management and Institutional Strengthening 1. Institutional Strengthening 1.13 0.42 0.28 2. Project Management 2.82 1.05 1.45 3. Public Awareness 0.77 0.29 0.19 Subtotal Project Management and Institutional Strengthening 4.73 1.76 3.39 Total BASELINE COST 11.40 4.24 3.39 Physical Contingencies 0.01 0.04 0 Price Contingencies 0.63 0.23 0.00 Total PROJECT COST 12.05 4.48 3.39 33 24. Financing Plan Table 2: Project Financing in US$ million Funding Source Original Actual Government 0.90 0.36 IDA 3.00 3.06 Private Sector 0.60 0.01 Total 4.50 20 3.39 Performance of the Borrower 25. Overall the Borrowers performance has been highly satisfactorily. At the start of the project everything seemed ready for takeoff, a few months into implementation the economic crisis occurred thus having the Client to revisit the project and make adjustments to the project component. Throughout the life of the project the cost tables had to be adjusted on a number of occasions to meet the challenges which we were faced. The project was therefore executed under abnormal conditions. 26. Notwithstanding the fact that the Government of Grenada’s contribution towards the project has reduced when the project was restructured significantly in January 2013 to address other areas of priority that emerged during implementation. 27. The Ministry of Education fulfilled its responsibility through ensuring that there were project staffs throughout implementation of the project. 28. The PCU’s capacity was placed under tremendous pressure due to changes in the economic climate. Notwithstanding this the PCU provided timely supervision reports as required under the project. 29. The Borrower demonstrated its commitment and ownership of the project through its responsiveness, its regular project team meetings on the progress of the project led by the various component focal points and its open collaboration with the World Bank. In light of the approach adopted by the Ministry through the implementation of the project, this offers well for the sustainability of those initiatives. 30. The Borrower has put in place measure for the sustainability of project activities to continue addressing training and skills needs in Grenada. An important activity completed to date was closer collaboration with the Ministry of Youth and Sport Activities. The MYSEA has just launched phase II of the youth program (phase I being the Skills for Inclusive Growth Project. 20 This discrepancy in total Project costs ($4.48m in Table 1) and financing ($4.50m in Table 2) originates in the original Project Appraisal Document. 34 31. The Project Advisory Committee [PAC] functioned well in the early years of the Project. Performance of the Financiers 32. Overall the Financier’s performance has been satisfactorily. Despite the challenges during the Project Financier’s performance was executed with little or no hitches. During implementation there were ongoing discussions with the Financial Specialist as well as the Disbursement Officers and the Task Team Leaders on matters that needed to be clarified. These resource persons were always available and willing to lend their support and give advice. However this was better executed is the later years of the Project with the coming on board of the new Task Team Leader. 33. The Task Team Leader assigned to the Project must be commended for her high professionalism exhibited throughout implementation of this project which was executed under extraordinary circumstances. The successes achieved under the project can also be attributed to the quality of supervision received by the final Task Team Leader assigned to the Project by the Bank. 34. The Financier fulfilled its responsibility in a timely manner and was very flexible in its approach to project implementation in the later years of the Project. At the earlier years of the project evident of the Bank flexibility was almost absence due to the reluctance of the Task Team Leader to changes in the domestic economic circumstances. 35. Financier was diligent in the monitoring of the project implementation however the midterm review occurred late into project implementation. The Aide Memoires prepared at the end of each supervision mission was extremely helpful in identifying activities completed in an effort to facilitate a smooth project implementation. The Aide Memoires were also very instrumental in assisting the Borrower to plan adequately for future activities. 36. The quality of the monitoring executed by the Financier during project implementation was extremely supportive in assisting the Borrower during project implementation. Factors Affecting Implementation and Outcome Challenges  Unavailability of reliable labour market information  Challenges early in implementation to get traineeship placements which hindered the startup of programmes  Impact of downturn in economy on the ability of the targeted sectors to participate as anticipated (esp. tourism and construction sectors)  Inability to respond quickly to changing circumstances during implementation due to inflexibility of project procedures/processes 35  Need for strengthening counseling services. Both the training providers and other officials pointed to the need for counseling being an integral part of the training program.  One challenge identified by trainees is the delay in payment of stipend.  Approval process for spending has too many steps to completion  Employers were hesitant to pay the stipend citing economic slow-down and lack liquid capital. To drive the Project forward the NTA signed a Memorandum of Understanding with the Ministry of Youth Empowerment and Sports to pay the traineeship stipends.  Inflexibility of Bank Staff towards changes in the domestic economy in the early years of the Project.  Grenada’s economy was hard hit by the 2008 financial crisis and has struggled to rebound. Economic growth at the height of the crisis plummeted to -5.7 percent in 2009, and was only a modest 1.1 percent in 2011. Construction and tourism sectors have been particularly hard hit; several large scale tourism-related construction projects identified at the time of Project preparation as a signal for the need of a large number of employees with Level 1 skills training did not materialize.  Due to the impact of the economic crisis, trainees have been challenged in finding and keeping employment.  The economic downturn has significantly affected private sector companies’ ability to offer traineeships which under the Project’s training model are an essential complement to classroom training. Firms that have weathered somewhat the economic downturn and remained participants in the Project are taking on fewer trainees than originally anticipated. They have been unable to pay trainee stipends and also hesitant to take on trainees that live far away without reliable transportation. In parallel, the introduction of a Value Added Tax is driving an increase in the cost of training, and the Government’s ability to meet its portion of the trainee stipend has been diminished. Lessons Learned 37. Project design should allow for flexibility during implementation phase so that the project has the ability to respond quickly to changes in the environment. 38. The need for effective coordination of all local training programmes to minimise the risk of similar activities implemented simultaneously activities. 39. The collaboration with other countries executing similar initiatives had provided tremendous economy of scales and knowledge that will go beyond the life of the Project. 40. The need to ensure that training facilities are adequately equipped. 41. Better projection of economic climate needed during project preparation. 42. The need to develop a system to gather labour market intelligence. 36 Proposed Arrangements for Future Operation of the Project 43. The financing for the NTA's activities has already been fully absorbed via the Government's budget. The NTA operating expenses have been included in the budget for the Financial Year 2013/14.In this regard adequate budgetary allocation for the NTA has been made. It was agreed that in addition to the Government budget allocation, the NTA would explore other options for generating, income including through charging for some services. In this regard, the NTA has already commenced charging for services. At the end of the Project the NTA remains fully staffed and all expenses in relation to same are fully met. The marketing of the Project continues at certification is now a household name throughout Grenada. Several of the activities under the Project have been fully adopted by the Ministry of Youth, this includes the adoption and extension of the Life Skills Manual developed under the Project. In an effort to facilitate the initiatives started under the Project, the NTA has already developed a three year Strategic Plan which will serve as the main priority for the next three years. The MIS developed under the Project can serve as a medium whereby project beneficiaries can be monitored both within country and within the region. 37 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable 38 Annex 9. List of Supporting Documents Alcala Gerez, A. (2009), Financing Agreement, C4534-GD Conformed. Washington, DC: World Bank. (http://documents.worldbank.org/curated/en/2009/03/10375981/financing-agreement-c4534-gd- conformed) Bassi, M., Busso, M., Urzúa, S. and Vargas, J. (2012), Desconectados: habilidades, educación y empleo en América Latina, Inter-American Bank, Washington, DC. Fazio, M. V. (2011), Análisis de la percepción de los empleadores acerca de las pasantías y las perspectivas de inserción laboral en esas empresas, Technical Note IDB-TN-240, Inter-American Development Bank, Washington, DC. Government of Grenada (2013), 2013 Budget Statement, Presented by Dr. the Rt. Hon. Keith C. Mitchell, Prime Minister and Minister of Finance and energy, St. George’s, Grenada. Heckman, J., Stixrud, J. and Urzua, S. (2006), “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior,” Journal of Labor Economics, 24: 3 (July), p. 411-482. Martínez, S. (2011), Impact Evaluation of Domenican Republic’s Juventud y Empleo, Mimeograph Report, World Bank, Washington, D.C. Mwenda, Edith R. (2013), Official Documents - Second Amendment to the Financing Agreement for Credit 4534-GD. Washington, DC: World Bank. (http://documents.worldbank.org/curated/en/2013/03/17573438/official-documents--second- amendment-financing-agreement-credit-4534-gd) Mwenda, Edith R. (2011), Amendment to Financing Agreement for Credit 4534-GD Conformed. Washington, DC: World Bank. (http://documents.worldbank.org/curated/en/2011/04/14361637/amendment-financing-agreement- credit-4534-gd-conformed) New National Party (2013), Manifesto 2013, St. George’s, Grenada. World Bank (2008-2013), Grenada - OECS (Grenada) Skills for Inclusive Growth Project. Aide Memoires, Implementation Status Results Reports (ISRs) Sequences 1 to 11, and other documents in Project Files, Washington D.C. - The World Bank. World Bank (2011), World Bank Education Strategy 2020: Learning for All - Investing in People’s Knowledge and Skills to Promote Development, Washington D.C. - The World Bank. 9http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTEDUCATION/0,,contentMDK:22474 207~menuPK:282402~pagePK:210058~piPK:210062~theSitePK:282386,00.html0 World Bank (2011), OECS Countries - (Grenada) Skills for Inclusive Growth Project: Restructuring. Washington D.C. - The World Bank. (http://documents.worldbank.org/curated/en/2011/04/14048290/oecs-countries-grenada-skills- inclusive-growth-project-restructuring). World Bank (2011 b), Strengthening Skills and Employability in Peru, Labor Skills Programmatic AAA Final Report, Report No. 61699-PE, Washington, DC. World Bank (2010), Organization of Eastern Caribbean States (OECS) - Regional partnership strategy for the period 2010 - 2014. Washington D.C. - The World Bank. (http://documents.worldbank.org/curated/en/2010/05/12298466/organization-eastern-caribbean-states- oecs-regional-partnership-strategy-period-2010-2014) World Bank (2008), Grenada - OECS (Grenada) Skills for Inclusive Growth Project. Washington D.C. - The World Bank. (http://documents.worldbank.org/curated/en/2008/12/10143567/grenada-oecs-grenada-skills-inclusive- growth-project) World Bank (2007). St. Lucia - OECS Skills for Inclusive Growth Project (Adaptable Program Lending), Washington D.C. – The World Bank. (http://documents.worldbank.org/curated/en/2007/04/12329910/st-lucia-oecs-skills-inclusive-growth- project-adaptable-program-lending) World Bank (2005), Organization of Eastern Caribbean States (OECS) - Country Assistance Strategy. Washington D.C. - The World Bank. (http://documents.worldbank.org/curated/en/2005/09/6227004/organization-eastern-caribbean-states- oecs-country-assistance-strategy) 39 Meetings Conducted during ICR Mission Ministry of Education and Human Resources Development - MoE Theresa Bowen, Permanent Secretary Pauline Finlay, Former CEO Peron Johnson, Skills for Inclusive Growth Project, Project Manager Pauline St. Paul, Students Support Services Unit, Acting Director Pearson Stroude, Guidance and Counseling Dept., Head Imi Chitterman, Information Technology Ministry of Youth, Sports and Ecclesiastic Affairs Veda Bruno Victor, Permanent Secretary, Ministry of Youth, Sports and Ecclesiastic Affairs National Training Agency - NTA Lincoln Morgan, National Training Agency, Chief Executive Officer Shevon Licorish, Monitoring and Evaluation, National Training Agency Trainees Sue Marie Lewis, garment production Tahira Brown, housekeeping Ursulla Lorainey, garment production Reesa Regis, data operation level 2 Delon Davidson, general construction Omari Noel, general construction Kenika Francis, graphics design Deseree Sandy, data operations Micah Joseph, general security Dilys George, general cosmetology Dwight Smart, bartending Employers Cecil Baptiste, Windward Supplies Plus Adam DeCaul, Deco Industries Evadne Bartholomew, garment production Trevor Xavier, Grenada Rechnical and Allied Workers Union Nigel Mathlin, Adua Design Inc. Augustine David, Public Workers Union Ruth Joseph, garment production Pancy Cross, Grenada Hotel and Tourism Assoc. Training Providers Chifford Browne, graphic design Ruth Joseph, garment production Wilma Forteau, cosmetology Beverly Grey, data operation, general office Dean Jules, cricket pitch and outfield maintenance Gabriel Clarke, crop and livestock production level 2 Andrew Belfor, building construction Christopher Stroude, industrial security and basic construction Fredel Halley, general office administration Karen Mapps, general office administration, data operations 40 Neilon Alexis, general office administration, data operations Evadne Bartholomew, garment production Maria Bennett, housekeeping Hilary Lawrence, data operations 1 and 2 Raphael Sylvester, restaurant services Genel Forteau, cosmetology Certified Labor Skills Assessors Annette St. Louis, garment production, fashion design level 2 Lloyd Panchoo, agriculture: livestock rearing; crop production levels 1 and 2 Gabriel Clarke, agriculture: crop production David Duncan, data operations level 1 and 2 Princess Ettienne, general office administration Alford Tannis, electrical installation levels 1 and 2; general construction level 5 Yolande LaBarnie, general office administration Kingsley Charles, carpentry: furniture making Kellisha Granger-Peters, general office administration Sebbe C. Greenidge, food services; cosmetology Beverly Pannotta, garment production World Bank Harriett Nannyonjo, TTL 41 IBRD 33412 61°45' 61°30' SAINT VINCENT AND THE S GRENADINES E I N Petit Martinique GRENADA D 12°30' 12°30' A Hillsborough N Grand Bay E CARRIACOU R Carriacou G Saline Island Frigate Island Large Island E Caribbean H Sea T Diamond Island Les Tantes Ronde Island Caille Island London Bridge 12°15' 12°15' Sauteurs Caille Island Levera Sandy Island Pond Grenada SAINT Bay Bird Island 0 1 2 3 4 5 6 Kilometers ck t ri Victoria MARK t. P a Lake SAINT S S t. Antoine 0 1 2 3 4 5 6 Miles Ma PATRICK rks Gouyave Tivoli Mt. Saint This map was produced by the Map Design Unit of The World Bank. Catherine The boundaries, colors, denominations and any other information (840 m) shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any SAINT Paradise endorsement or acceptance of such boundaries. Grand Roy Great River JOHN Gr eat Bay Grenville 61°30' Grenville SAINT Bay Beau s ANDREW Marquis Grenada GR E N A D A Grand ej ur Étang o South Vendôme East Mt. SAINT (716 m) GEORGE SAINT SELECTED CITIES AND TOWNS ST. GEORGE'S Belmont DAVID Saint David's PARISH CAPITALS Grand Anse Corinth NATIONAL CAPITAL Bay RIVERS Calivigny Point MAIN ROADS Salines 12°00' PARISH BOUNDARIES Glove Island INTERNATIONAL BOUNDARIES 61°45' DECEMBER 2004