PAPUA NEW GUINEA URBAN YOUTH EMPLOYMENT PROJECT 2018 Impact Evaluation and Results PAPUA NEW GUINEA URBAN YOUTH EMPLOYMENT PROJECT 2018 Impact Evaluation and Results Prepared by Sonya Woo, Team Leader, World Bank (East Asia and Pacific Region) and Darian Naidoo, Economist, World Bank. Special thanks to Christopher Hoy, World Bank Task Team members, the Government of Papua New Guinea, the National Capital District Commission for managing the project and data collection, and to our development partners, including the Government of Australia, Exxon Mobil, Bank of South Pacific Ltd and the Republic of Korea for their cooperation and guidance. Contents Acronyms 4 Executive Summary 5 1. Introduction 10 1.1 Background 10 a) Formal Sector Employment in Papua New Guinea 10 b) Challenges for urban youth 11 c) Social problems related to low employment and urban poverty 12 d) The Urban Youth Employment Project 13 1.2 Impact Evaluation Objectives, Data and Methods 14 a) Objectives 14 b) Methodology 14 c) The Screening and Baseline Surveys 14 d) The follow-up Surveys 15 e) Other surveys - Employer Survey, Community Survey and other qualitative data 15 2. UYEP Impact Evaluation Results 16 2.1 Employment Effects 16 a) Evidence of increased employment rates for both OJT and YJC youth. 16 b) Figure 2-a: Evidence of Employment Impacts from the Community Survey 16 c) No evidence of a decrease in informal work and no evidence of an impact on wage rates or hours worked 17 Figure 2-b: Youth involved in informal work in the past month 17 Figure 2-c: Youth who had some employment in the past 6 months 18 c) Evidence of job satisfaction 18 d) Evidence of a participant’s employment in a range of occupations and industries in firms of different sizes 19 e) Evidence for OJT specific employment impacts 20 f) Supporting evidence from the 2017 Employer Survey 21 g) The importance of the three components of UYEP: Training, Wage Subsidies and Work Placements 21 h) The importance of targeted skills training 21 i) Subsidized wages are attractive to employers 22 j) Work placements and job matches are important to employers 22 2.2 Employment prospects and labour market engagement 22 a) Increase in job search behaviour and seeking further education 22 b) Increased optimism about employment 23 Figure 2-d: Optimism about future employment 23 c) Access to formal financing but with limited sustainability 24 d) Some connections to increased educational opportunities 24 2.3 UYEP in the Context of Active Labour Market Programs Elsewhere 24 2.4 Secondary impacts 25 a) Some positive social impacts of UYEP 25 b) Perceived social impacts and perceived impact on crime 26 c) Attitudinal changes on gender and increase in confidence among women 26 d) Impact of the OJT program on subjective well-being 28 3. Conclusions and Recommendations 29 3.1 Five key conclusions 29 3.2 Areas for further research and evaluation 30 4. References 32 APPENDIX A - OJT Impact Evaluation Methodology 34 APPENDIX B - Data Collection - Further information 36 APPENDIX C - Gender Disaggregated Data 42 APPENDIX D - Estimation of Impacts in the OJT Impact Evaluation Using 2017 FUS Data 44 Acronyms ADB Asian Development Bank ALLT Alternative Literacy and Livelihood Training BLST Basic Life Skills Training BPNG Bank of Papua New Guinea CT Control Group FUS Follow Up Survey GBV Gender-based violence GDP Gross Domestic Product GII Gender Inequality Index HIES Household Income and Expenditure Survey ILO International Labour Organization LFPR Labour force participation rate LMIS Labour Market Information System NCD National Capital District NCDC National Capital District Commission NSO National Statistical Office OJT On-the-Job-Training PET Pre-Employment Training PMU Project Management Unit PNG Papua New Guinea SME Small and Medium Enterprise UYEP Urban Youth Employment Project YJC Youth Jobs Corp 4 5 Executive Summary Youth unemployment is an increasingly pressing social UYEP is now the Government of PNG’s most significant and economic problem in Papua New Guinea (PNG). public intervention addressing the economic and social More than half of the population is 24 years old or marginalization of out-of-school and out-of-work (also younger, yet for at least the past decade, the labour referred to as “unattached”) youth in the country. The market of PNG has not grown fast enough to absorb injection of cash into the economy and operation of a the increasing number of youth that are available to social safety net for this under-served demographic group is work, especially those with limited skills and experience. significant in an environment where there is low economic PNG is a fragile state with relatively low productivity and growth and where the cost of living is significantly higher growth in formal sector employment. A large proportion of than in rural areas. UYEP plays a critical role in three areas: youth in PNG have either never had formal employment or (i) the provision of training, job creation and employment have been out of work and education for protracted periods. opportunities for out-of-school and out-of-work youth While data demonstrating a correlation between crime and between the ages of 16 and 35 years; youth unemployment is not available, with limited avenues for employment or education and limited prospects for (ii) the provision of an income transfer to youth; and improving their economic situation, the accumulation of risk the provision of secondary benefits, including (iii) factors are on the rise and many of these young people small-scale infrastructure aligned with the local have turned to crime. Available international evidence, development priorities of cities. while not uncontested, suggests that countries with a large youth demographic composition like PNG face risks related UYEP is a rare example of a truly comprehensive to social instability, particularly public, group violence (or “full service” Active Labour Market Program involving young men. To some extent, this concern is already that takes long-term unemployed youth through playing-out in various places across the country, with vocational training, job matching and fully subsidised periodic, and highly destabilizing, street rioting having taken work placements. At the end of 2018, UYEP has screened place in Lae, Port Moresby (the capital city) and Mt Hagen more than 24,000 youth, engaged 18,500 youth in over the last decade. Robbery and assault are the most training and work placement activities, established about commonly reported crimes. 18,000 new bank accounts and created approximately 815,000 labour days. Seventy percent of these youth In 2010, the Government of PNG requested the World reported never having a waged job before UYEP, 33% Bank to establish the Urban Youth Employment Project reported never having attended high school, 35% reported (UYEP), a development project aimed at investing in being involved in crime and 74% reported never having PNG’s marginalised youth. Commencing implementation a bank account prior to UYEP. UYEP consists of two main activities in September 2012, UYEP seeks to improve program components. The first component is the Youth Job employment opportunities, earning potential, and living Corps (YJC), which targets less educated youth and provides standards of urban youth between the ages of 16 and soft skills training and short-term public works employment. 35 years in Port Moresby by providing skills training and The second component is known as On-the-Job Training short-term placement into employment. Further additional (OJT), an internship program with a range of employers in financing was provided by the Government of Australia in various sectors who partner with the project. The wage 2016 to extend the project to mid-2019. subsidy provides youth with income while the skills training and work experience increases their human capital and employability. 6 From 2015 to 2017, there were two impact evaluations the work placement with a firm. In the 2017 follow-up of UYEP conducted to examine how well the project survey, 41% of OJT participants reported having been is meeting its development objectives, as well as employed in full-time or part-time work in the last six other related impacts. The implementation of the months. This was 28 percentage points higher than for a impact evaluations is a significant achievement in itself control group of participants, who did not receive training because of the lack of availability of such impact and who had no prospects of continuing employment. The evaluations operating at scale in PNG and the Pacific. UYEP employment results are also corroborated by surveyed UYEP impact evaluations involved a baseline survey of community members, 68% of whom report an increase in ‘treatments’ (youth who participated in the UYEP study) and employment opportunities for youth in their communities. ‘controls’ (selected participants that do not full participate in These results are particularly significant because none the program2), two major follow-up surveys of treatments in of the participants was employed in the six months 2015 and 2017, an annual employer survey and an annual prior to the project. community survey. The collected data was then analysed to The UYEP’s employment results are much stronger measure the following key indicators: than for similar youth employment projects in other • Whether UYEP increases the employment and labour countries.3 There has been a significant number of impact market engagement (including job and education evaluations of active labour market programs (ALMPs) in a seeking behaviour) of participant youth; variety of contexts and the clear majority have shown they tend to have limited effect ranging from no employment • If UYEP provides other benefits to participant youth gains to increasing, at best, employment by seven to eleven beyond employment, particularly in relation to social percent. Other studies show that providing participants with behaviour, attitudes to gender and crime; technical and vocational training rarely generates jobs • If the project has had a positive impact on communities (unless a cash grant is provided) and almost always the costs and business, more broadly; and outweigh the benefits.4 • Whether all three components of UYEP contribute to Second, there is evidence of increased youth employabi­ the increasing employment of participant youth. lity. While the main reason for employer participation in the OJT was to contribute to development, employers The main results of the UYEP demonstrated in this working also confirmed the importance of all three components paper are: (i) increased youth employment; (ii) increased of UYEP training, job matching and fully subsidised youth employability; and (iii) increased perceptions work placements. The wage subsidy helps to off-set financial of youth resiliency to crime and violence. In risks for employers and provides youth with income while support of these reported achievements, this working the skills training and work experience are designed to paper presents the five major findings of the impact increase the human capital and employability of youth. evaluations and supporting surveys. Employers reported that youth are developing technical skills First, there are large employment impacts from UYEP. and that most of youth are “fairly” or “very qualified” for The data shows that participation in the UYEP significantly formal employment. 97% of employers also reported that increases employment rates for both young men and participants were perceived to be qualified for a full-time job. women aged between 16 to 35 years by eight to eleven The main reported constraint for firms taking on more percentage points. The follow-up survey conducted in 2015 permanent employees were difficulties in finding staff found that 14% of YJC participants and 21% of OJT with the “right skills”, training costs being too high participants were currently employed, while only 3% of followed by poor economic conditions. Taken together control youth reported being employed. These results were with the high employment rates, particularly for OJT statistically the same for men and women. The 2017 youth, these findings suggest the importance each of follow-up study also provides strong evidence of substantial these three activities have on employment outcomes. employment gains due to the pre-employment training and 2 The 2015 ‘control’ group consists of observably similar youth living in areas not serviced by the program, while the 2017 ‘control group’ consists of participants who did not receive pre-employment training or OJT but were instead part of a Fédération Internationale de Football Association (FIFA) volunteer work program. 3 McKenzie, David (2017) “How Effective Are Active Labor Market Policies in Developing Countries? A Critical Review of Recent Evidence” World Bank Research Observer, World Bank Group, vol. 32(2), pages 127-154 4 Blattman, Christopher and Laura Ralston (2015) “Generating employment in poor and fragile states: Evidence from labor market and entrepreneurship programs”, Mimeo. Chicago 7 Third, youth who participated in UYEP were shown to Fourth, there is evidence that UYEP has improved the be more engaged with the labour market than youth lives of youth participants and their communities who did not participate in the project, as evidenced in a few non-pecuniary ways beyond employment, by increased job search behaviour and increased including through increased happiness and well-being. confidence in future employment. In the 2015 follow-up 85% of community members in project locations believed survey, the number of participant youth seeking wage crime and violence reduced because of the project. The employment in the last three months increased from 21% to number of youth who reported having friends involved in 33%, reflecting their belief that they are more likely to violence or robbery fell by eight percent (from 24% be employed in the future, because of their newfound pre-UYEP to 16% by the time of the 2017 follow-up survey). experience and skills (gained in CV writing, public speaking, The results also show a significant decline in the reported work place experience). Similarly, 76% of OJT graduates and incidence of “using threat or force with somebody” for 64% of YJC graduates said that they felt participation in the participant youth (from 16% to eight percent), while this project increased their employability. Youth participating in behaviour increased among control youth. The survey results UYEP were also more likely to have bank accounts than are also corroborated by community members in project controls. The evidence shows that UYEP youth are also more areas, 85% of whom believed UYEP had reduced crime in likely to want to enrol in education in the next six months their community. Because of the training and work experience compared with control groups: half of the participant youth provided by the OJT program, the number of OJT participants reported increased interest in pursuing education and training who reported “feeling happy most of the time” in the past following the project (46% of YJC youth and 50% of OJT four weeks, rose by 15%. youth, respectively), while four percent of YJC and six Fifth, there is evidence that UYEP has helped to percent of OJT youth were already enrolled in education. empower women and improve attitudes to gender Pre-UYEP, the majority (60%) of participants expressed equality. A Gender and Labour Study for the UYEP5 interest in further education in the baseline survey and this confirms that in most aspects of training, women have fell to 27% for control youth and 47% for treatment youth. shown a higher propensity to complete the training offered, The effect of OJT is therefore a 21.7 percentage point which suggests that the project has been successful at increase relative to control youth, which is highly significant. engaging young women. UYEP recruits both young men and women into both its programs, and it also provides gender training to all its participants. In providing these opportunities, UYEP has improved not only the status and economic opportunities available to these young women, but it is also transforming gender attitudes among its youth more broadly. For many women, the biggest incentive to participate was the fortnightly stipends paid by UYEP to supplement household expenses. Survey results show that the percentage of participant women who believed that women should be at home and not in waged work, fell from 13% beforethe project to two percent after the project (and from 16% to nine percent for men). 5 Voight-Graf, C. 2018. Gender and Labor Market Study, Port Moresby, Papua New Guinea. Other key challenges faced by females related to household and family obligations includes: not finding a babysitter and pregnancy. Most women were expected to tidy their houses before leaving in the morning and therefore often arrived late. For others, unsupportive husbands were a major obstacle. Previous studies reported that some female participants had to deal with aggressive resistance from husbands towards their participation in UYEP. Focus group participants also talked about incidents of jealous husbands turning up at worksites and forcing women to withdraw. Interviews for this study suggest that there are considerable costs for providing childcare for mothers in terms of energy, financial expenses, and missing training and work experience days. 8 FIFA U20 Women’s World Cup. 1000 UYEP trainees assigned to major event to provide ticketing, logistics and first aid support Recommendations and moving forward (ii) Targeting of unattached youth, many of whom were highly transient and reported to have prior experience The results of the follow-up surveys and the impact with crime and violence, which made the implementation evaluation suggest that UYEP has, to date, been successful in of the impact evaluation, including the screening and improving the economic situation and social welfare of selection process particularly challenging. The project’s participant youth. The evidence also suggests that these experience highlighted the importance of consulting gains occur relatively quickly – within two years of youth with a range of local level community stakeholders in graduating from UYEP, however the longevity of these advance of screening to raise awareness and disseminate impacts is yet to be determined. As such, the project’s short project information regarding target groups and the to medium term impacts and trends will need to be verified application of the randomisation process as well as over the longer term6, and with supplementary data and maintaining effective communication with youth on evidence to capture the value and effects of individual project participation selection. This includes potentially project interventions. While beyond the scope of the project, offering the program to the control group at a later further collection and analysis of labour market data is also stage so that they do not feel discriminated against; required to strengthen our understanding of the nature of youth unemployment and how it interacts with labour force (iii) Problems acquiring and maintaining the quality and participation in PNG, as well as the economy more generally. longevity of control groups with the re-contacting of Finally, more focused research on UYEP’s impacts on gender, participants proving to be difficult even after six months. crime and urban safety needs to be undertaken. This suggested the need to ensure that both control and treatment groups were oversubscribed to compensate Lessons learned, and evidence gathered from the impact for attrition; and evaluation will be used as a guide for the design of the follow-on project. Some of these lessons learned include: (iv) Problems in the management and execution of monitoring and evaluation activities suggest the need for additional (i) The importance of retaining all three programmatic resourcing and enhanced implementation support to be components of UYEP ̶ soft skills training, job-matching provided by the World Bank and/or external partner with and fully subsidised work placements given their experience in implementing impact evaluations. collective role in promoting positive employment outcomes for youth and other externalities; 6 Globally, there has been a significant number of impact evaluations of Active Labor Market Programs (ALMPs) in a variety of contexts and the clear majority have shown they tend to have limited effect on employment according to a recent World Bank working paper. See http://ibread.org/bread/system/files/bread_wpapers/507.pdf The direct employment benefits from subsidized job placements, whereby participants complete ‘internships’ and their wage is subsidized, tend to only be short lived as the effect quickly diminishes once the program is completed. 9 1. Introduction 1.1 Background a) Formal Sector Employment in Papua New Guinea in quantifying the impact of UYEP on PNG’s labour market. The most recent data comes from Jones and PNG’s economy has tripled in size since it gained McGavin’s 2014 labour market study, which shows a independence in 1972 and its Gross Domestic Product working-age population of 4.6 million, two-thirds of which (GDP) growth has steadily averaged 3.4% per year, form part of the labour force. A large majority of the labour (not much higher than population growth) mostly due force is employed, although over 80% of them work in to a significant resources boom. PNG’s economy remains the informal or subsistence economy. If this is by choice, dominated by two main sectors: the agricultural, forestry, however, remains unclear, as does the extent to which youth and fishing sector, which engages most of the labour force, specifically (as opposed to older segments of the population) and the minerals and energy extraction sector, which are interacting with the informal economy. Most Papua New accounts for most of export earnings and GDP. The country Guineans are engaged in agriculture, hunting and forestry has a small formal sector, focused mainly on the export of its (71% of men, 81% of women). The wholesale and retail natural resources and the private sector (two-thirds of trade sectors are in second place (10% of men, 12% of all formal sector jobs are in the private sector, the remainder women).10 are in the public sector);7 and an informal sector, employing most of the population. Agriculture provides a subsistence Over the last decade, the strong performance of livelihood for 85% of the people. Despite its abundance of PNG’s economy has seen formal employment grow natural resources, PNG continues to fall short of the potential by an average of six percent per year. However, due to that its enormous natural wealth, strategic geographic an overall downturn in the economy, most youth location and young and diverse population imply. have limited access to education and employment opportunities. Formal employment in the private sector Although there has been rapid economic growth over recent peaked in 2012 at the height of the PNG LNG construction decades and significant increases in government expenditures, phase.11 With the end of the mining boom, however, the poverty remains persistent in some areas and progress in non-mining economy has suffered a significant downturn, human development indicators are insufficient to support with private sector employment falling by 20% in Port PNG’s ambition of becoming an upper middle-income country Moresby, mainly due to large declines in the construction, by 2030. The 2010 Household Income and Expenditure wholesale and trade industries.12 Today, the formal labour Survey reveals that 38% of the population lived below the market only provides livelihoods to about 10-15% of the internationally recognized extreme poverty line of $1.90 per working age population in PNG.13 Compounding this problem day.8 PNG’s Human Development Index (HDI) rank, was 154 is a rapidly growing population and education policy in a in 2015, the second lowest in the Pacific region. The country country that graduates approximately 80,000 young people is also vulnerable to natural hazards.9 These include floods, from school a year. However, the formal labour force can droughts, landslides and sea-level rise, which have affected only absorb less than 10,000 per year, mostly in jobs that the agriculture and fisheries sectors, leading to lower require secondary education. Enrolment numbers in both economic growth and negatively impacting the most vulnerable universities and technical colleges in PNG are only at populations. approximately 11,000 per year, and graduation rates are half There is generally a lack of reliable and up-to-date labour that.14 This leaves a vast number of youth without access to market data in PNG, which presents a significant barrier either further education or employment. not only to evidence-based policy development but also 10 Despite limitations in the job market, reports by Although there are vast and unmet skills shortages employers suggest that there is an opportunity for in PNG, there are also few opportunities for youth to creating and filling more jobs; and that incorporating a enter the formal sector labour market in Port Moresby. In combination of soft and practical skills training in 2011, only eight percent of 15 to 24-year old youth had all stages of the work cycle is at least as important as a wage-paying job (nine percent of men and six percent of building technical skills. Employers in the formal sector women, respectively), and most young people secured face several challenges when recruiting new staff. A 2016 their livelihoods from non-monetary activities.18 The World business survey found that businesses were hesitant to hire Bank estimates that there are at least 40,000 unemployed new staff mostly due to general economic conditions in PNG youth currently in Port Moresby.19 (88%), followed by difficulties in finding staff with the Each year more than 80,000 youth leave secondary appropriate skill set (83%), and challenges associated with school across the country; however, most are not poor staff productivity, work ethic and staff attendance/ provided with the training they need to transition into punctuality (71%). In the wholesale, hospitality and retail formal sector roles. There are only limited places available sectors, skill shortages were rated as the number one at technical colleges and universities (just over 5000 across factor affecting recruitment (93%).15 PNG). This restricts the options available for youth to be There are also skills shortages across all sectors and trained to fill skills shortages and creates an oversupply of trades in PNG, including carpentry, hospitality, retail low skilled labour. This issue is particularly acute in Port and office administration.16 More generally, basic literacy Moresby and other urban areas, where the creation of and numeracy skills are low in PNG. The PNG National formal jobs and other livelihood opportunities have failed to Labour Market Assessment Report showed that two-thirds match rapid population growth. (69%) of employees in the formal labour market had only Social problems related to low employment and c) basic education up to Grade 8 and only 16% had urban poverty post-secondary qualifications.17 Youth unemployment is high in PNG, with young b) Challenges for urban youth people in urban areas living ‘day-to-day’, facing In Port Moresby, youth between the ages of 15 and difficulties in accessing state services and being far 29 represent 35% of the population, and this cohort is more likely to engage in opportunistic crime. Crime and estimated to grow by at least another 13% in coming victimization rates are among the highest in the world. years due to natural population increases and internal A recent World Bank report estimated that the homicide rate migration. Meanwhile, urbanization and a dearth of in the capital city of Port Moresby was 33 per 100,000 informal sector opportunities is contributing to the growth persons in 2010, which would rank among the top 50 cities of an underclass of young urban poor, and coupled in the world.20 This is not surprising: among young people with rapid demographic growth, this creates tremendous aged 15 to 24 in Port Moresby, 29% were estimated to live pressures for employment generation for these young on less than $3.10 per day in 2009/2010, and 31% were people. Specifically, three underlying issues pose significant unemployed. The evidence shows that where there are few challenges for the youth of Port Moresby: employment opportunities, youth idleness leads to high levels of crime. In a 2004 survey of youth aged 15 to i) Their lack of equity in and access to the formal primary 35 in Port Moresby, 36% of respondents admitted to having and secondary education; committed an offence for which they could have been ii) Their lack of equity in and access to the arrested,21 24% of young men admitted to having taken labour market; and part in a burglary, 20% in petty crimes, 18% in an assault, 19% in a carjacking, 10% in a rape and four perecent in iii) Their increasing involvement in urban crime and a murder. violence, and risk exposure to other social pathologies. 7 NSO 2013, Jones and McGavin 2015. 13 Lowy Institute, 2016. 8 NSO 2012, PNG Household Income and 14 Lowy Institute, 2016. Expenditure Survey, 2009-2010. 15 Deloitte and UNDP, 2017. 9 The University of Notre Dame’s Global Adaptation 16 Imbun, 2015. Initiative ranks PNG 173 out of 181 countries 17 OHE and DNPM, 2010a. for its vulnerability and readiness to the challenges 18 Voigt-Graf, 2017. of climate change. 19 World Bank, 2018. 10 Jones and McGavin, 2015. 20 The World Bank, 2014. 11 Voigt-Graf, 2017. 21 UNHabitat, 2004. 12 Jones and McGavin, 2015; BPNG 2018. 11 Women are especially at risk of gender-based violence youth are enrolled every two to four months with over in PNG. The UN Special Rapporteur on Violence against 20 intakes occurring since the program began in 2010. Women recognized violence against women as a “pervasive All youth begin the program with five days of basic life phenomenon… in the home, the community and institutional skills training. settings” in Papua New Guinea.22 As many as two in three About 75% of participants complete the YJC – a 40-day women in PNG have experienced physical and/or sexual public works program, such as cleaning, landscaping, paving violence at some point in their life – twice the global and maintaining roads. The other 25% of participants average.23 In addition, women face prohibitive gender complete pre-employment training (PET) of 20 days followed stereotyping that constrain them from seeking employment by On-the-Job Training (OJT), which is five months with a and education opportunities. Relative to their numbers in private or public company. Generally, the YJC is more society and relative to their overall involvement in economic accessible to youth with a lower baseline skill set. However, activity, women are under-represented in formal sector it is shorter in duration and less skill-intensive than the OJT, activity – only 38% of women versus 68% of men are in and therefore likely to have a smaller impact on the future waged employment.24 employability of participants. The structure of the project d) The Urban Youth Employment Project is described in Box 1, with additional information about OJT and PET. The government of PNG, multilateral and bilateral aid donors have invested in a range of strategies to address the lack of opportunities in the formal sector for young people in PNG. One of the largest programs to date in Port Moresby is 1.2 Impact Evaluation UYEP. Launched in 2010, UYEP aims to contribute to PNG’s National Youth Policy priorities to support livelihood Objectives, Data and Methods promotion, poverty reduction, and socially unhealthy a) Objectives behaviour for youth, by assisting the growing segment of disadvantaged youth who have dropped out of school An impact evaluation of the UYEP was undertaken or university and lack the basic competencies required from 2015 to 2017 to determine whether the program to compete in the labour market. The program provides is achieving its development objectives and/or having unemployed, out of school youth in Port Moresby with any secondary impacts. The impact evaluation was designed human capital/training, work experience, and well as to answer the following questions: income from temporary employment opportunities, aimed 1. Are participants more likely to be employed after the at ultimately increasing their employability. project than controls (similar non-participant youth)? UYEP targets disadvantaged youth firstly with skills training How is this different for OJT graduates compared to YJC (both life skills and some work skills) and then with short graduates? Are the three components of UYEP (skills term subsidised employment in two streams. One stream training, wage subsidy and matching) important? Do is a public works program called the Youth Jobs Corp (YJC) earnings and hours worked differ across participant and the other is an internship-style program with a range groups and controls? of employers who partner with UYEP in an ‘On the Job T 2. Are job search, employment expectations and education raining’ program (OJT). The wage subsidy in both streams plans impacted by the project? provides youth with income while the skills training and work experience are designed to increase their human 3. Are there other benefits for participating youth beyond capital and employability. To be eligible, participants must be employment, particularly in relation to social behaviour, aged between 16 and 35 years, have been out of work and gender attitudes, and crime? Does the project improve not in education for the last six months, and have been living the well-being of participants? in Port Moresby for the past six months. New intakes of 22 Fulu et al. 2013. Why do some men use violence against women and how can we prevent it? Quantitative Findings from the United Nations Multi-Country Study on Men and Violence in Asia and the Pacific. Bangkok: UNDP, UNFPA, UN Women and UNV; and, Ganster-Breidler 2010. Gender Based Violence and the Impact on Women’s Health and Well-being in Papua New Guinea. Contemporary PNG Studies: DWU Research Journal 13 (17). 23 Haley, N and R. Muggah (2006), Jumping the Gun: Armed Violence in Papua New Guinea, in Small Arms Survey 2006. Unfinished Business, Geneva. 24 Pacific women, 2016. 12 4. Has the project had a positive impact on communities UYEP participation. It is also administered to samples of and business? Do communities and business have non-participating youth, to compare any changes over time positive views of the project and its impacts? between the two groups. Outcomes that are measured include employment gains and income, job search behaviour, b) Methodology well-being, involvement in crime, social behaviour and The project’s rigorous data collection system involves optimism about the future. collecting extensive baseline data from all youth who apply The two major follow-up surveys were conducted in 2015 for the UYEP. The same baseline data is then also collected and 2017. The 2015 follow-up survey included non-participant from a “control group”. The 2015 control group is controls from 2014. For the 2017 FUS, a group of participant comprised of similar youths in Port Moresby that did not controls was used in comparison to OJT youth. The participant participate in the project.25 The 2017 “control group” consists controls had been through the YJC and entered a FIFA of participants who did not receive pre-employment training volunteering program that was not designed to build their or OJT but did complete the YJC and were engaged in occupational or industry-specific skills or connect them with the FIFA volunteer work program. Samples of both these a potential long-term employer. Thus, this group’s employment participants and controls were then re-interviewed in outcomes could be compared to the regular OJT youth, since follow-up surveys in 2015 and 2017. In addition, the impact none of the participant controls completed OJT and with any evaluation also involves surveys of the OJT employers and of difference being exclusively attributable to the OJT. the community members where the project operates. This broad range of data provides different perspectives on the e) Other surveys - Employer Survey, Community Survey project and demonstrates where and how it is having an and other qualitative data impact on participants and their communities. These surveys The main objective of the annual Employer Survey is to are outlined briefly below. gather employer feedback on their OJT trainees as well as (A more detailed summary of the impact evaluation their general perceptions of the UYEP OJT program. The methodology and a summary of the annual data collection latest survey (2017), involved a sample of 65 employers, who cycle is presented in Appendix A, and an analysis of the provided information on OJT trainee’s performance and different surveys’ limitations is provided in Appendix B.) post OJT employability. The key objective of the annual Community Survey is to measure community perceptions of c) The Screening and Baseline Surveys UYEP and its impact. The 2017 community had a sample When youth apply for the UYEP program, they are required of 66 community members from 11 communities, who were to fill out the Eligibility Screening Survey, which, in addition asked the following questions: to verifying basic eligibility criteria (such as age and residency i.) Have the employment prospects of youth, as perceived requirements), collects basic socio-economic characteristics by the community, improved because of UYEP? of these youth. Those who are determined eligible to participate in the scheme are then required to fill out the ii) Is youth involvement in crime perceived to have Eligibility Baseline Survey (EBS), which collects a richer set of decreased by UYEP? socio‐economic variables compared to ESS. It should be iii) Is the program perceived as having any negative noted that from Intake 13 onwards, these surveys have been impacts on the community? combined. The survey is given to all participants and in 2014 it was also administered to non-participants, who formed Finally, additional qualitative data was gathered through the basis of the 2015 control group. focused group discussions with UYEP participants and trainers, and with employers and contractors in Port Moresby d) The follow-up Surveys collected to investigate their views on gender-related issues The follow-up survey questionnaire mirrors the screening/ in the project.26 baseline questionnaires. It is administered to samples of participating youth, six months to one and a half years after 25 The control group was defined during the 4th and 5th intakes of the Project, which took place in mid‐2013. In the first and second intakes, the program was oversubscribed and randomly selected participants among a large pool of applicants with the intention of conducting a randomized control trial to evaluate the program. However, controls were not clearly informed that they had to reapply to the program in subsequent intakes, and reportedly expressed their displeasure with physical threats made in later intakes. As a result, the NCDC and the World Bank decided to evaluate the program against a control group selected from neighbouring villages. 26 Voigt-Graf (2018) 13 2. UYEP Impact Evaluation Results 2.1 Employment Effects The main questions addressed in analysing the % Currently employed (All jobs and UYEP related jobs) employment effects of UYEP were: 25.0 • Are UYEP participants more likely to be employed 20.7 (All) after the project than other, similar youth who do not participate? 20.0 • How is this different for OJT graduates compared 14.3 (All) to YJC graduates? 15.0 • Do earnings and hours worked differ across participant 7.7 groups and non-participants? 10.0 (UYEP only) 6.9 (UYEP only) The results and analysis of this data is outlined below. 5.0 a) Evidence of increased employment rates for both 3.2 (all) OJT and YJC youth. . Participation in the UYEP successfully boosts employment PET YJCs CTs rates for both men and women. The survey revealed that only three percent of non-participating youth reported being Figure 2-a: Evidence of Employment Impacts from the Community Survey currently employed. In comparison, YJC youth had an employment rate of 14%, which is 11 percentage points The data is mostly supported by the members of the higher than control youth, while OJT youth had an communities in which UYEP operates, the majority of whom employment rate of 21%, which is 18 percentage points reported that they believed the project resulted in “an higher than control youth. This means YJC youth are more increase in employment opportunities” for their youth than four times as likely to be employed as controls and OJT (68%). This means that public perception of the UYEP is held youth are seven times more likely to be employed than in reasonably high regard. On average, 63% of UYEP youth controls. There are no significant differences in employment known to the community members are reported as working rates for men and women. These results demonstrate the post-UYEP.27 Furthermore, community members reported positive impact that UYEP has in increasing employment that over 50% the youth who were not currently working for participating youth in Port Moresby, particularly when were “likely or very likely to find waged work in the next contrasted with the employment rates of controls – who are six months”. significantly less likely to be employed – and compared to international evidence, the most successful examples of which yield maximum increases in employment of 7-11 percentage points. 27 These figures suggest higher employment rates than the follow-up survey results, but it must be reiterated that these views are just perceptions data. As a result, the NCDC and the World Bank decided to evaluate the program against a control group selected from neighbouring villages. 14 b) No evidence of a decrease in informal work and no FUS 2017: % of Youth Employed in the past 6 months, evidence of an impact on wage rates or hours worked Full-time or Part-time, in regular or ongoing work. Participation in UYEP does not negatively affect rates 60% of participation in informal employment. In some comparable Latin American ALMPs, participation in a 50% subsidized employment project led to a decrease in informal work following project completion. By contrast, there was 40% no evidence that the number of YJC and OJT youth engaged 30% in informal work decreased after UYEP participation. While 30% 15% it is difficult to attribute this increase directly to UYEP participation, it demonstrates that the positive effects of 20% 13% UYEP participation do not come at the expense of informal work, as the international evidence has suggested to date. 10% 15% 22% 6% 17% 14% FUS 2017: Involved in informal work during the past month 8% 0% 60% PET & OJT PET & OJT YJC YJC Women Men Women Men 49% 50% Full-time Part-time 39% 40% Figure 2-c: Youth who had some employment in the 44% past 6 months 30% 30% Finally, looking at employed men and women, there are no statistically significant differences in hours worked or 20% wages. These results are reported in the Appendix C. c) Evidence of job satisfaction 10% Participant youth in post UYEP employment report satisfaction 0% with their employment. This applies for both OJT and YJC YJC YJC OJT OJT participants. Table 2-1 shows that when participant youth Pre-UYEP Post-UYEP Pre-UYEP Post-UYEP were asked in the 2017 FUS about job satisfaction, 35% of YJC youth and 37% of OJT youth reported being Figure 2-b: Youth involved in informal work in the past month “very satisfied”, with a minority (less than 20%), reporting dissatisfaction with their post-UYEP employment. The results Although participants are more likely to be employed than are not significantly different between YJC youth and non-participant youth, participants that are employed are OJT youth. Taken together with the similar wage levels of the not working longer hours than the non-participant youth two groups, these findings suggest the YJC employment is sampled (43.6 and 44 hours, respectively). similarly beneficial to OJT employment (at least in terms of In the 2017 FUS, the treatment and control groups were too the contemporaneous benefits). However, as this data has small to estimate differences by gender, but figure 2b reports no comparison to controls, there is no evidence as to whether the percentages by work type and gender, for both YJC the employment UYEP youth obtain is better in any way and OJT youth in the FUS sample (including youth not than the jobs controls find themselves in. comparable to the control group). Based on these figures, women have a higher employment rate in OJT while men have a comparatively higher employment rate in YJC. 15 Table 2-1 Satisfaction with work Very Fairly Neither Fairly Very satisfied satisfied satisfied/ dissatisfied dissatisfied dissatisfied Total YJC 35% 43% 10% 6% 7% 100% Observations 31 38 9 5 6 89 PET, OJT 37% 33% 11% 7% 11% 100% Observations 10 9 3 2 3 27 Source: FUS 2017 d) Evidence of a participant’s employment in a range of YJC youth are not surprising given the connection between occupations and industries in firms of different sizes the construction industry and YJC public works activities which are unskilled. In addition, the fact that wholesale There are participant youth in post-UYEP employment and retail trade are typically large employers of low skilled across a range of industries (Table 2-2), and occupations workers, explains the large shares in that industry and in (Table 2-3). The construction industry and wholesale and the occupation category of service workers/shop sales. retail trade are the leading industry sectors. For YJC youth, Table 2-4, reports the firm size of employers, with no clear elementary (unskilled) occupations dominate. In contrast for pattern emerging (a range of small and medium enterprises, OJT youth, the most common occupation is service workers with 25% or less of employers having 200 or more employees). and shop and market sales, which accounts for almost half of OJT graduate employment (46%). The statistics for the 16 Table 2-1 Satisfaction with work Industry YJC PET+ OJT Agriculture, hunting and forestry 3% 0% observations 3 Manufacturing 2% 7% observations 2 2 Construction 30% 11% observations 27 3 Wholesale and retail trade, repair of vehicles and household goods 20% 21% observations 18 6 Hotels and restaurants 7% 18% observations 6 5 Transport, storage and communications 10% 14% observations 9 4 Real estate, renting and business activities 2% 4% observations 2 1 Public administration and defence; compulsory and social security 17% 14% observations 15 4 Education 0% 4% observations - 1 Health and social work 0% 4% observations - 1 Other community/social /personal services 7% 4% observations 6 1 Private households with employed persons 1% 0% observations 1 - Total observations 89 28 17 Table 2-3: Post-UYEP Employment, by type of industry Occupation YJC PET+ OJT Teaching and associate professionals 0% 4% observations 0 1 Clerks 10% 21% observations 9 6 Service workers and shop and market sales 26% 46% observations 23 13 Craft and building trade workers 16% 11% observations 14 3 Plant and machine operators and assemblers 1% 0% observations 1 0 Elementary occupations (unskilled) 47% 18% observations 42 5 Total observations 89 28 Table 2-4: Size of employer Table 2-1 Satisfaction with work 1 to 9 staff 10 to 24 25 to 49 50 to 99 100 to 199 200 or staff staff staff staff more staff Total YJC 18% 19% 17% 13% 18% 15% 100% Observations 16 17 15 12 16 13 89 PET, OJT 11% 21% 14% 7% 21% 25% 100% Observations 3 6 4 2 6 7 28 e) Evidence for OJT specific employment impacts In the treatment group, the rate of full-time employment was 22% (compared to five percent in the control group), The 2017 follow-up study provides strong evidence that and the part-time employment rate was 19% (compared UYEP’s OJT component has a significant effect on to eight percent in the control group). There were no the wage employment of its participants, with a significant differences in the results by gender. Like the 2015 41-percentage point increase in the share of participants follow-up survey, no significant differences were found for in the treatment group who are currently working or hours worked or the wage rate of employed youth.Table 2-4, who have worked in the last six months, compared to a reports the firm size of employers, with no clear pattern 14-percentage point increase in the control group. This emerging (a range of small and medium enterprises, with represents a difference of 28 percentage points, which is 25% or less of employers having 200 or more employees). highly statistically significant, given that this is from a starting point of no wage employment in the last six months at the time of the baseline survey. 18 f) Supporting evidence from the 2017 Employer Survey of soft and practical skills training in all stages of the work cycle is at least as important as building Almost one in five of OJT participants are successful in technical skills. This is seen to make a difference in increasing securing ongoing positions with their OJT employer employment and social outcomes for poor uneducated beyond their six-month internship, and the majority youth, looking for work. The evidence shows that the three are deemed to be more qualified than pre-placement. main constraints for firms in PNG taking on more permanent Nearly all (97%) employers reported that participants were employees are difficulties in finding staff with appropriate perceived to be qualified for a full-time job. They also skills, prohibitive training costs and poor economic conditions. reported that more than three-quarters (78%) of the youth More than half of all employers (52%) cite the cost of training learnt a specific technical skill. Of the youth that were as one of the most significant barriers to hiring more youth reported to have finished OJT, 19% were offered an ongoing staff and that “staff with the right skills are difficult to find”. job and accepted the offer. 55% of these jobs were full-time In addition, half of all employers (51%) reported that they and 45% were part-time. Employers reported that of the valued the training provided to youth prior to OJT placements youth offered a job, all were qualified “to at least a minimum as the most important aspect of the program, with 49% degree” and 15% were “highly qualified”. reporting that they would like the amount of pre-placement g) The importance of the three components of UYEP: training increased. The results from employers are corroborated Training, Wage Subsidies and Work Placements by community members, more than half (55%) of whom felt that UYEP increases both the knowledge and skills of UYEP While the main reported reason for employer youth in their communities. The results are also consistent participation in the OJT is generally to contribute to with existing literature.29 development, participating OJT employers confirmed the importance of all three components of UYEP – i) Subsidized wages are attractive to employers training, job matching and fully subsidised work One of the essential components of the UYEP – the full placements – as crucial factors in deciding to participate subsidisation of wages for five months – is one in the program and to employ OJT youth beyond their of the most attractive elements of the program to initial placement. Essentially, it is not possible to single out employers. Nearly half (47%) of employers reported that the marginal effects of any one intervention, nor is any one they planned to accept more than five OJT youth next year, component considered more attractive to employers than compared to only 25% if there was no stipend. On average, the others, which suggests that the synergistic relationship only 30% of employers indicated that they would keep the between all three elements is crucial to the success of UYEP same number of placements if they received half the subsidy overall. The wage subsidy helps to off-set financial risks for and less than 15% of employers would keep the same employers and provides youth with income, while the skills number if there was no subsidy. Nearly half (45%) of training, work experience and facilitation are designed to employers indicated that they would like to increase the increase the human capital and employability of youth. Half time-period of OJT beyond five months, assuming the (50%) of employers said it was the pre-employment training full subsidy remained. Finally, the subsidy allowed some provided to the OJT youth that motivated their decision to employers to reduce the total labour costs of their business participate in the program, with 20% citing the wage as nearly one-third (28%) reported that they would need subsidy and 22% reporting “the experience gained in YJC” more staff in the absence of the UYEP (compared to 10% as motivating factors. who say they would have less staff). However, this data is not h) The importance of targeted skills training conclusive, as there is no substantive evidence to consider any significant displacement effects caused by the project.30 Despite limitations in the job market, reports by employers suggest that there is an opportunity for creating and filling more jobs; and consistent with global evidence,28 that incorporating a combination 28 Fox, L & Kaul, U, 2017, The evidence is in: How should youth employment programs in low income countries be designed? 29 Deloitte Touche Tomatsu & UNDP 2017, Fulfilling the Land of Opportunity: How to Grow Employment in PNG 30 Importantly, even though we show large employment impacts for participant youth, we cannot rule out the possibility that the YJC and OJT components did not increase the total number of jobs in the labour market. A displacement effect could have occurred whereby employers would have hired more people from elsewhere in the absence of the program and as a result total employment could have remained unchanged. Few studies have been designed in a way allows them to be able to rigorously test whether these ‘general equilibrium’ effects exist (Groh et al, 2016). 19 j) Work placements and job matches are important to be seek further education than controls. The 2015 to employers follow-up survey showed that, among those jobless youth who had participated in UYEP, the number seeking paid Most employers (57%) reported that they offered employment in the last three months increased by more ongoing jobs to OJT participants due to a positive than ten percentage points (from 21% at the start of the internship performance, as opposed to any need for program, to 32% by the follow-up survey). OJT also appears additional staff (37%). Employers also reported that UYEP to encourage participants to seek a job in the formal sector, is the third most common way employers recruit new staff, increases their confidence about finding a job in the formal after newspaper advertisements and job agencies. Feedback sector and motivates them to seek further education. Results from the survey shows that employers use the OJT placements from the 2017 follow-up survey indicate that OJT youth are as a low-cost way to screen youth prior to deciding whether far more likely to report seeking a job in the formal sector to offer them a job. over the last three months (a statistically significant increase of 19 percentage points) compared to youth in the control group (who reported no significant change). 2.2 Employment prospects and b) Increased optimism about employment labour market engagement UYEP may increase participants’ optimism about future Generally, international evidence shows that in addition employment. The increased job search activity among to an increase in reported employment rates, participation in unemployed graduates reflects their own belief that the skills an ALMP may lead to greater engagement with the labour attained in the program make them more employable. Prior market, as well as an increase in the confidence and wellbeing to participating in the program, only 22% of YJC youth felt of participating youth. UYEP participants were surveyed optimistic about finding work in the next six months while specifically about their job search behaviour, employment after participating in UYEP this increased to almost 37%. expectations and education plans, post-UYEP. A greater level Similarly, while around half of the participants felt optimistic of job searching, and subjective well-being might suggest to be working by 35 years of age pre-YJC, three-quarters felt that youth are more confident about their chances of finding confident they would be working by age 35 years because work. of participating in the program. For OJT youth, prior to participating in the program, only one quarter felt optimistic a) Increase in job search behaviour and seeking further about finding work in the next six months (26%). This education increased to 53% (with most of this increase being driven by Youth who participate in UYEP are more likely to be the program).31 searching for work than youth who did not participate in the program, with OJT youth also being more likely FUS 2017, Optimism about future Employment 90% 80% 80% 74% 70% 60% 55% 57% 53% 50% 40% 37% 30% 26% 22% 20% 10% 0% YJC Pre-UYEP YJC Post-UYEP OJT Pre-UYEP OJT Post-UYEP Figure 2-b: Youth involved in informal work in the past month 31 The causal impact of OJT is estimated to be 18.3 percentage points, significant at a p-value below 0.01. Like the case for seeking a job, the effect is largely driven by youth in the treatment group who had not worked in the formal sector for at least 6 months at the time of the follow up survey. Among these youths, the effect of OJT is a 24.2 percentage point change in the share of participants who believed they would work in the formal sector within six months. 20 c) Access to formal financing but with limited sustainability Although the provision of bank accounts in UYEP is a by-product of participating in the project, participant youth are more likely than control youth to have bank accounts. For young people, having a bank account may be an advantage when it comes to future employment, as some employers prefer their workers to already have their own bank accounts. While both treatment and control youth started out with similar levels of bank account ownership (on average almost three-quarters of participant youth reported to have never had a bank account prior to joining the project), results from the 2015 follow-up survey reveal that nearly three quarters (71%) of treatments had retained a bank account, while less than one quarter (23%) of control youth had a bank account. During the program, access to bank accounts was almost universal among participating youth, though some closed accounts because of the account fees. d) Some connections to increased educational opportunities Large proportions of UYEP participants reported that they want to pursue education in the next six months (46% for YJC youth and 50% of OJT youth, respectively). OJT hospitality trainees serving the Prime Minister’s table at fundraising event A further four percent of YJC youth and six percent of OJT While there are some examples of Latin American ALMPs youth indicated having already enrolled in education at the that combine vocational training with subsidised work time of the 2017 follow-up study. There is also evidence that placements, in these contexts, the positive effects on formal OJT participants are more likely to seek formal educational employment have come at the expense of informal than control youth. Pre-UYEP, the majority (60%) of employment.34 To date, the most successful ALMP has been participants expressed interest in further education in the a Ugandan project which combined vocational training or baseline survey and this fell to 27% for control youth and subsidised work placements with job matching, resulting in 47% for treatment youth. The effect of OJT is therefore a an employment benefit of between seven to 11 percentage 21.7 percentage point increase relative to control youth, points, which lasted four years after the program.35 which is highly significant. UYEP is a rare example of a comprehensive ALMP in a fragile developing state, that takes disadvantaged youth through 2.3 UYEP in the Context of vocational training, job matching and fully subsidised work placements. The evidence shows that UYEP substantially Active Labour Market increases employment rates by 11-18 percentage points. While perhaps not directly comparable given the different in Programs Elsewhere country context, economic structure, and time frame of the Active labour market programs (ALMPs) aim to increase impact evaluation, the UYEP is four to seven percentage the employment, employability and income of participants. points higher than the most successful ALMP to date (in Most of the evidence on ALMPs suggests that they struggle Uganda).36 Since UYEP is the first rigorously evaluated labour to substantially increase employment in nearly all contexts, market program in PNG, evidence of such substantial including those with limited formal sector opportunities and employment gains would be particularly noteworthy for limited youth capabilities. Most subsidised employment policy-makers in PNG addressing the economic and social programs (about 60% according to Kluve et al.),32 including marginalization of unemployed youth. The results could comprehensive ALMPS, have had no positive employment also significantly contribute to the international research on effects.33 ALMPs, more broadly. Kluve et al, 2016. 32 McKenzie, 2017. 33 Aluza et al., 2016; Attanasio et al., 2015; Dias and Rosas, 2016). 34 Alfonsi et al, 2017. 35 21 b) Perceived social impacts and perceived impact on crime 2.4 Secondary impacts There is a general perception among community In addition to employment impacts, recent studies of ALMPs members that UYEP reduces crime in their communities. in contexts where unemployment and crime are both high Of respondents in the community survey, 85% reported have also examined other, non-pecuniary impacts for young a reduction in crime/antisocial behaviour. 77% of participants, such as a reduction in risky behaviour and respondents also reported that UYEP youth they know criminal activity. To date however, little evidence exists are less likely to be involved in crime. The second most regarding the impacts of ALMPs on social indicators such as widely reported positive impact is positive character crime, especially in fragile contexts.37 In this regard, the UYEP and behavioural changes in participant youth (increasing impact evaluation sought to examine the following:38 respect, responsibility, attitudes toward women etc.), which a) Some positive social impacts of UYEP was reported by 82% of respondents. However, 29% of men and 36% women also reported negative impacts UYEP has strong and lasting effects on participants’ of UYEP. The most concerning negative impacts include social behaviour, by encouraging them to interact with “domestic violence increase” and “too much public different people. While more than two-thirds (36%) of disturbance by male youth”. This implication that some participants reported having friends involved in fights or types of anti-social behaviour may become more likely robbery in the past three months prior to the program, by because of UYEP requires further investigation to better the time of the 2015 follow-up study this had decreased to understand the nature of the results. under one-quarter (23%), while for control youth the number increased from 35% to 39% over the same period. c) Attitudinal changes on gender and increase in This finding could either result from both participants’ direct confidence among women friendships made through the program itself, or it could be By involving women in a range of different work tasks, the indirect result of higher aspirations due to exposure to including roles traditionally dominated by men, and work. Consistent with this finding is the fact that UYEP through the inclusion of gender-equality training in participation leads to a strong increase in the probability the ‘Basic Life Skills Training’, UYEP has improved the of employment, which both raises the opportunity cost of social status and the economic opportunities of its anti-social behaviour and may alter participants’ shared female participants. Based on the 2017 follow-up survey, sense of mission with gangs that interact at night and the percentage of female participants who believed that engage in criminal activity. women should be at home and not in waged work, fell from Participants are also less likely to want to threaten 13% before the program to two percent after the program others after participating in the program. The number (and fell from 16% to nine percent for men.) A Gender and of participants who reported using threat or force with Labour Study for the UYEP  40 confirms that in most aspects somebody halved (from 16% to 7%) because of participating of training, women have shown a higher propensity to in UYEP, while for control youth, the number increased over complete the training offered, which suggests that the the same period. project has been successful at engaging young women. Evidence of the program’s impact on participants’ Apart from skills development and work experience, engagement in or exposure to other types of more receiving a certificate and reference from UYEP were major serious crime, such as assault, robberies and trespassing, reported factors facilitating the search for employment is inconclusive. One hypothesis is that the program has had among young women. However, the study also reveals that weaker effects on the type of youth that are likely to engage women encountered several challenges with respect to in crime, and more investigation is necessary to better completing the training, which resulted in them dropping understand the relative importance of economic and social out of the project, including gender-based and domestic factors on this type of criminal activity.39 violence, difficulty accessing safe transport, cultural 36 Such as Alfonsi et al., 2017, which can be used as a benchmark. 37 Ralston and Blattman, 2015. 38 Research for this section involved conducting additional focus group discussions to understand and verify the findings of the impact evaluation. See Voigt-Graf, 2018. 39 Ivaschenko et al. 2017 40 Voight-Graf, C. 2018. 22 expectations and discrimination, among others.41 To mitigate own husbands. Receiving a stipend and becoming financially gender disparities, UYEP I promoted gender equity in its more independent and able to contribute to the household communications strategy, to reassure families about the budget has raised the status of women at home, and some nature of the training and work women were being offered, women report feeling more empowered to “overcome it sensitized trainees, contractors and PMU personnel difficult issues”. They also reported a greater sense of through training, and arranged for contractors to provide awareness of gender issues and of their rights, leading to transport to ensure safe passage in some areas. UYEP also an increase in their status and sense of security. provided all participants with care responsibilities additional days to complete the training and work activities or offers to % that agree with the statement make-up times. Incorporating an electronic banking component proved to be popular among women, to help 0.2 protect their income and savings.42 In addition, through 0.18 its Gender Committee, the project developed a Sexual 0.16 Harassment and GBV Policy and Maternity Leave Statement 0.14 in 201643 in accordance with the NCDC’s Family and Sexual 0.12 Violence Action Committee (FSVAC) Strategy 2016–2018, 0.1 and the PNG National Strategy to Prevent and Respond to 0.08 Gender-Based Violence 2016-2025. UYEP is using NCDC’s 0.06 Gender and FSVAC Desks to refer and provide transport for 0.04 GBV survivors. 0.02 0 The Gender and Labour Study also found that, prior to 15% Men Men Women Women participating in the program, few participants had any prior Pre-UYEP Pre-UYEP Post-UYEP Post-UYEP knowledge of gender disparities between men and women. Women stressed that due to UYEP training, they had gained Figure 2-e: Percentage of those who agree with the statement more confidence when dealing with men, including their that “Women should stay at home to do housework and should not get a wage job.” Community leaders from Motu Koita villages mapping out the consultation arrangements 41 Other key challenges faced by females related to their vulnerability to becoming victims of crime, difficulties accessing market opportunities, financial exclusion, legal factors (such as inheritance and ownership laws), household and family obligations, not finding a babysitter and pregnancy. For others, unsupportive husbands were a major obstacle. Previous studies reported that some female participants had to deal with aggressive resistance from husbands towards their participation in UYEP. Focus group participants also talked about incidents of jealous husbands turning up at worksites and forcing women to withdraw. Interviews for this study suggest that there are considerable costs for providing 42 According to the end March 2018 Quarterly Progress report, 41% of women had never had a bank account prior to the UYEP. 43 The Policy establishes a comprehensive set of workplace strategies to address the impact of GBV in the workplace and guide UYEP responses to employees and trainees whose work life is affected by GBV. The Gender Committee is responsible for implementing and monitoring the GBV Policy and UYEP’s response protocol. It also has responsibility for GBV focal points which provide support and referral. According to UYEP’s Maternity Leave Statement for Youth Trainees of April 2016, women trainees will be entitled to unpaid maternity leave in line with the Public Services General Order. Women will not lose their place in the YJC, PET and OJT when they withdraw due to pregnancy. While the Statement is not yet being implemented, UYEP I currently deals with the issue of pregnant women by advising them to start UYEP later while their places are guaranteed. 23 Participating in UYEP has also changed how men d) Impact of the OJT program on subjective well-being and women relate to each other. During focus group The OJT program appears to increase the sense of discussions,44 most of the UYEP participants commented well-being of its participants. At the time of the baseline that male-female relations during training and at work survey and prior to partaking in OJT, around half of the had improved, a development which they supported. Both participants reported “feeling happy most of the time over men and women said that they had benefitted from the the past four weeks”. By the 2017 follow-up survey, the inclusion of gender as a topic in the BLST curriculum. During number of youth who felt “happy most of the time over the the early training days, many male and female participants past four weeks” had increased by 16 percentage points felt shy with each other, but there are reports of this for participants, while it decreased by 22 percentage gradually reducing as participants mixed with each other points for control youth. This represents a difference of socially. Trainers and participants agree that UYEP had also 38 percentage points, which is statistically significant. led to changes in male participants’ attitudes, who generally have become more respectful in their dealings with women. These results are somewhat in contrast with the reported results obtained through the community survey and may therefore require further investigation. Voigt-Graf, 2018. 44 24 is PET trainee in metal workshop at Port Moresby Technical College 25 PET business and computer training at Port Moresby Business College 3. Conclusions and Recommendationsdations III. Lastly, the program has provided links to formal employers who are active in the labour market and “real” work, 3.1 Five key conclusions increasing the connection between youth and the labour (i) There is a substantial increase in UYEP participants’ market. For OJT youth, the subsidised job placements employment compared to control youth, both for men help provide a low-cost option to screen candidates and and women. YJC youth report employment gains of terminate poorly performing youth. 11 percentage points while OJT youth report gains of (iii) For the youth that are not working, UYEP leads 18 percentage points - which is significantly higher than any to improved labour market engagement, such as other comparable ALMP to date. These impacts are sizeable greater job searching, and increased confidence in considering that none of the participants had any formal securing future employment. UYEP participants are more employment at baseline. Most community members also optimistic about future employment prospects and are more believe that UYEP has increased youth employment likely to be actively searching for work. They are also more opportunities. Furthermore, for OJT youth, the evidence is likely to want to enrol in education and study. These are all also supported by reports from UYEP employers, who verify signs that UYEP has been motivating, encouraging and offering ongoing employment to a significant percentage of building confidence in its young participants. UYEP youth beyond their OJT placement. OJT participants report being more likely (by 28 percentage points) to have (i) UYEP provides the marginalized, urban youth had work in the six months prior to the 2017 follow-up of PNG with much more than just work. There is survey than control youth. evidence that the program leads to improved well-being and potentially healthier social behaviour for (ii) UYEP promotes employability of participants participants. This is echoed by the community members in through a combination of the three components of project areas, who report that UYEP reduces crime and leads ALMPs (training, wage subsidisation, and matching to improved behaviour of the youth who participate, through work placements). The range of mechanisms consistent with the broad public support for UYEP. by which UYEP increases employability include: (ii) UYEP has shown to empower women and help I. Firstly, through increased labour market engagement improve attitudes to gender equality. Through its (e.g. job search) of out-of-work youth, which in turn, inclusion of women in a range of work and training increases their employment prospects. For many youth, opportunities, and through its gender training, UYEP has UYEP is also their first source of formal employment, improved the status and economic opportunities of its removing any barriers that they (or employers) see female participants. between themselves and the labour market. II. Secondly, the program increases the human capital among its participants (albeit to varying degrees depending on the training received and amount of work undertaken) and increases their skill and capability to perform well in the jobs they are hired for. 26 data and evidence is needed to capture the value and effects of them exclusively. There is currently limited data 3.2 Areas for further research available to assess whether participants are learning work and evaluation relevant skills that are transferrable (and what those skills are), or to determine what proportion of youth are proficient Several questions remain for the future research and in what they have been taught. There is also inadequate evaluation of UYEP and its impacts: Firstly, there is strong data to separate and compare the impact of UYEP training evidence of employment gains and positive social from the impact of the work experience and wage impacts in the short to medium term, however the subsidy components. While the OJT employers are a source longevity of these impacts has not been determined. of information about the fitness of OJT participants for The follow-up survey gathers data for a period between a employment in the areas they have received training in, there few months after project participation, to no more than two is no such equivalent information for YJC youth to assess the years after project participation. This means that the project’s incentives and value of the program with the City Authority short to medium term impacts and trends need to be verified and/or to civil works contractors. over the longer term, for example, to three to five years after project participation. This could be achieved by conducting More labour market data is also required to understand a third follow-up-survey of participants who have already the PNG labour force more generally. The project needs completed either the 2015 follow-up survey or the 2017 to explore how well the labour market is functioning. It is follow-up-survey. Alternatively, the future project based on beyond the scope of this impact evaluation to determine if UYEP could design an impact evaluation with follow-up- UYEP creates new jobs or fills jobs that would have existed surveys planned for appropriate times. anyway with disadvantaged youth. While answering this question is not easy, the lack of data on the labour market of Secondly, the causal impacts of individual project PNG makes this virtually impossible. There is currently no interventions are not well understood and more robust or up-to-date data on levels of youth unemployment in PNG – one of the most basic labour market statistics, and it is sorely needed, not only to facilitate a greater understanding of UYEP’s impact, but also to inform education and training policy in PNG more broadly. UYEP participants report feeling empowered by their training and optimistic about the future, however these long-term impacts are not easy to quantify. As such, the project may benefit from more in-depth qualitative data on participants’ personal journeys from unemployment, through training, experience and long-term employment. More research on the project’s impacts on crime and urban safety would also be valuable, to better understand the nature of the results as well as some inconsistencies in findings. Finally, the evidence and lessons described in this paper should be used as a guide for the design of the Program Evaluation of the follow-on project, UYEP II. The next evaluation could be designed to have annual participant (treatment) and non-participant (control) groups and/or variations in project participation to enable estimation of component effects. It could also involve a longer-term follow-up survey (a second survey for participants one or two years after they complete the first follow-up survey, and three to five years after concluding participation in OJT or YJC). 27 4. References Alfonsi, Livia, Bandiera, Oriana, Bassi, Vittorio, Burgess, Robin, Imbun, Ben (2015), Supporting Demand Led Vocational Rasul, Imran, Sulaiman, Munshi and Vitai, Anna (2017) Skills Development in Papua New Guinea: A Labour “Tackling Youth Unemployment: Evidence from a Labor Market Approach, Study prepared for Human Resources Market Experiment in Uganda”, STICERD - Development Programme Phase 2 (HRDP2), December 2015, Development Economics Paper Waigani Alzúa, María Laura, Guillermo Cruces and Carolina Lopez Ivaschenko, Oleksiy, Naidoo, Darian, Newhouse, David and (2016) “Long run effects of youth training programs: Sultan, Sonya (2017) “Can public works programs reduce Experimental Evidence from Argentina”, Economic Inquiry youth crime? Evidence from Papua New Guinea’s Urban 54(4): 1839-59 Youth Employment Project”, IZA Journal of Development and Migration (2017) 7:9 Attanasio, Orazio Arlen Guarín, Carlos Medina and Costas Meghir (2015) “Long term impacts of vouchers for Jones, L.T. and P.A, McGavin (2015), Grappling afresh with vocational training: Experimental evidence for Colombia”, labour resource challenges in Papua New Guinea – A NBER working paper no. 21390. Framework for Moving Forward, Institute of National Affairs Discussion Paper No. 96, Port Moresby: Institute of National Beam, Emily, David McKenzie and Dean Yang (2016) Affairs. “Unilateral Facilitation Does Not Raise International Labor Migration from the Philippines”, Economic Development Kelly, Tom (2016) “Middle East and North Africa jobs crisis and Cultural Change, 64(2): 323-68 ‘may import terror into Europe’”, Daily Mail, 26 May, http://www.dailymail.co.uk/news/article-3611803/Middle- Blattman, Christopher and Laura Ralston (2015) “Generating East-North-Africa-jobs-crisis-import-terror-Europe.html employment in poor and fragile states: Evidence from labor market and entrepreneurship programs”, Mimeo. Chicago Levinsohn, James and Todd Pugatch (2014) “Prospective Analysis of a Wage Subsidy for Cape Town Youth”, Journal of Bank of Papua New Guinea (BPNG) (2018), December 2016 Development Economics 108:169-183 Quarterly Economic Bulletin, Port Moresby: BPNG, available at: https://www.bankpng.gov.pg/statistics/quarterly-econom- Pyke, Jonathan (2016), “Creating Jobs in Papua New Guinea: ic-bulletin-statistical-tables/ The Role of Young Entrepreneurs”, The Lowy Institute, available at https://www.lowyinstitute.org/the-interpreter/cre- Diaz, Juan Jose and David Rosas (2016) “Impact Evaluation of ating-jobs-papua-new-guinea-role-young-entrepreneurs the Job Youth Training Program ProJoven”, IADB Working Paper no. 693. McKenzie, David (2017) “How Effective Are Active Labor Market Policies in Developing Countries? A Critical Review of Galasso, Emanuela, Martin Ravallion and Augustin Salvia Recent Evidence” World Bank Research Observer, World Bank (2004) “Assisting the Transition from Workfare to Work: A Group, vol. 32(2), pages 127-154 Randomized Experiment”, Industrial and Labor Relations Review 57(5): 128-42 National Statistical Office (2013), Census 2011: Final Figures Papua New Guinea, Port Moresby: National Statistical Office. Groh, Matthew, Nandini Krishnan, David McKenzie and Tara Vishwanath (2016a) “Do wage subsidies provide a stepping United Nations Population Fund (UNFPA) (2018), Data, stone to employment for recent college graduates? Evidence available at: https://www.unfpa.org/data from a Randomized Experiment in Jordan” Review of Voigt-Graf, Carmen (2017) “Labour Market in PNG Economics and Statistics 98(3): 488-502 Background paper for Systematic Country Diagnostic (SCD) Hirshleifer, Sarojini, David McKenzie, Rita Almeida and for PNG” World Bank Report Cristobal Ridao Cano (2016) “The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey”, Economic Journal 126: 2115-2146 28 29 APPENDIX - A OJT Impact Evaluation Methodology To determine the impact of the OJT component of this the time the intake started, and they were only informed program (beyond just the YJC component) we conduct of this change part way through the YJC component. a difference in difference analysis and exploit a one off, Finally, statistical balance calculations show that there were unexpected change in the implementation of the program. almost no statistically significant differences between the Specifically, during intake 13 there was an unanticipated demographic characteristics of groups at the time they intervention in the program and the standard OJT component participated in the screening interviews. Table i) below shows was removed part way through the YJC component. Instead the only statistically significant differences between these participants were offered the opportunity to be paid to be a groups is those in the treatment group were slightly volunteer at the 2016 Under 23 Women’s FIFA World Cup. older than those in the ‘placebo’ control group at the This volunteer placement involved basic manual labour and time of the baseline survey. there was no potential for ongoing employment with the employer as this was a one-off event. As such program participants in intake 13 who were eligible to complete OJT because of their academic performance in the screening test and satisfactory completion of BLST did not complete OJT. Following intake 13, the usual program resumed whereby people who were eligible due to their academic performance completed OJT as was the case in earlier intakes. The youth who were eligible for OJT in intake 13 but missed out due to the temporary change to the program (‘placebo control group’) are comparable to the youth in the intake just before and just after (12 and 14) who completed OJT (‘treatment group’) for at least five reasons (Figure 1). Firstly, all youth in each of the three intakes had not worked in the formal sector in the last 6 months and on average between 75-80% of the youth had never worked in the formal sector throughout their life. Secondly, intakes 12, 13 and 14 occurred at a relative similar time (the screening interviews occurred in April, July and November 2015 respectively) and the timeframe when OJT took place is similar across intakes. Thirdly, the youth are sourced from the same areas in Port Moresby and expressed interest in the program through the same channels. Fourthly, the removal of OJT from the program during intake 13 was not known by participants at 30 Figure A-1 – Process of selecting treatment and ‘placebo’ control groups Intake 12 Completed OJT (52% of Youth in Port Moresby who (April 2015) ‘Treatment’ group) has been outside the formal labour market for atleast 6 months, who applied to UYEP Eligible but Did not between April to November Intake 13 Complete OJT 2015, were deemed eligible, (July 2015) (‘Placebo control’ group) performed highly on an academic test and satisfactorily completed BLST Intake 14 Completed OJT (48% of (November 2015) ‘Treatment’ group) Box A-1: Annual data collection cycle 1. Communication and awareness building campaign in selected districts inviting youth that meet the described eligibility criteria to apply. This process usually takes 2‐4 days. 2. Interested applicants show up at a screening site on a given date to undertake the Eligibility Screening Survey (ESS). Intakes occur on a rolling basis every 2-3 months. In early rounds the ESS was administered by an independent survey firm, and since mid-2016, administered by the PMU. The ESS collects basic information, including the socio‐economic characteristics of youth. 3. The ESS data is processed by the PMU to establish eligibility. To balance the interests of competing groups, each neighbourhood group is allocated a certain number of places, with a target of 50% women in each location. Participants are selected randomly from the pool of eligible applicants in rounds when the pool of applicants exceeded available places in the projects. This randomisation is conducted separately by neighbourhood group and gender. Eligible applicants are given the Eligibility Baseline Survey (EBS) that collects a richer set of information compared to ESS. This includes some eligible applicants who will not be admitted to the program (controls). 4. During the BLST training, each BLST participant takes a literacy and numeracy test, which is used to determine the quarter of each intake that is then streamed into the OJT component. Usually, more than a quarter of the participants are literate, so only those with top scores and strong participation in the BLST may proceed to the next stage of the OJT component. 5. Participants are enrolled in either Component 1 (YJC) or Component 2 (OJT) of the Project. 6. All work days are tracked through the project management information system (MIS) that records the stipend payments for each individual youth. As youths need to have their work participation confirmed by their manager, the stipend payment record should be an accurate reflection of the level of participation in the project. The MIS also allows the tracking of individual youth across project components, including when a youth participates in two different OJT placements, or both the YJC and OJT programs. 7. For selected intakes up to once a year, a follow up survey (FUS) is administered to a sample of youth 6 to 12 months after participation in the program. Where possible, a matching control group that was interviewed with the baseline survey at the same time as the participants, is also interviewed with the FUS 8. In the second half of the year, and Employer survey is conducted for employers of youth in the OJT program, who participated in the program in the past 12 months. Similarly, a survey of community members of project communities from the past 12 months is conducted. 31 APPENDIX - B Data collection - Further information Limitations of the Screening and Baseline Surveys: as the youth were not participants. Based on an assumption In later rounds, the strategy for recruiting controls changed of equal trends in the absence of the project, we deduce that due to operational difficulties with maintaining a pure the real impact of the project can be estimated to be 4% control group. This involved offering controls the opportunity (that is 10%-6%). This method of measuring impacts is to re-apply to the program 6 months later, with the promise known as difference-in-difference estimation and is the basis of an increased chance of being accepted then. The project for the causal estimates in the next section of the paper. had mixed success in maintaining these youth as controls Ivaschenko et al (2017) provide a more detailed explanation as there was nothing to stop them from applying to the of the “equal trends” assumption and the difference in program earlier or later than 6 months, and there were difference approach to estimating program effects. problems identifying and keeping track of youth when It is important to consider the treatment and controls groups they returned to the program if they did so. This was a major in the two FUS, to understand the basis of causal estimates, limitation, of the baseline after rounds 4 and 5- the difficulty described in box 3. For both the 2015 and 2017 FUS survey in obtaining controls that could be re-interviewed. samples, the analysis of the treatment and control groups Limitations of the Follow Up Surveys: The sample demonstrates that they are comparable at the time of the for both the 2015 and 2017 FUS were relatively small, baseline survey. This is presented in Appendix C. Given that meaning that only effects of a substantial size are likely to be these groups are similar enough, we can assume that their detected. The control group from 2015 was from a different outcomes at the time of the FUS would have been similar in location to the participant youth. Also, the question asked of the absence of the project. That is, whatever trends would controls and participants regarding employment in the occur in the outcomes of interest for both groups (for past 6 months was not comparable. The control group in example employment and involvement in crime), would the 2017 FUS could only be compared to OJT youth, not YJC move in the same direction over time. Lastly, these estimates youth. of causal impacts are then supplemented by data on employer and community perceptions. For example, let us consider the case of employment outcomes, which in addition to being impacted by the Characteristics of UYEP youth in the Impact Evaluation project, could also be impacted by changing labour market Samples for 2015 and 2017 conditions. Suppose that we observe that no treatment Before considering the impacts of the project, we briefly youth are employed at the time of the baseline, but consider the baseline characteristics of the samples of youth 10% of youth are employed at the time of the FUS. used in the impact evaluations, for the 2015 and 2017 FUS How do we know that it was the project that lead to this sample, for which there is comparable baseline and FUS increase in employment? Some of youth may have found data. This gives us an idea of the comparability of the jobs regardless of involvement in the project. We then check participant and control youth in the evaluations. In addition the trends for a similar group of youth (controls). to sample comparability, using the FUS 2017 data, we We observe that the controls had no employment at time of investigate the extent to which UYEP has been targeting the baseline and 6% had employment at the time the FUS. Thus disadvantaged. 6% of employment is due to something else-perhaps a buoyant labour market but certainly not project participation 32 2015 FUS sample throughout their life. Secondly, intakes 12, 13 and 14 occurred at a relative similar time (the screening interviews occurred in It is useful to compare the baseline characteristics of the April, July and November 2015 respectively) and the re-interviewed treatment and re‐interviewed control groups, timeframe when OJT took place is similar across intakes. since these are the two groups that are used to estimate the Thirdly, the youth are sourced from the same areas in Port effects of the program. The full results of this analysis for Moresby and expressed interest in the program through the the 2015 report are reported in Table i) in the appendix same channels. Fourthly, the removal of OJT from the (taken from Ivaschenko et al, 2017). In terms of individual program during intake 13 was not known by participants at characteristics, the treatment group is on average 1.67 years the time the intake started, and they were only informed of older, has 0.75 extra years of education, and is less likely to this change part way through the YJC component. report having a formal job in the past. These differences are significant at a 1% level. The treatment group youth also Due to the need to ensure comparability, the sample size for seem to be coming from somewhat larger (by about one treatment and placebo control groups included in both the person) households. We do not find significant differences in baseline and follow up surveys is quite low (293 in total). This other characteristics, including gender, marital status and means that for the 2017 impact evaluation sample, even propensity to have earnings from informal jobs. The treatment more so than the for 2015, it is only possible to detect effects group is somewhat more likely to look for a paid job over the with a large effect size. Attrition was around 38% in the past month, with a difference in this indicator between groups treatment group and the only major difference was that being significant at a 5% level. Overall, the differences in youth who were re-interviewed were more likely to have baseline characteristics due to attrition appear to be lower levels of education at baseline. To put the sample size relatively minor. This implies that the groups are likely to be into perspective, it is lower than any study in a 2017 World appropriate for comparison to determine program impacts. Bank literature review on ALMPs in developing countries (the One limiting factor is the sample size of both groups, which lowest is 407 and average is around 3500) (McKenzie, 2017). is relatively low, meaning only effects with a substantial In addition, the sample size in each of the groups is not equal effect size can will be detected. Further information on (191 in control and 102 in treatment), which means that sample balance can be found in Ivaschenko et al (2017). there is even less chance to detect a statistically significant effect. As such the statistically significant effects reported 2017 FUS sample in this study are due to the large effect size as opposed to Statistical balance calculations for the 2017 FUS sample having a large sample. show that there were almost no statistically significant differences between the demographic characteristics of treatment and control youth at baseline. Table i) and Table ii) shows the only statistically significant differences between these groups is those in the treatment group were slightly older than those in the ‘placebo’ control group at the time of the baseline survey. Furthermore, the treatment and control groups are comparable for four other reasons. Firstly, all youth in each of the three intakes had not worked in the formal sector in the last 6 months and on average between 75-80% of the youth had never worked in the formal sector 33 Box B-1: Treatments and Controls in the FUS The 2015 FUS- Following the participation of the 4th and 5th intakes in the Project, a follow‐up survey (FUS) was conducted on the samples of “treatment” and “control” groups in mid-2015 (about a year to a year and a half after these intakes participated in the Project). The treatment group consisted of 338 re-interviewed participants out of a possible 743 participants. This included a mix of youth that participated in YJC and those that participated in OJT. The control group consisted of 250 re-interviewed non-participants, out of a possible 569 non-participants. Analysis of the baseline data for treatments and controls establishes that the two groups are very similar (reported in the next section of this paper). More detailed information can be found in Ivaschenko et al (2017) and the appendix. The 2017 FUS- During intake 13 there was an unanticipated intervention in the program and the standard OJT component was removed part way through the YJC component. Instead participants were offered the opportunity to be paid to be a volunteer at the 2016 Under 23 Women’s FIFA World Cup. This volunteer placement involved basic manual labour and there was no potential for ongoing employment with the employer as this was a one-off event. They also did not receive pre-employment training. As such program participants in intake 13 who were eligible to complete OJT because of their academic performance in the screening test and satisfactory completion of BLST did not complete OJT. These youth are used to construct a ‘placebo control group’. That is, they are youth that may benefit from being involved in the project, but not from its full suite of training and employment services. These youth can be compared to the youth in the intakes just before and after (12 and 14) who completed PET and OJT. This second group is the ‘treatment group’. By comparing the placebo control group and the treatment group, the impact of OJT can be estimated. (beyond youth simply being involved in the project). 34 Table B-1: Balance between treatments that were re-interviewed and controls that were re-interviewed in 2015 FUS. Re-interviewed Re-interviewed Difference Variable of interest Controls (mean/sd) Treatments (mean/sd) (mean/se) Individual’s basic characteristics: 1 2 3 Age 23.45 25.11 -1.66 (4.81) (4.77) (0.40) Male (1 if male, 0 if female) 0.65 0.62 0.04 (0.48) (0.49) (0.04) Married 0.39 0.39 -0.01 (0.49) (0.49) (0.04) Years (grades) of education completed 8.44 9.18 -0.74 (3.06) (2.81) (0.25) Individual’s household characteristics: Household size 8.13 9.34 -1.21 (4.06) (4.53) 0.3615181 N of females in a household 3.66 4.18 -0.52 (2.23) (2.63) (0.21) N of males (16+) in paid work 1.06 1.15 -0.10 (1.02) (1.07) (0.09) Dwelling has floors made of concrete 0.04 0.06 0.02 (0.19) (0.23) (0.02) Dwelling has floors made of wood 0.93 0.91 0.01 (0.26) (0.28) (0.02) Individual’s employment characteristics: Never had a wage (formal) job 0.64 0.78 -0.14 (0.48) (0.42) (0.04) If earned money through informal job/self-employment last month 0.32 0.39 -0.06 (0.47) (0.49) (0.04) If searched for a paid job last month 0.25 0.33 -0.09 (0.43) (0.47) (0.04) Number of observations 250 338 Source: Ivaschenko et al (p16, 2017) 35 Table B-2: Statistical balance across treatment and control youth for OJT impact evaluation for background characteristics (1) (2) t-test Control Treatment (1)-(2) Variable N Mean/SE N Mean/SE Difference Age 191 24.361 102 25.706 -1.345** [0.341] [0.444] Share that 191 0.592 102 0.647 -0.055 are male [0.036] [0.048] Share that 191 0.366 102 0.412 -0.045 are married [0.035] [0.049] Household 191 8.607 102 8.392 0.215 members [0.317] [0.448] Female 191 3.916 102 3.892 0.024 household [0.164] [0.265] members Male working 191 1.120 102 1.029 0.091 household [0.082] [0.115] members Share with 191 0.084 102 0.078 0.005 concrete floor [0.020] [0.027] Share with 191 0.859 102 0.882 -0.024 wood floor [0.025] [0.032] The value displayed for t-tests are the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical level. 36 Table B-3: Statistical balance across treatment and control youth for OJT impact evaluation, employment, job seeking and education histories (1) (2) t-test Control Treatment (1)-(2) Variable N Mean/SE N Mean/SE Difference EMPLOYMENT Share that 191 0.770 102 0.794 -0.024 have never [0.031] [0.040] worked in formal sector Number of 191 0.340 102 0.343 -0.003 formal sector [0.044] [0.065] jobs in lifetime Earn any 191 0.450 102 0.510 -0.060 money over [0.036] [0.050] last 6 months JOB SEEKING Sought after a 191 0.168 102 0.147 0.020 formal sector [0.027] [0.035] job in last 3 months Sought after 191 0.152 102 0.225 -0.074 any job last [0.026] [0.042] month Sought after 191 0.084 102 0.069 0.084 any job last [0.020] [0.025] [0.020] week EDUCATION Years of 189^ 9.296 102 9.461 -0.164 education [0.153] [0.198] Total score in 191 90.696 73 88.863 1.833 numeracy and [0.325] [1.877] literacy test Intend to 191 0.592 102 0.578 0.013 receive [0.036] [0.049] further education in next 6 months The value displayed for t-tests are the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical level. ^Two people in the placebo control group did not provide an answer to this question `Unfortunately there was a large amount of missing data regarding some student’s test score 37 APPENDIX - C Gender Disaggregated Data Figure C-1 and Figure C-2 report the equivalent percentages for women and men, respectively, as was reported in Figure 2-a. Figure C-1 Figure C-1 % Men Currently employed (All jobs and UYEP % Women Currently employed (All jobs and UYEP related related jobs) ence of Employment Impacts from the jobs) Community Survey 25.0 25.0 21.1 20.4 20.0 20.0 15.8 15.0 15.0 13.3 10.5 10.0 8.8 10.0 7.1 5.0 5.0 4.4 3.4 3.0 . . PET YJCs CTs PET YJCs CTs In the Follow-up Survey for 2015, the treatment group employment in the last 6 months, and we find that only was asked if they had a wage employment after the program 15.1% report to have been employed. This provides some participation. We find that 43.6% of program participants indication that the rate of wage employment in the treatment report having a formal sector job after the program group is substantially higher. Interestingly, 40.8% of employed participation, with 49% and 39.2% for OJT participants and OJT participants and 32.4% of employed YJC participants YJC (public works) participants, respectively. Unfortunately, the report staying with the same employer after graduation from same question could not be asked for the control group. the program. However, the control group was asked about having wage 38 Table B-3: : UYEP participants who were employed after UYEP in the 6 months prior to the FUS 2017: Average hours worked and average wages. YJC PET+OJT Women Men All Women Men All Hours 50.2 44.3 45.9 42.5 50.6 47.4 worked standard 16.2 16.1 16.3 17.4 23.3 21.2 deviation Wages 165.3 190.8 183.9 155.5 163.7 160.5 standard 44.4 74.0 68.0 47.5 70.3 61.5 deviation Observations 24 65 89 11 17 28 Source: FUS 2017 39 APPENDIX - D Estimation of Impacts in the OJT Impact Evaluation Using 2017 FUS Data We test the difference in outcomes between the treatment where Yj is the difference in answers to question j between and ‘placebo’ control groups using an OLS regression of the the baseline and follow up survey. T1 is an indicator variable difference in outcomes measured in the baseline and follow that takes on the value one for members of the treatment up surveys between the two groups. The follow up survey group and zero for members of the ‘placebo’ control group. was conducted 6 to 12 months after the youth had ε is the model error term. Our parameter of interest is the graduated from UYEP (in July 2017). We create a dummy OLS coefficient B1j , which captures differences in differences in variable for the treatment group, which takes on the value the response to question j between treatment and ‘placebo’ one if the respondent belongs to the treatment group and control groups over time (between the baseline and follow the value zero if the respondent belongs to the ‘placebo’ up surveys). In the case whereby Yj is a continuous variable, control group. We estimate the following OLS regression: B1j shows the difference between the groups overtime in terms of the units the variable is measured in. In the case Yj = B0 j + B 1 j T1 + , whereby Yj is a binary variable, B1j represents the percentage point difference between the groups overtime. These estimates of the OLS regression (known as a linear probability model) were compared to those obtained from a binary logit model and they were shown to be qualitatively similar. 40 PacificIslands@Worldbank.org @WB_AsiaPacific WorldBankPacific worldbank.org/pacificislands worldbank.org/png